Coding and decoding method, device, equipment, storage medium and computer program product

Abstract

The embodiments of the present application disclose a coding and decoding method, apparatus, device, storage medium and computer program, which belong to the field of coding and decoding technology. In the embodiments of the present application, a plurality of feature points are divided into a plurality of groups based on a specified numerical value during the decoding process, and the probability distribution of each feature point in the same group is determined in parallel, which can speed up the decoding efficiency. Correspondingly, during the encoding process, the plurality of feature groups are grouped in the same grouping manner, and the first image features of each feature point in the plurality of groups are encoded into the bitstream in turn. In other words, this solution can break through the efficiency bottleneck brought about by serial calculation when decoding based on variational autoencoder VAE, and effectively improve the decoding efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of coding and decoding technology, and in particular to a coding and decoding method, apparatus, device, storage medium, and computer program. Background Art

[0002] Image compression technology enables the efficient transmission and storage of image information, playing a crucial role in the current media age, where the variety and volume of image information are increasing. Image compression involves both encoding and decoding, with encoding and decoding performance (reflecting image quality) and encoding and decoding efficiency (reflecting processing time) being key considerations.

[0003] In related technologies, after long-term research and optimization, lossy image compression standards such as JPEG and PNG have been established. However, these more traditional image compression technologies have encountered bottlenecks in improving encoding and decoding performance, and are no longer able to meet the needs of the ever-increasing amount of multimedia application data. With the widespread application of deep learning technology in fields such as image recognition and object detection, deep learning has also been applied to image compression tasks, achieving higher encoding and decoding efficiency than traditional image compression technologies. For example, using a variational auto-encoder (VAE) based on deep learning technology for image encoding and decoding can significantly improve encoding and decoding performance.

[0004] However, in the research of image compression methods based on deep learning technology, how to effectively ensure codec performance while improving codec efficiency is an issue that requires attention and research. For example, in the related art of using VAE for image decoding, a neural network model is used to serially calculate the probability distribution of each feature point in the image, and the image is decoded based on the probability distribution. Since the probability distribution is calculated by the neural network model, serial calculation will reduce the decoding efficiency. How to break through the efficiency bottleneck caused by serial calculation during decoding while ensuring that the codec performance is not reduced is an issue that requires attention in the research of codec methods based on VAE. Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, apparatus, device, storage medium, and computer program that can overcome the efficiency bottleneck caused by serial calculations during VAE decoding without reducing coding and decoding performance. The technical solution is as follows:

[0006] In a first aspect, a decoding method is provided, the method comprising:

[0007] Determine, based on a code stream, a priori features of each feature point among a plurality of feature points of an image to be decoded; divide the plurality of feature points into a plurality of groups based on a specified numerical value; and sequentially determine, based on the priori features of the plurality of feature points, a first image feature of each group of feature points in the plurality of groups; wherein the steps of determining the first image feature of any group of feature points are as follows: determine, in parallel, a probability distribution of each feature point in the any group, and, based on the probability distribution of each feature point in the any group, parse out, from the code stream, a first image feature of each feature point in the any group; and reconstruct an image based on the first image features of the plurality of feature points.

[0008] That is, in this embodiment of the application, during the decoding process, multiple feature points are divided into multiple groups based on specified values, and the probability distribution is determined in parallel for each feature point in the same group, which can accelerate decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial calculations during VAE-based decoding, effectively improving decoding efficiency.

[0009] It should be noted that this method is applied to a codec that includes a context model. When decoding to any one of the multiple groups, the surrounding information of each feature point in any one group has been fully decoded, that is, the feature points in any one group meet the condition that the surrounding information has been decoded.

[0010] Among them, the multiple feature points include a first feature point, and determining the probability distribution of the first feature point includes: if the first feature point is not the first feature point among the multiple feature points, then determining the surrounding information of the first feature point from the first image features of each decoded feature point, where the first feature point is a feature point in any group; inputting the surrounding information of the first feature point into the context model to obtain the context feature of the first feature point output by the context model; and determining the probability distribution of the first feature point based on the prior feature of the first feature point and the context feature of the first feature point.

[0011] Optionally, the surrounding information of the first feature point includes the first image features of decoded feature points within a neighborhood with the first feature point as the geometric center. The size of the neighborhood is determined based on the size of the receptive field used by the context model. The surrounding information includes at least the first image features of n feature points surrounding the first feature point, where n is greater than or equal to 4. In other words, to ensure codec performance and image quality, this solution maximizes the use of as much surrounding information as possible while maintaining compression rate.

[0012] Optionally, the multiple feature points include a first feature point, and determining the probability distribution of the first feature point includes: if the first feature point is the first feature point among the multiple feature points, determining the probability distribution of the first feature point based on a priori features of the first feature point.

[0013] Optionally, the specified value is determined based on a size of a receptive field used by the context model; and dividing the plurality of feature points into a plurality of groups based on the specified value includes: determining a slope based on the specified value, the slope indicating the degree of inclination of a line containing feature points grouped into the same group; and dividing the plurality of feature points into the plurality of groups based on the slope. That is, this solution determines a group of feature points that can be decoded in parallel based on the size of the receptive field.

[0014] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0015] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0016] In a second aspect, an encoding method is provided, the method comprising:

[0017] Based on an image to be encoded, a first image feature, a probability distribution, and a first super-prior feature of each feature point among multiple feature points of the image are determined; the multiple feature points are divided into multiple groups based on specified values; based on the probability distribution of the multiple feature points, the first image feature of each group of feature points in the multiple groups is sequentially encoded into a bitstream; and the first super-prior features of the multiple feature points are encoded into the bitstream.

[0018] That is, in this embodiment of the present application, in order to determine the probability distribution in parallel during the decoding process to improve decoding efficiency, multiple feature points are divided into multiple groups based on specified values ​​during the encoding process, and the first image feature of each feature point in each group is sequentially encoded into the bitstream. This grouping is then repeated during the decoding process, and the probability distribution is determined in parallel for each feature point in the same group to improve decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial calculations during VAE-based decoding, effectively improving decoding efficiency.

[0019] Optionally, based on the image to be encoded, the first image feature, probability distribution and first super prior feature of each feature point among multiple feature points of the image are determined, including: based on the image, determining the first image feature of the multiple feature points; based on the first image feature of the multiple feature points, determining the first super prior feature of the multiple feature points, and determining the probability distribution of each feature point among the multiple feature points in parallel.

[0020] Corresponding to the decoding method, the method is also applied to a codec including a context model. Optionally, the multiple feature points include a first feature point, and determining the probability distribution of the first feature point includes: if the first feature point is not the first feature point among the multiple feature points, then determining the prior feature of the first feature point based on the first image feature of the first feature point, the first feature point being one of the multiple feature points; determining the surrounding information of the first feature point from the first image features of the multiple feature points; inputting the surrounding information of the first feature point into the context model to obtain the context feature of the first feature point output by the context model; and determining the probability distribution of the first feature point based on the prior feature of the first feature point and the context feature of the first feature point.

[0021] Optionally, the multiple feature points include a first feature point, and determining the probability distribution of the first feature point includes: if the first feature point is the first feature point among the multiple feature points, determining the probability distribution of the first feature point based on a priori features of the first feature point.

[0022] Optionally, the specified numerical value is determined based on the size of the receptive field used by the context model; the multiple feature points are divided into multiple groups based on the specified numerical value, including: determining a slope based on the specified numerical value, the slope being used to indicate the degree of inclination of the straight line where the feature points divided into the same group are located; based on the slope, the multiple feature points are divided into multiple groups.

[0023] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0024] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0025] In a third aspect, a decoding device is provided, wherein the decoding device has the function of implementing the decoding method described in the first aspect. The decoding device includes one or more modules, wherein the one or more modules are used to implement the decoding method described in the first aspect.

[0026] That is, a decoding device is provided, the decoding device comprising:

[0027] A first determining module, configured to determine, based on a bitstream, a priori features of each feature point among a plurality of feature points of an image to be decoded;

[0028] A grouping module, configured to divide the plurality of feature points into a plurality of groups based on a specified value;

[0029] a second determining module, configured to sequentially determine, based on the prior features of the plurality of feature points, a first image feature of each group of feature points in the plurality of groups; wherein the step of determining the first image feature of any group of feature points comprises: concurrently determining a probability distribution of each feature point in the group, and parsing the first image feature of each feature point in the group from a bitstream based on the probability distribution of each feature point in the group;

[0030] A reconstruction module is used to reconstruct an image based on the first image feature of the multiple feature points.

[0031] Optionally, the multiple feature points include a first feature point, and the second determining module includes:

[0032] a first processing submodule, configured to determine, if the first feature point is not the first feature point among the plurality of feature points, surrounding information of the first feature point from the decoded first image features of each feature point, the first feature point being one feature point in any one of the groups;

[0033] A second processing submodule is configured to input surrounding information of the first feature point into a context model to obtain a context feature of the first feature point output by the context model;

[0034] The third processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point and context features of the first feature point.

[0035] Optionally, the surrounding information of the first feature point includes the first image features of the decoded feature points in a neighborhood with the first feature point as the geometric center, the size of the neighborhood is determined based on the size of the receptive field used by the context model, and the surrounding information includes at least the first image features of n feature points around the first feature point, where n is greater than or equal to 4.

[0036] Optionally, the multiple feature points include a first feature point, and the second determining module includes:

[0037] The fourth processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point if the first feature point is the first feature point among the multiple feature points.

[0038] Optionally, the specified value is based on the size of the receptive field used by the context model;

[0039] The grouping module includes:

[0040] A first determining submodule is configured to determine a slope based on a specified value, where the slope indicates the degree of inclination of the straight line where the feature points grouped into the same group are located;

[0041] The division submodule is used to divide the multiple feature points into multiple groups based on the slope.

[0042] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0043] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0044] In a fourth aspect, a coding device is provided, wherein the coding device has the function of implementing the coding method described in the second aspect. The coding device includes one or more modules, wherein the one or more modules are used to implement the coding method described in the second aspect.

[0045] That is, a coding device is provided, the device comprising:

[0046] A first determination module is configured to determine, based on the image to be encoded, a first image feature, a probability distribution, and a first super-prior feature of each feature point among a plurality of feature points of the image;

[0047] A grouping module, configured to divide the plurality of feature points into a plurality of groups based on a specified value;

[0048] A first encoding module is configured to sequentially encode a first image feature of each of the plurality of groups of feature points into a bitstream based on a probability distribution of the plurality of feature points;

[0049] The second encoding module is configured to encode the first super-a priori features of the multiple feature points into a bitstream.

[0050] Optionally, the first determining module includes:

[0051] A first determining submodule, configured to determine a first image feature of the plurality of feature points based on the image;

[0052] The second determination submodule is configured to determine first super-prior features of the plurality of feature points based on the first image features of the plurality of feature points, and to determine a probability distribution of each of the plurality of feature points in parallel.

[0053] Optionally, the multiple feature points include a first feature point, and the second determining submodule is configured to:

[0054] If the first feature point is not the first feature point among the multiple feature points, determining a priori features of the first feature point based on the first image feature of the first feature point, the first feature point being one of the multiple feature points;

[0055] determining peripheral information of the first feature point from the first image features of the plurality of feature points;

[0056] Inputting surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model;

[0057] Based on the priori feature of the first feature point and the context feature of the first feature point, a probability distribution of the first feature point is determined.

[0058] Optionally, the multiple feature points include a first feature point, and the second determining submodule is configured to:

[0059] If the first feature point is the first feature point among the multiple feature points, a probability distribution of the first feature point is determined based on a priori features of the first feature point.

[0060] Optionally, the specified value is determined based on the size of the receptive field used by the context model;

[0061] The grouping module includes:

[0062] A third determining submodule is configured to determine a slope based on the specified value, where the slope indicates a degree of inclination of the straight line where the feature points grouped into the same group are located;

[0063] The division submodule is used to divide the multiple feature points into multiple groups based on the slope.

[0064] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0065] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0066] In a fifth aspect, a decoding end device is provided, the decoding end device comprising a processor and a memory, the memory being configured to store a program for executing the decoding method provided in the first aspect. The processor is configured to execute the program stored in the memory to implement the decoding method provided in the first aspect.

[0067] Optionally, the decoding end device may further include a communication bus, which is used to establish a connection between the processor and the memory.

[0068] In a sixth aspect, an encoding end device is provided, the encoding end device comprising a processor and a memory, the memory being configured to store a program for executing the encoding method provided in the second aspect. The processor is configured to execute the program stored in the memory to implement the encoding method provided in the second aspect.

[0069] Optionally, the encoding end device may further include a communication bus, which is used to establish a connection between the processor and the memory.

[0070] In the seventh aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions. When the instructions are executed on a computer, the computer executes the steps of the decoding method described in the first aspect or the steps of the encoding method described in the second aspect.

[0071] In an eighth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer, the computer is caused to execute the steps of the decoding method described in the first aspect, or the steps of the encoding method described in the second aspect. Alternatively, a computer program is provided. When executed, the computer program implements the steps of the decoding method described in the first aspect, or the steps of the encoding method described in the second aspect.

[0072] The technical effects obtained in the above-mentioned third, fourth, fifth, sixth, seventh and eighth aspects are similar to the technical effects obtained by the corresponding technical means in the first or second aspect, and will not be repeated here.

[0073] The technical solutions provided in the embodiments of the present application can at least bring the following beneficial effects:

[0074] In this embodiment of the present application, during the decoding process, multiple feature points are divided into multiple groups based on specified values. The probability distribution of each feature point in the same group is then determined in parallel, which improves decoding efficiency. Accordingly, during the encoding process, the multiple feature groups are grouped in the same manner, and the first image feature of each feature point in each group is encoded into the bitstream. In other words, this solution can overcome the efficiency bottleneck caused by serial computation during VAE-based decoding, effectively improving decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0076] Figure 2 This is a schematic diagram of the structure of a coding and decoding framework provided in an embodiment of the present application;

[0077] Figure 3 This is a schematic diagram of an encoding and decoding sequence provided by an embodiment of the present application;

[0078] Figure 4 This is a flowchart of an encoding method provided in an embodiment of the present application;

[0079] Figure 5 This is a schematic diagram of utilizing surrounding information in encoding provided by an embodiment of the present application;

[0080] Figure 6 This is a schematic diagram of another encoding and decoding order provided by an embodiment of the present application;

[0081] Figure 7 This is a schematic diagram of another encoding and decoding sequence provided in an embodiment of the present application;

[0082] Figure 8 This is a schematic diagram of another encoding and decoding sequence provided in an embodiment of the present application;

[0083] Figure 9 This is a schematic diagram of another encoding and decoding sequence provided in an embodiment of the present application;

[0084] Figure 10 This is a schematic diagram of the structure of another encoding and decoding framework provided in an embodiment of the present application;

[0085] Figure 11 This is a flowchart of a decoding method provided in an embodiment of the present application;

[0086] Figure 12 This is a schematic structural diagram of a decoding device provided in an embodiment of the present application;

[0087] Figure 13 This is a schematic structural diagram of an encoding device provided in an embodiment of the present application;

[0088] Figure 14 This is a schematic block diagram of a coding and decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0090] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0091] Before explaining the encoding and decoding method provided in the embodiment of the present application in detail, the terminology and implementation environment involved in the embodiment of the present application are first introduced.

[0092] To facilitate understanding, the terms involved in the embodiments of the present application are first explained.

[0093] Pixel depth (bits per pixel, BPP): also known as bits / pixel, BPP is the number of bits used to store each pixel. The smaller the BPP, the lower the compression bit rate.

[0094] Bit rate: In image compression, it refers to the encoding length required to encode a unit pixel. The higher the bit rate, the better the image reconstruction quality.

[0095] Peak signal to noise ratio (PSNR): It is an objective standard for evaluating image quality. The higher the PSNR, the better the image quality.

[0096] Multi-scale structural similarity index measure (MS-SSIM): It is an objective criterion for evaluating images. The higher the MS-SSIM, the better the image quality.

[0097] Convolutional neural network (CNN): is a feedforward neural network with a deep structure that includes convolution calculations. It is one of the representative algorithms of deep learning.

[0098] Variational autoencoder (VAE): is a type of autoencoder used for data compression or denoising.

[0099] Next, the implementation environment involved in the embodiments of this application is introduced.

[0100] Please refer to Figure 1 , Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate an encoded image. Therefore, the source device 10 can also be referred to as an image encoding device. The destination device 20 can decode the encoded image generated by the source device 10. Therefore, the destination device 20 can also be referred to as an image decoding device. The link 30 can receive the encoded image generated by the source device 10 and transmit the encoded image to the destination device 20. The storage device 40 can receive the encoded image generated by the source device 10 and store the encoded image. Under such conditions, the destination device 20 can directly obtain the encoded image from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded image generated by the source device 10. Under such conditions, the destination device 20 can stream or download the encoded image stored by the storage device 40.

[0101] The source device 10 and the destination device 20 may each include one or more processors and a memory coupled to the one or more processors, wherein the memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other medium that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, the source device 10 and the destination device 20 may each include a mobile phone, a smartphone, a personal digital assistant (PDA), a wearable device, a pocket PC (PPC), a tablet computer, a smart car computer, a smart TV, a smart speaker, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an in-vehicle computer, or the like.

[0102] Link 30 may include one or more media or devices capable of transmitting encoded images from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media that enable source device 10 to send encoded images directly to destination device 20 in real time. In embodiments of the present application, source device 10 may modulate the encoded images based on a communication standard, such as a wireless communication protocol, and transmit the modulated images to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from source device 10 to destination device 20, although this embodiment is not particularly limited in this regard.

[0103] In one possible implementation, the storage device 40 may store the received encoded image sent by the source device 10, and the destination device 20 may directly obtain the encoded image from the storage device 40. In this case, the storage device 40 may include any of a variety of distributed or locally accessible data storage media, for example, any of the various distributed or locally accessible data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded images.

[0104] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device that can store the encoded images generated by source device 10. Destination device 20 can stream or download the images stored on storage device 40. The file server can be any type of server capable of storing and transmitting the encoded images to destination device 20. In one possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. Destination device 20 can obtain the encoded images via any standard data connection, including an internet connection. Any standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for obtaining the encoded images stored on the file server. The encoded images can be transmitted from storage device 40 by streaming, downloading, or a combination of both.

[0105] Figure 1 The implementation environment shown is only one possible implementation method, and the technology of the embodiment of the present application is not only applicable to Figure 1 The source device 10 that can encode images and the destination device 20 that can decode encoded images shown can also be applied to other devices that can encode images and decode encoded images, and the embodiments of the present application do not specifically limit this.

[0106] exist Figure 1In the illustrated implementation, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a modem and / or a transmitter, which may also be referred to as a transmitter. Data source 120 may include an image capture device (e.g., a camera), an archive containing previously captured images, a feed interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these sources of images.

[0107] The data source 120 may send an image to the encoder 100, and the encoder 100 may encode the image received from the data source 120 to generate an encoded image. The encoder may send the encoded image to an output interface. In some embodiments, the source device 10 directly sends the encoded image to the destination device 20 via the output interface 140. In other embodiments, the encoded image may also be stored on the storage device 40 for later retrieval by the destination device 20 for decoding and / or display.

[0108] exist Figure 1 In the illustrated implementation, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded images via the link 30 and / or from the storage device 40, and then transmit the encoded images to the decoder 200. The decoder 200 may decode the received encoded images to obtain decoded images. The decoder may transmit the decoded images to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded images. The display device 220 may be any of a variety of types of display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0109] although Figure 1Although not shown, in some aspects, encoder 100 and decoder 200 can be integrated with an encoder and decoder, respectively, and can include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or in separate data streams. In some embodiments, the MUX-DEMUX units can conform to the ITU H.223 multiplexer protocol, or other protocols such as the User Datagram Protocol (UDP), if applicable.

[0110] The encoder 100 and the decoder 200 can each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology of the embodiments of the present application is partially implemented in software, the device can store instructions for the software in a suitable non-volatile computer-readable storage medium, and can use one or more processors to execute the instructions in hardware to implement the technology of the embodiments of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be considered to be one or more processors. Each of the encoder 100 and the decoder 200 can be included in one or more encoders or decoders, and either of the encoders or decoders can be integrated as part of a combined encoder / decoder (codec) in the corresponding device.

[0111] Embodiments of the present application may generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The terms "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data used to decode a compressed image. This transmission may occur in real time or near real time. Alternatively, this communication may occur over time, such as when syntax elements are stored in the encoded bitstream to a computer-readable storage medium during encoding, and a decoding device may then retrieve the syntax elements at any time after they are stored to this medium.

[0112] The encoding and decoding method provided in the embodiment of the present application can be applied to various scenarios. In various scenarios, the encoded and decoded images can be images included in image files or images included in video files. Figure 1In the illustrated implementation environment, any of the following encoding methods may be executed by the encoder 100 in the source device 10 , and any of the following decoding methods may be executed by the decoder 200 in the destination device 20 .

[0113] It should be noted that the encoding and decoding method provided in the embodiment of the present application can be applied to any video and image compression framework of the VAE method. Next, the encoding and decoding model of the basic VAE method is introduced.

[0114] Please refer to Figure 2 At the encoding end, the original image is input into the encoding network model to extract features and obtain the image features to be quantized at multiple feature points. , the image features to be quantified In this embodiment of the application, it is also referred to as the second image feature , the second image features of the multiple feature points Quantify to obtain the first image features of the multiple feature points The first image feature of the multiple feature points Input into the super coding network model to obtain the super prior features to be quantified of the multiple feature points , the hyper-prior feature to be quantified In this embodiment of the application, it is also called the second super prior feature , the second super prior feature of the multiple feature points Quantify to obtain the first super prior features of the multiple feature points The first super-prior features of the multiple feature points are calculated according to the specified probability distribution. Perform entropy coding to convert Encoded into the code stream. Figure 2 Shown The bit sequence obtained by entropy coding is the partial bit sequence included in the code stream. This partial bit sequence (such as Figure 2 The black and white bars on the right side of the figure can be called the super-prior bit stream.

[0115] In addition, the first super prior features of the multiple feature points Input the super decoding network model to obtain the prior features of the multiple feature points The first image feature of the multiple feature points Input context model (CM) to obtain the context features of the multiple feature points . Combined with the prior features of the multiple feature points and contextual features , the probability distribution of the multiple feature points is estimated through the probability distribution estimation model (shown as gather model, GM) , based on the probability distribution of the multiple feature points , and sequentially calculate the first image feature of each feature point in the multiple feature points To encode into the code stream. Figure 2 Shown The bit sequence obtained by entropy coding is the partial bit sequence included in the code stream. This partial bit sequence (such as Figure 2 The black and white bars on the left of the image are called image bit streams.

[0116] At the decoding end, first, the first super-prior feature of the multiple feature points is obtained by entropy decoding from the super-prior bit stream included in the code stream according to the specified probability distribution. , the first super prior features of the multiple feature points Input the super decoding network model to obtain the prior features of the multiple feature points . For the first feature point among the multiple feature points, the probability distribution of the first feature point is estimated based on the prior features of the first feature point, and based on the probability distribution of the first feature point, the first image feature of the first feature point is parsed from the image bit stream included in the code stream. For the non-first feature point among the multiple feature points, such as the first feature point, the peripheral information of the first feature point is determined from the first image features of each decoded feature point, and the peripheral information of the first feature point is input into the context model CM to obtain the context feature of the first feature point. The probability distribution of the first feature point is estimated by the probability distribution estimation model GM in combination with the prior features and the context features of the first feature point, and based on the probability distribution of the first feature point, the first image feature of the first feature point is parsed from the image bit stream included in the code stream. The first image feature of the multiple feature points is entropy decoded from the code stream Afterwards, Input the decoding network model to obtain the reconstructed image.

[0117] In the calculation process of estimating the probability distribution of the multiple feature points, both the encoding end device and the decoding end device in the related art calculate the probability distribution of each feature point in the multiple feature points in sequence, such as Figure 3 As shown, in the related art, the decoding end is based on Figure 3 The probability distribution of each feature point is estimated in the order indicated by the arrows. The encoder side follows Figure 3 The first image features of each feature point are encoded into the code stream in the order indicated by the arrows. Figure 3 As shown, in the related technology, it is assumed that the current decoding end decodes a feature point filled with black, and the feature point filled with diagonal lines is the decoded feature point. For this feature point filled with black, the surrounding information of the feature point includes the first image feature of the 12 feature points filled with diagonal lines in the black thick line frame.

[0118] As can be seen above, the VAE encoding and decoding model consists of two parts: a feature extraction and decoding module, and an entropy coding module. By introducing contextual information (i.e., surrounding information) and super-prior information into the entropy coding module, compression performance can be significantly improved.

[0119] Next, the encoding method provided in the embodiment of the present application is introduced.

[0120] Please refer to Figure 4 , Figure 4 This is a flowchart of an encoding method provided in an embodiment of the present application. The encoding method is applied to an encoding end device and includes the following steps.

[0121] Step 401: Based on an image to be encoded, determine a first image feature, a probability distribution, and a first super-prior feature of each feature point among a plurality of feature points of the image.

[0122] The image to be encoded is an image in an image file or an image in a video file, and the image to be encoded can be in any form, which is not limited in the embodiment of the present application.

[0123] In an embodiment of the present application, the implementation process of determining the first image feature, probability distribution and first super prior feature of each feature point among multiple feature points of the image based on the image to be encoded is: based on the image, determine the first image feature of the multiple feature points, based on the first image feature of the multiple feature points, determine the first super prior feature of the multiple feature points, and determine the probability distribution of each feature point among the multiple feature points in parallel.

[0124] Among them, based on the image, the implementation process of determining the first image features of the multiple feature points is: inputting the image into the coding network model, obtaining the second image features of the multiple feature points output by the coding network model, and quantizing the second image features of the multiple feature points to obtain the first image features of the multiple feature points.

[0125] Based on the first image features of the multiple feature points, the implementation process of determining the first super prior features of the multiple feature points is as follows: input the first image features of the multiple feature points into the super coding network model, obtain the second super prior features of the multiple feature points output by the super coding network model, quantize the second super prior features of the multiple feature points, and obtain the first super prior features of the multiple feature points.

[0126] There are various quantization methods involved in the above implementation process, such as scalar quantization. The quantization step size of variable quantization can be determined based on different coding rates. That is, the corresponding relationship between the coding rate and the quantization step size is stored in advance, and the corresponding quantization step size is obtained from the corresponding relationship based on the coding rate used in the embodiment of the present application. In addition, scalar quantization can also have an offset, that is, the data to be quantized (such as the second image feature or the second super prior feature) is offset by the offset and then scalar quantization is performed according to the quantization step size.

[0127] It should be noted that the quantization processing method below is similar to that here. The quantization processing method below can refer to the method here, and the embodiments of this application will not be repeated hereafter.

[0128] The multiple feature points include a first feature point, and the implementation process of determining the probability distribution of the first feature point is as follows: if the first feature point is not the first feature point among the multiple feature points, determining a priori features of the first feature point based on the first image features of the first feature point, determining surrounding information of the first feature point from the first image features of the multiple feature points, inputting the surrounding information of the first feature point into a context model, obtaining a context feature of the first feature point output by the context model, and determining the probability distribution of the first feature point based on the priori features of the first feature point and the context feature of the first feature point. The first feature point is one of the multiple feature points.

[0129] Among them, the implementation process of determining the prior feature of the first feature point based on the first image feature of the first feature point is: based on the first image feature of the first feature point, determine the first super prior feature of the first feature point, and based on the first super prior feature of the first feature point, determine the prior feature of the first feature point. It should be noted that the implementation process of determining the first super prior feature of the first feature point, that is, the implementation process of determining the first super prior feature of any feature point among the multiple feature points, has been introduced above and will not be repeated here. And the implementation process of determining the prior feature of the first feature point based on the first super prior feature of the first feature point is: input the first super prior feature of the first feature point into the super-decoding network model to obtain the prior feature of the first feature point output by the super-decoding network model.

[0130] The encoding network model, super encoding network model, and super decoding network model introduced above are all pre-trained. The embodiments of the present application do not limit the network structure and training method of the encoding network model, super encoding network model, and super decoding network model. For example, the network structure of the encoding network model, super encoding network model, and super decoding network model can all be a fully connected network or a convolutional neural network CNN, and the convolution in the convolutional neural network can be a 2D convolution or a 3D convolution. In addition, the embodiments of the present application do not limit the number of layers contained in the network structure of the encoding network model, super encoding network model, and super decoding network model and the number of nodes in each layer.

[0131] In the embodiment of the present application, taking the network structures of the encoding network model, the super-encoding network model, and the super-decoding network model as CNNs, and the convolution in the CNN as a 2D convolution as an example, the second image features of the multiple feature points output by the encoding network model are represented by a C*W*H dimensional matrix, and the first image features of the multiple feature points obtained by quantization are also represented by a C*W*H dimensional matrix, where C is the number of channels of the CNN, and W*H represents the size of the feature map composed of the multiple feature points. Accordingly, the second super-prior features of the multiple feature points obtained based on the super-encoding network model, the first super-prior features of the multiple feature points obtained by quantization, and the prior features of the multiple feature points obtained based on the super-decoding network model are also represented by C*W*H dimensional matrices.

[0132] In addition, the context model in the embodiment of the present application is also pre-trained, and the embodiment of the present application does not limit the network structure and training method of the context model. For example, the network structure of the context model can be a mask region CNN (Mask R-CNN), where the receptive field is used in Mask R-CNN to extract context features. One or more receptive fields can be used in the context model, and the sizes of the one or more receptive fields are different. The embodiment of the present application does not limit this. Optionally, in the embodiment of the present application, the receptive field used by the context model includes a receptive field of size 5*5. In addition, the convolution in the context model can be a 2D convolution or a 3D convolution. Assuming that the convolution in the context model is a 2D convolution, the size of the receptive field can be 3*3, 5*5 or 7*7, etc.

[0133] It should be noted that the peripheral information of the first feature point is the image feature required to determine the context feature of the first feature point. Optionally, the implementation process of determining the peripheral information of the first feature point from the first image features of the multiple feature points is: determining the peripheral information of the first feature line from the first image features of the multiple feature points according to a preset rule. Optionally, the peripheral information of the first feature point includes the first image features of at least n feature points in a neighborhood with the first feature point as the geometric center, and the size of the neighborhood is determined based on the size of the receptive field used by the context model. The peripheral information of the first feature point includes at least the first image features of n feature points around the first feature point, and n is greater than or equal to 4.

[0134] For example, see Figure 5 Assuming that the context model uses a receptive field of size 5*5 and the convolution in the context model is a 2D convolution, the size of the field is 5*5. The surrounding information of feature point A determined according to the preset rules includes the first image features of the 6 feature points within the thick line frame with feature point A as the geometric center, and these 6 feature points are located above feature point A (including directly above and diagonally above). The surrounding information of feature point B includes the first image features of 12 feature points within the thick line frame with feature line B as the geometric center, and these 12 feature points are located at the 2 feature points directly to the left and 10 feature points above feature point B. The surrounding information of feature point C also includes the first image features of the 12 feature points. The relative positions of these 12 feature points and feature point C are similar to the relative positions of feature point B and the 12 feature points corresponding to feature point B. The surrounding information of the feature point D includes the first image features of eight feature points within a thick-line frame with the feature point D as the geometric center, and these eight feature points are located at two feature points to the left of the feature point D and six feature points above it.

[0135] If the first feature point is not the first feature point among the multiple feature points, after determining the surrounding information of the first feature point, the surrounding information of the first feature point is input into the context model to obtain the context features of the first feature point output by the context model. Thereafter, a probability distribution of the first feature point is determined based on the prior features of the first feature point and the context features of the first feature point.

[0136] Optionally, based on the prior features of the first feature point and the context features of the first feature point, the implementation process of determining the probability distribution of the first feature point is: inputting the prior features of the first feature point and the context features of the first feature point into the probability distribution estimation model, and obtaining the probability distribution of the first feature point output by the probability distribution estimation model, where the probability distribution is represented by a mean and a standard deviation. The probability distribution estimation model is pre-trained, and the network structure of the probability distribution estimation model is a neural network, such as a CNN. The embodiment of the present application does not limit the number of layers contained in the network structure of the probability distribution estimation model and the number of nodes in each layer. Optionally, the probability distribution estimation model is the GM model introduced above.

[0137] In addition, if the first feature point is the first feature point among the multiple feature points, the probability distribution of the first feature point is determined based on the prior features of the first feature point. That is to say, for the first feature point, the surrounding information is not used in the encoding process, or the surrounding information of the first feature point is set to 0. It should be noted that if the first feature point is the first feature point among the multiple feature points, the implementation process of determining the probability distribution of the first feature point is: inputting the prior features of the first feature point into the probability distribution estimation model to obtain the probability distribution of the first feature point output by the probability distribution estimation model. Alternatively, the surrounding information of the first feature point is 0, and the implementation process of determining the probability distribution of the first feature point is: inputting the surrounding information of the first feature point into the context model to obtain the context features of the first feature point output by the context model, the context features of the first feature point are 0, the prior features of the first feature point and the context feature 0 are input into the probability distribution estimation model to obtain the probability distribution of the first feature point output by the probability distribution estimation model.

[0138] It should be noted that if multiple receptive fields are used in the context model, then in the process of determining the context features of each feature point based on the context model, feature extraction is performed on the surrounding information of each feature point based on each receptive field in the multiple receptive fields, and multiple first context features of each feature point corresponding to the corresponding receptive field are obtained. That is, the context features of each feature point are determined based on the multiple first context features of the corresponding feature point, and the multiple first context features correspond one-to-one to the multiple receptive fields. Simply put, by using several receptive fields, several first context features are obtained for each feature point.

[0139] Based on this, in one implementation, the context features of each feature point include multiple first context features of the corresponding feature point. After obtaining the multiple first context features of each feature point based on the multiple receptive fields used in the context model, the context model outputs the multiple first context features of each feature point. The prior features of each feature point and the multiple first context features of the corresponding feature point are then input into the probability distribution estimation model to obtain the probability distribution of the corresponding feature point output by the probability distribution estimation model. In this implementation, the context features of each feature point include multiple first context features of the corresponding feature point.

[0140] Exemplarily, the context model uses three receptive fields with sizes of 3*3, 5*5 and 7*7 respectively, then three first context features are obtained for each feature point, the prior features of each feature point and the three context features of the corresponding feature point are input into the probability distribution estimation model, and the probability distribution of the corresponding feature point output by the probability distribution estimation model is obtained.

[0141] In another implementation, after obtaining multiple first context features for each feature point based on the multiple receptive fields used in the context model, the multiple first context features of each feature point are further processed by the context model to obtain the context features of the corresponding feature point output by the context model. Subsequently, the prior features and context features of each feature point are input into the probability distribution estimation model to obtain the probability distribution of the corresponding feature point output by the probability distribution estimation model. In this implementation, the context features of each feature point are the context features obtained by fusing the multiple first context features of the corresponding feature point.

[0142] The above describes the implementation process of determining the first image features, probability distribution and first super prior features of each feature point among multiple feature points of the image based on the image to be encoded. In the embodiment of the present application, the implementation process is similar to the relevant process of the VAE method introduced above. In the above embodiment, after obtaining the first image features of each feature point, the prior features and context features of each feature point are determined based on the first image features of each feature point. The two steps of determining the prior features and the context features can be regarded as two branches. These two branches can be executed in parallel to speed up the encoding speed. In addition, by determining the probability distribution of each feature point in parallel, the encoding efficiency can be guaranteed.

[0143] Step 402: Divide the plurality of feature points into a plurality of groups based on designated values.

[0144] To enable parallel decoding of feature points at the decoding end to improve decoding efficiency, this solution optimizes the encoding and decoding order of each feature point compared to related VAE-based technologies, enabling the parallel determination of the probability distribution of some feature points. In this embodiment of the present application, the multiple feature points are divided into multiple groups based on specified values, each group including at least one feature point. The subsequent encoding end device can then sequentially encode the first image feature of each feature point in the multiple groups into the bitstream as described in step 403 below.

[0145] The specified value is determined based on the size of the receptive field used by the context model. Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0146] For example, the encoding process uses a convolutional network, and the convolution in the convolutional network is a 2D convolution. The specified value is determined by ks Indicates that if the context model uses a receptive field of size 5*5, then the specified value ks =5. If the context model uses receptive fields of size 3*3, 5*5, and 7*7, then specify the value ks =7.

[0147] The implementation process of dividing the multiple feature points into multiple groups based on the specified value is as follows: determining the slope based on the specified value, and dividing the multiple feature points into multiple groups based on the slope. The slope is used to indicate the degree of inclination of the straight line where the feature points divided into the same group are located. It should be noted that in the grouping method corresponding to 2D convolution, the slope is intuitive, such as Figure 5 As shown, according to this solution, feature points A, B, C, and D are divided into the same group. It can be seen that feature points A, B, C, and D are actually on a straight line, and the slope can indicate the degree of inclination of the line. In the implementation of 3D convolution, the slope is not intuitive. In the embodiment of this application, 2D convolution is used as an example to introduce the grouping method. The principle of the grouping method corresponding to 3D convolution is the same as that of the grouping method corresponding to 2D convolution.

[0148] For example, taking 2D convolution as an example, specify the value ks =5, then the slope k= ,in, In other words, the slope is determined based on the specified value as follows: According to the formula k= Determine the slope k.

[0149] Still taking 2D convolution as an example, assuming that the first feature point is the feature point in the upper left corner of the feature map, and the coordinates of the first feature point are (0,0), based on the slope, the multiple feature points are divided into multiple groups by: based on the slope, the multiple feature points are divided into multiple groups in a cyclic manner. Among them, the t-th cycle of the cyclic method includes: if there are undivided feature points among the multiple feature points, then the feature points with abscissa ti*k and ordinate i among the multiple feature points are divided into a group. Among them, k is the slope, t, i, and ti*k are all integers, and the minimum value of t and i is 0.

[0150] like Figure 6 As shown, the first feature point is the feature point in the upper left corner, and the encoding and decoding sequence number of the first feature point is 1. Figure 6 The feature points with the same codec number belong to the same group. The order of codec numbers from small to large is the codec order of each group in the multiple groups. The last feature point is the feature point in the lower right corner, and the codec number of the last feature point is the largest. Figure 6 The encoding and decoding order shown is from the upper left corner, first to the right, and then gradually to the lower right corner.

[0151] It is conceivable that if Figure 6 After rotating 90 degrees counterclockwise, we get Figure 7 Another encoding and decoding order is shown in Figure 7 In the feature map, the first feature point is the feature point in the lower left corner. The encoding and decoding order is from the lower left corner, first upward, then gradually to the upper right corner, and the last feature point is the feature point in the upper right corner. Or, Figure 6 Based on the diagonal line formed by connecting the upper left corner to the lower right corner, a mirror transformation is performed, and the result is as follows Figure 8 Another encoding and decoding order shown in Figure 8 In the feature map, the first feature point is the feature point in the upper left corner. The encoding and decoding order starts from the upper left corner, first downward, then gradually to the lower right corner, and the last feature point is the feature point in the lower right corner. It can be seen that according to this scheme, multiple encoding and decoding orders can actually be obtained. Intuitively, by Figure 6 By rotating and / or mirroring the image, eight encoding and decoding orders can be obtained, and the eight encoding and decoding orders are essentially similar. In the grouping method corresponding to these eight encoding and decoding orders, the first feature point is used as the coordinate origin, the direction from the first feature point to the second feature point is the horizontal axis, and the other side of the feature map perpendicular to the horizontal axis is the vertical axis. The multiple feature points are divided into multiple groups according to the slope.

[0152] For 3D convolution, based on the slope, the implementation method of dividing the multiple feature points into multiple groups is similar to the implementation method corresponding to 2D convolution. The tth cycle of the cyclic method corresponding to 3D convolution includes: if there are undivided feature points among the multiple feature points, then the feature points whose coordinates (x, y, z) satisfy x+k*y+k*k*zt=0 in the multiple feature points are divided into a group, wherein k is the slope, x, y, z and t are all integers, and the minimum values ​​of x, y, z and t are all 0. That is, the multiple feature points are regarded as feature points included in a 3D feature map, and the 3D feature map includes multiple 2D feature maps. The plane where each 2D feature map is located is parallel to the xy plane, and the feature points divided into the same group are scattered on each 2D feature map included in the 3D feature map. In this way, spatial parallel encoding and decoding can be achieved with a high degree of parallelism.

[0153] Alternatively, the feature points in each of the multiple 2D feature maps are grouped in order from small to large z, in a manner similar to the grouping method corresponding to the above 2D convolution. Figure 9 As shown in the figure, the feature points on the plane z = 0 are first grouped, that is, the feature points in the 2D feature map of the xy plane are grouped. The grouping method is similar to the 2D method. After the grouping of the plane z = 0 is completed, the feature points on the plane z = 1 are grouped. The grouping method is similar to the 2D method. This method is repeated until all feature points are grouped. The encoding and decoding implemented in this way is actually parallel in the plane and serial in space.

[0154] It should be noted that, compared with related technologies, this solution can achieve parallel determination of probability distribution in the subsequent decoding process by adjusting the encoding and decoding order of each feature point without changing the surrounding information available for each feature point. Figure 5 As shown, similar to the related art, while ensuring the maximum available surrounding information during encoding and decoding, only the encoding and decoding order is adjusted, which neither reduces the encoding and decoding performance nor reduces the encoding and decoding efficiency.

[0155] Step 403: Based on the probability distribution of the multiple feature points, sequentially encode the first image feature of each group of feature points in the multiple groups into the bitstream.

[0156] In the embodiment of the present application, after the multiple feature points are divided into multiple groups, the first image features of each group of feature points in the multiple groups are sequentially encoded into the bitstream based on the probability distribution of the multiple feature points. That is, according to the encoding and decoding order after the grouping, the group of feature points with smaller codec numbers is encoded first, and then the group of feature points with larger codec numbers is encoded, until the first image features of each feature point in the multiple feature points are encoded into the bitstream.

[0157] The implementation process of sequentially encoding the first image feature of each of the multiple groups of feature points into the bitstream based on the probability distribution of the multiple feature points is as follows: sequentially performing entropy coding on the first image feature of each of the multiple groups of feature points based on the probability distribution of the multiple features to obtain an image bit sequence corresponding to the feature points in the corresponding group, and writing the image bit sequence of the feature points in the corresponding group into the bitstream. Optionally, the image bit sequences of the multiple feature points in the bitstream constitute an image bitstream.

[0158] In the embodiment of the present application, entropy coding is performed using an entropy coding model based on probability distribution. Entropy coding may use one of arithmetic coding (arithmetic coding), range coding (RC) or Huffman coding, which is not limited in the embodiment of the present application.

[0159] Step 404: Encode the first super-prior features of the plurality of feature points into a bitstream.

[0160] In an embodiment of the present application, since the decoding at the decoding end needs to rely on the super-prior features of the feature points, the first super-prior features of the multiple feature points also need to be encoded into the bitstream at the encoding end. The implementation process of encoding the first super-prior features of the multiple feature points into the bitstream is as follows: according to a specified probability distribution, the first super-prior features of the multiple feature points are encoded into the bitstream. In one implementation method, according to a specified probability distribution, the first super-prior features of the multiple feature points are entropy encoded to obtain the super-prior bit sequences of the multiple feature points, and the super-prior bit sequences of the multiple feature points are written into the bitstream. That is, the first super-prior features can also be encoded into the bitstream by entropy encoding. Optionally, the super-prior bit sequences of the multiple feature points in the bitstream constitute a super-prior bitstream. That is, the bitstream includes two parts, one part is the image bitstream, and the other part is the super-prior bitstream.

[0161] Among them, the specified probability distribution is a probability distribution determined in advance by a probability distribution network model, and the embodiments of the present application do not limit the network structure and training method of the probability distribution network model used to train the specified probability distribution. For example, the network structure of the probability distribution network model can be a fully connected network or a CNN. In addition, the embodiments of the present application do not limit the number of layers contained in the network structure of the probability distribution network model and the number of nodes in each layer.

[0162] At this point, the encoding device has completed encoding of the image to be encoded through steps 401 to 404, thus obtaining a codestream. It should be noted that steps 402 and 403 can be performed serially, i.e., the feature points are grouped and then encoded sequentially, or they can be performed in parallel, i.e., while grouping in the aforementioned cyclic manner, each time a group is formed, the first image feature of the feature points in that group is encoded into the codestream, and then the next group is formed, and so on until the last group is formed, at which point the first image feature of the feature points in the last group is encoded into the codestream.

[0163] Next, combine Figure 2 The encoding method provided in the embodiment of the present application is further explained exemplarily through the following steps 1 to 7.

[0164] 1. Input the image to be encoded into the encoding network model to obtain the second image features of multiple feature points ,right Quantify to obtain the first image features of the multiple feature points , the first image feature That is the image feature to be encoded into the code stream.

[0165] 2. The first image feature of the multiple feature points Input the super coding network model to obtain the second super prior features of the multiple feature points ,right Quantify to obtain the first super prior features of the multiple feature points .

[0166] 3. Input the super decoding network model to obtain the prior features of the multiple feature points .

[0167] 4. Input the context model to obtain the context features of the multiple feature points .

[0168] 5. Combine prior features and contextual features , through the probability distribution estimation model, the probability distribution of the multiple feature points is obtained.

[0169] 6. According to the specified probability distribution, Perform entropy coding to obtain a super-prior bit stream.

[0170] 7. Yes Perform entropy coding, including steps a to c:

[0171] a. The current loop number is t.

[0172] b. Encode the first image feature of the feature point with coordinates (tk*i, i), where k is the slope and i is an integer.

[0173] ct=t+1, return to step a until the first image features of all feature points are encoded.

[0174] It should be noted that the convolution in each network model involved in steps 1 to 7 above is a 2D convolution, and the encoding starts from the feature point in the upper left corner, first to the right, and then gradually to the lower right corner. Assuming k=3, the encoding and decoding order of the encoding end is as follows Figure 6 shown.

[0175] In summary, in the embodiments of the present application, in order to determine probability distributions in parallel during the decoding process to improve decoding efficiency, multiple feature points are divided into multiple groups based on specified values ​​during the encoding process, and the first image feature of each feature point in each group is sequentially encoded into the bitstream. This grouping is then repeated during the decoding process, and probability distributions are determined in parallel for each feature point in the same group to improve decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial calculations during VAE-based decoding, effectively improving decoding efficiency.

[0176] Next, the decoding method provided in the embodiment of the present application is introduced.

[0177] Please refer to Figure 11 , Figure 11 This is a flowchart of a decoding method provided in an embodiment of the present application. The decoding method is applied to a decoding end device and includes the following steps.

[0178] Step 1101: Determine a priori features of each feature point among a plurality of feature points of an image to be decoded based on a code stream.

[0179] In an embodiment of the present application, the implementation process of determining the prior features of each of the multiple feature points of the image to be decoded based on the code stream is: determining the first super prior features of the multiple feature points based on the code stream, and determining the prior features of the multiple feature points based on the first super prior features of the multiple feature points.

[0180] The process of determining the first super a priori features of the multiple feature points based on the bitstream may include: performing entropy decoding on the bitstream according to a specified probability distribution to obtain the first super a priori features of the multiple feature points. The process of determining the a priori features of the multiple feature points based on the first super a priori features of the multiple feature points may include: inputting the first super a priori features of the multiple feature points into a super-decoding network model to obtain the a priori features of the multiple feature points output by the super-decoding network model.

[0181] like Figure 2As shown, at the decoding end, the super priori bit stream included in the code stream is entropy decoded according to the specified probability distribution to obtain the first super priori feature of the multiple feature points ,Will Input the super decoding network model to obtain the prior features of the multiple feature points output by the super decoding network model .

[0182] It should be noted that the decoding method in this step corresponds to the encoding method in the encoding end, the specified probability distribution in this step is the same as the specified probability distribution in the encoding end, and the super-decoding network model in this step is consistent with the network structure of the super-decoding network model in the encoding end.

[0183] Step 1102: Divide the plurality of feature points into a plurality of groups based on designated values.

[0184] Similar to the encoding end, the decoding end also needs to divide the multiple feature points into multiple groups based on the specified numerical value, and the grouping method in this step is the same as the grouping method in the encoding end, that is, the implementation process of dividing the multiple feature points into multiple groups based on the specified numerical value can be: determine the slope based on the specified numerical value, and divide the multiple feature points into multiple groups based on the slope. Among them, the specified numerical value is determined based on the size of the receptive field used by the context model, and the slope is used to indicate the degree of inclination of the straight line where the feature points divided into the same group are located. Optionally, if the context model uses multiple receptive fields of different sizes, the specified numerical value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes. It should be noted that the specific implementation method of grouping refers to the relevant introduction in the aforementioned encoding method, which will not be repeated here.

[0185] Step 1103: Based on the prior features of the multiple feature points, determine the first image features of each group of feature points in the multiple groups in turn, wherein the step of determining the first image features of any group of feature points is: determining the probability distribution of each feature point in any group in parallel, and based on the probability distribution of each feature point in any group, parsing the first image features of each feature point in any group from the code stream.

[0186] In the embodiment of the present application, when the multiple feature points are divided into multiple groups, the first image features of each of the multiple groups are sequentially determined at the decoding end based on the prior features of the multiple feature points. For each feature point in any group, the probability distribution of each feature point in the group is determined in parallel. Then, based on the probability distribution of each feature point in the group, the first image features of each feature point in the group are parsed from the bitstream.

[0187] For example, it is assumed that after the encoding end is grouped, Figure 6 The codec number shown in the codec is also the same as the one shown in the codec number. Figure 6The probability distribution of the codec number shown is determined. First, the probability distribution of the feature point with the codec number 1 is determined, then the probability distribution of the feature point with the codec number 2 is determined, then the probability distribution of the feature point with the codec number 3 is determined, and so on, until the probability distribution of the multiple feature points is determined. Figure 6 It can be seen that in this scheme, the probability distribution of a group of feature points with the same encoding and decoding sequence number can be determined in parallel, and the decoding efficiency is greatly improved.

[0188] The plurality of feature points includes a first feature point, and determining the probability distribution of the first feature point is implemented as follows: if the first feature point is not the first feature point among the plurality of feature points, determining surrounding information of the first feature point from the first image features of each decoded feature point, inputting the surrounding information of the first feature point into a context model, obtaining context features of the first feature point output by the context model, and determining the probability distribution of the first feature point based on the prior features of the first feature point and the context features of the first feature point. The first feature point is a feature point in any one of the groups.

[0189] Optionally, the surrounding information of the first feature point includes the first image features of the decoded feature points in a neighborhood with the first feature point as the geometric center, the size of the neighborhood is determined based on the size of the receptive field used by the context model, and the surrounding information of the first feature point includes at least the first image features of n feature points surrounding the first feature point, where n is greater than or equal to 4.

[0190] It should be noted that the surrounding information of the first feature point in the decoding method of the decoding end is the same as the surrounding information of the first feature point in the encoding method of the encoding end, and will not be repeated here.

[0191] In addition, if the first feature point is the first feature point among multiple feature points, the probability distribution of the first feature point is determined based on the prior features of the first feature point. The implementation process of determining the probability distribution of the first feature point on the decoding end is the same as that on the encoding end and will not be repeated here.

[0192] Step 1104: Reconstruct an image based on the first image feature of the plurality of feature points.

[0193] In the embodiment of the present application, the method for reconstructing an image based on the first image features of the multiple feature points is as follows: the first image features of the multiple feature points are input into a decoding network model to obtain a reconstructed image output by the decoding network model. In this step, the decoding network model corresponds to the network structure of the encoding network model at the encoding end, that is, the decoding step in the decoding network model is the inverse process of the encoding step in the encoding network model. For example, Figure 10 In the encoding and decoding framework shown, the network structure of the decoding network model is opposite to that of the encoding network model.

[0194] At this point, the decoding end device has completed decoding the code stream through steps 1101 to 1104, that is, reconstructing the image. It should be noted that steps 1102 and 1103 can be performed serially, that is, the feature points are grouped and then decoded in sequence, or steps 1102 and 1103 can be performed in parallel, that is, while grouping in the above-mentioned cyclic manner, each time a group is divided, the probability distribution of each feature point in the group is determined in parallel, and the first image feature of the feature point in the group is parsed from the code stream based on the probability distribution. The next group is then divided, and so on, until the last group is divided and the probability distribution of the feature points in the last group is parsed from the code stream.

[0195] Next, combine Figure 2 The decoding method provided in the embodiment of the present application is further explained exemplarily through the following steps 1 to 4.

[0196] 1. Read the code stream and perform entropy decoding on the super-prior bit stream included in the code stream according to the specified probability distribution to parse the super-prior features of multiple feature points from the code stream. .

[0197] 2. Input the super priori decoding network to obtain the prior features of the multiple feature points .

[0198] 3. According to the following steps a to e, entropy decoding is performed on the image bit stream included in the code stream to obtain the first image feature of the multiple feature points .

[0199] a. The current number of cycles is t;

[0200] b. Input the surrounding information of the feature point with coordinates (tk*i, i) into the context model to obtain the context features of the feature point with coordinates (tk*i, i).

[0201] c. Combining the prior features and context features of the feature point with coordinates (tk*i, i), the probability distribution of the feature point with coordinates (tk*i, i) is obtained through the probability distribution estimation model.

[0202] d. Based on the probability distribution of the feature point with coordinates (tk*i, i), decode the first image feature of the feature point with coordinates (tk*i, i) from the code stream.

[0203] et=t+1, return to step a until the first image features of all feature points are decoded .

[0204] 4. The first image feature of the multiple feature points Input the decoding network to obtain the reconstructed image.

[0205] It should be noted that the convolution in each network model involved in steps 1 to 4 above is a 2D convolution, and decoding starts from the feature point in the upper left corner, first to the right, and then gradually to the lower right corner. Assuming k=3, the decoding order at the decoding end is as follows: Figure 6 shown.

[0206] In order to verify the performance and efficiency of the encoding and decoding method provided in the embodiment of the present application, the encoding method provided in the embodiment of the present application was used to conduct experiments on the test sets Kodak and CLIC respectively. Among them, the resolution of the image to be encoded in the test set Kodak is 512*768, and the resolution of the image to be encoded in the test set CLIC is 2048*1367. In one experiment, the context model in the encoding and decoding uses a single receptive field with a size of 5*5. The experimental results of the experiment are shown in Table 1. Among them, Ctx serial represents the encoding and decoding method of the related technology, Ctx parallel represents the encoding and decoding method provided in the embodiment of the present application, Enc represents encoding, and Dec represents decoding. The encoding and decoding framework of this scheme is the same as that of the related technology, but the encoding and decoding order of the feature points is different. It can be seen that compared with the existing technology, this scheme can significantly save decoding time and the encoding and decoding efficiency of this scheme is higher. It should be noted that since this scheme does not reduce or change the available surrounding information compared with the related technology, the encoding and decoding performance of this scheme is comparable to that of the related technology, that is, this scheme does not reduce the quality of the reconstructed image.

[0207] Table 1

[0208]

[0209] In another experiment, Figure 10 The codec framework shown in the figure uses three receptive fields in the context model. The sizes of these three receptive fields are 3*3, 5*5, and 7*7, respectively. This scheme has the same codec framework as the related art, but the encoding and decoding order of the feature points is different. The experimental results of this experiment are shown in Table 2. Here, the ratio Ratio represents the encoding and decoding time saving rate of this scheme compared with the related art. Enc-R is the encoding time saving rate, and Dec-R is the decoding time saving rate. A positive saving rate indicates time saving, and a negative saving rate indicates time increase. It can be seen that compared with the related art, this scheme can save 84.6% of the decoding time on the test set Kodak and 92% of the encoding time on the test set CLIC. When using this scheme, the decoding time saving rate increases with the increase of image resolution. This is precisely because the higher the image resolution, the higher the proportion of feature points that can be decoded in parallel when using this scheme.

[0210] Where, Ratio= *100%, represents the encoding time of the relevant technology, and represents the encoding time of this scheme, or, represents the decoding time of the related art, and Indicates the decoding time of this solution.

[0211] Table 2

[0212]

[0213] As can be seen from the above, this solution is actually a parallelized method for probability distribution-based entropy coding using contextual features. Compared with related technologies, it achieves a significant reduction in decoding time without changing the available surrounding information. Furthermore, the higher the image resolution, the greater the encoding and decoding time savings. The more complex the context model (e.g., more receptive fields), the greater the encoding and decoding time savings. In multi-layer context models and probability distribution estimation models, this solution can save nearly 10 times the time compared to related technologies. Furthermore, this solution does not require changing the overall method of related technologies, and therefore does not require retraining the network model in the encoding and decoding framework. This means that this solution is more convenient to apply and does not reduce encoding and decoding performance.

[0214] In summary, in this embodiment of the present application, during the decoding process, multiple feature points are divided into multiple groups based on specified values, and the probability distribution is determined in parallel for each feature point in the same group. This can accelerate decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial calculations during VAE-based decoding, effectively improving decoding efficiency.

[0215] Figure 12 This is a schematic diagram of the structure of a decoding device 1200 provided in an embodiment of the present application. The decoding device 1200 can be implemented by software, hardware, or a combination of both to become part or all of a decoding end device. The decoding end device can be Figure 1 Source device shown. Figure 12 The device 1200 includes: a first determination module 1201, a grouping module 1202, a second determination module 1203 and a reconstruction module 1204.

[0216] A first determining module 1201 is configured to determine, based on a bitstream, a priori features of each feature point among a plurality of feature points of an image to be decoded;

[0217] A grouping module 1202 is configured to divide the plurality of feature points into a plurality of groups based on a specified value;

[0218] A second determining module 1203 is configured to sequentially determine, based on the prior features of the plurality of feature points, a first image feature of each group of feature points in the plurality of groups; wherein the step of determining the first image feature of any group of feature points comprises: concurrently determining a probability distribution of each feature point in the group, and parsing the first image feature of each feature point in the group from the bitstream based on the probability distribution of each feature point in the group;

[0219] The reconstruction module 1204 is configured to reconstruct an image based on the first image features of the plurality of feature points.

[0220] Optionally, the multiple feature points include the first feature point, and the second determining module 1203 includes:

[0221] a first processing submodule, configured to determine, if the first feature point is not the first feature point among the plurality of feature points, surrounding information of the first feature point from the decoded first image features of each feature point, the first feature point being one feature point in any one of the groups;

[0222] A second processing submodule is configured to input surrounding information of the first feature point into a context model to obtain a context feature of the first feature point output by the context model;

[0223] The third processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point and context features of the first feature point.

[0224] Optionally, the surrounding information of the first feature point includes the first image features of the decoded feature points in a neighborhood with the first feature point as the geometric center, the size of the neighborhood is determined based on the size of the receptive field used by the context model, and the surrounding information includes at least the first image features of n feature points around the first feature point, where n is greater than or equal to 4.

[0225] Optionally, the multiple feature points include the first feature point, and the second determining module 1203 includes:

[0226] The fourth processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point if the first feature point is the first feature point among the multiple feature points.

[0227] Optionally, the specified value is based on the size of the receptive field used by the context model;

[0228] The grouping module 1202 includes:

[0229] A first determining submodule is configured to determine a slope based on a specified value, where the slope indicates the degree of inclination of the straight line containing the feature points grouped into the same group;

[0230] The division submodule is used to divide the multiple feature points into multiple groups based on the slope.

[0231] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0232] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0233] In summary, in this embodiment of the present application, during the decoding process, multiple feature points are divided into multiple groups based on specified values, and the probability distribution is determined in parallel for each feature point in the same group. This can accelerate decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial calculations during VAE-based decoding, effectively improving decoding efficiency.

[0234] It should be noted that the decoding device provided in the above embodiment is merely illustrated by the division of the above functional modules during decoding. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the decoding device provided in the above embodiment and the decoding method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0235] Figure 13 This is a schematic diagram of the structure of a coding device 1300 provided in an embodiment of the present application. The coding device 1300 can be implemented by software, hardware, or a combination of both to become part or all of a coding end device. The coding end device can be Figure 1 The destination device is shown. Figure 13 The device 1300 includes: a first determining module 1301, a grouping module 1302, a first encoding module 1303 and a second encoding module 1304.

[0236] A first determining module 1301 is configured to determine, based on an image to be encoded, a first image feature, a probability distribution, and a first super-prior feature of each feature point among a plurality of feature points of the image;

[0237] A grouping module 1302 is configured to divide the plurality of feature points into a plurality of groups based on a specified value;

[0238] A first encoding module 1303 is configured to encode the first image feature of each of the plurality of groups of feature points into a bitstream in sequence based on the probability distribution of the plurality of feature points;

[0239] The second encoding module 1304 is configured to encode the first super a priori features of the multiple feature points into a bitstream.

[0240] Optionally, the first determining module 1301 includes:

[0241] A first determining submodule, configured to determine a first image feature of the plurality of feature points based on the image;

[0242] The second determination submodule is configured to determine first super-prior features of the plurality of feature points based on the first image features of the plurality of feature points, and to determine a probability distribution of each of the plurality of feature points in parallel.

[0243] Optionally, the multiple feature points include a first feature point, and the second determining submodule is configured to:

[0244] If the first feature point is not the first feature point among the multiple feature points, determining a priori features of the first feature point based on the first image feature of the first feature point, the first feature point being one of the multiple feature points;

[0245] determining peripheral information of the first feature point from the first image features of the plurality of feature points;

[0246] Inputting surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model;

[0247] Based on the priori feature of the first feature point and the context feature of the first feature point, a probability distribution of the first feature point is determined.

[0248] Optionally, the multiple feature points include a first feature point, and the second determining submodule is configured to:

[0249] If the first feature point is the first feature point among the multiple feature points, a probability distribution of the first feature point is determined based on a priori features of the first feature point.

[0250] Optionally, the specified value is determined based on the size of the receptive field used by the context model;

[0251] The grouping module 1302 includes:

[0252] A third determining submodule is configured to determine a slope based on a specified value, where the slope indicates the degree of inclination of the straight line containing the feature points grouped into the same group;

[0253] The division submodule is used to divide the multiple feature points into multiple groups based on the slope.

[0254] Optionally, if the context model uses multiple receptive fields of different sizes, the specified value is determined by the size of the largest receptive field among the multiple receptive fields of different sizes.

[0255] Optionally, the receptive field used by the context model includes a receptive field of size 5*5.

[0256] In summary, in this embodiment of the present application, in order to determine probability distributions in parallel during the decoding process and thus improve decoding efficiency, multiple feature points are divided into multiple groups based on specified values ​​during the encoding process, and the first image feature of each feature point in each group is sequentially encoded into the bitstream. This grouping is then repeated during the decoding process, and probability distributions are determined in parallel for each feature point in the same group, thereby improving decoding efficiency. In other words, this solution can overcome the efficiency bottleneck caused by serial computation during VAE-based decoding, effectively improving decoding efficiency.

[0257] It should be noted that the encoding device provided in the above embodiment is merely illustrated by the division of the above functional modules during encoding. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the encoding device provided in the above embodiment and the encoding method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0258] Figure 14 14 is a schematic block diagram of a coding and decoding apparatus 1400 used in an embodiment of the present application. The coding and decoding apparatus 1400 may include a processor 1401, a memory 1402, and a bus system 1403. The processor 1401 and the memory 1402 are connected via the bus system 1403. The memory 1402 is used to store instructions, and the processor 1401 is used to execute the instructions stored in the memory 1402 to perform the various coding or decoding methods described in the embodiments of the present application. To avoid repetition, a detailed description is not given here.

[0259] In the embodiment of the present application, the processor 1401 may be a central processing unit (CPU). The processor 1401 may also be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0260] The memory 1402 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as the memory 1402. The memory 1402 may include code and data 14021 accessed by the processor 1401 using the bus 1403. The memory 1402 may further include an operating system 14023 and an application 14022, wherein the application 14022 includes at least one program that allows the processor 1401 to execute the encoding or decoding method described in the embodiment of the present application. For example, the application 14022 may include applications 1 to N, which further include an encoding or decoding application (hereinafter referred to as a codec application) that executes the encoding or decoding method described in the embodiment of the present application.

[0261] In addition to the data bus, the bus system 1403 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus system 1403 in the figure.

[0262] Optionally, the codec apparatus 1400 may further include one or more output devices, such as a display 1404. In one example, the display 1404 may be a touch-sensitive display that combines a display with a touch-sensitive unit operable to sense touch input. The display 1404 may be connected to the processor 1401 via a bus 1403.

[0263] It should be noted that the encoding and decoding device 1400 can execute the encoding method in the embodiment of the present application, and can also execute the decoding method in the embodiment of the present application.

[0264] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another (e.g., based on a communication protocol). In this manner, computer-readable media can generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0265] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather are directed to non-transitory tangible storage media. As used herein, disks and optical disks include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0266] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements. In one example, the various illustrative logic blocks, units, and modules in encoder 100 and decoder 200 may be understood as corresponding circuit devices or logic elements.

[0267] The techniques of the embodiments of the present application can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., a chipset). The various components, modules, or units described in the embodiments of the present application are intended to emphasize the functional aspects of the apparatus for performing the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by interoperable hardware units (including one or more processors as described above).

[0268] That is, the above embodiments can be implemented in whole or in part via software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions. When loaded and executed on a computer, the computer instructions fully or partially perform the processes or functions described in the embodiments of this application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid-state drive (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0269] It should be understood that the "plurality" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0270] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A decoding method, characterized in that: The method comprises: Determining, based on the code stream, a priori features of each feature point among a plurality of feature points of the image to be decoded; Determining a slope based on a specified value, and dividing the multiple feature points into multiple groups based on the slope, wherein the specified value is determined based on the size of the receptive field used by the context model, and the slope is used to indicate the degree of inclination of the straight line containing the feature points divided into the same group; wherein, if the context model uses a receptive field of one size, the specified value is consistent with the size of the receptive field used by the context model; and if the context model uses multiple receptive fields of different sizes, the specified value is consistent with the size of the largest receptive field among the multiple receptive fields of different sizes; Based on the prior features of the multiple feature points, first image features of each group of feature points in the multiple groups are sequentially determined; wherein the step of determining the first image features of any group of feature points comprises: determining the probability distribution of each feature point in the any group in parallel, and parsing the first image features of each feature point in the any group from the bitstream based on the probability distribution of each feature point in the any group; wherein the probability distribution of feature points other than the first feature point in the multiple feature points is determined using the context model; The image is reconstructed based on the first image features of the plurality of feature points.

2. The method according to claim 1, wherein The plurality of feature points include a first feature point, and determining a probability distribution of the first feature point includes: If the first feature point is not the first feature point among the plurality of feature points, determining peripheral information of the first feature point from the decoded first image features of each feature point, the first feature point being a feature point in any one of the groups; Inputting surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model; A probability distribution of the first feature point is determined based on the priori feature of the first feature point and the context feature of the first feature point.

3. The method according to claim 2, wherein The surrounding information of the first feature point includes the first image features of the decoded feature points in a neighborhood with the first feature point as the geometric center, the size of the neighborhood is determined based on the size of the receptive field used by the context model, and the surrounding information includes at least the first image features of n feature points surrounding the first feature point, where n is greater than or equal to 4.

4. The method according to claim 1, wherein The plurality of feature points include a first feature point, and determining a probability distribution of the first feature point includes: If the first feature point is the first feature point among the multiple feature points, a probability distribution of the first feature point is determined based on a priori features of the first feature point.

5. A coding method, characterized in that: The method comprises: Determining, based on an image to be encoded, a first image feature, a probability distribution, and a first super-prior feature for each of a plurality of feature points of the image; wherein the probability distribution of a feature point other than the first feature point among the plurality of feature points is determined using a context model; Determining a slope based on a specified value, dividing the multiple feature points into multiple groups based on the slope, the specified value being determined based on the size of a receptive field used by the context model, and the slope being used to indicate the degree of inclination of a straight line containing feature points divided into the same group; wherein, if the context model uses a receptive field of one size, the specified value is consistent with the size of the receptive field used by the context model; and if the context model uses multiple receptive fields of different sizes, the specified value is consistent with the size of a maximum receptive field among the multiple receptive fields of different sizes; Based on the probability distribution of the plurality of feature points, sequentially encoding a first image feature of each of the plurality of groups of feature points into a bitstream; The first super-prior features of the plurality of feature points are encoded into the bitstream.

6. The method according to claim 5, wherein The determining, based on the image to be encoded, a first image feature, a probability distribution, and a first super-prior feature of each feature point among a plurality of feature points of the image includes: determining, based on the image, a first image feature of the plurality of feature points; Based on the first image features of the plurality of feature points, first super-prior features of the plurality of feature points are determined, and a probability distribution of each of the plurality of feature points is determined in parallel.

7. The method according to claim 6, wherein The plurality of feature points include a first feature point, and determining a probability distribution of the first feature point includes: If the first feature point is not the first feature point among the multiple feature points, determining a priori features of the first feature point based on a first image feature of the first feature point, the first feature point being one of the multiple feature points; determining peripheral information of the first feature point from the first image features of the plurality of feature points; Inputting surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model; A probability distribution of the first feature point is determined based on the priori feature of the first feature point and the context feature of the first feature point.

8. The method according to claim 6, wherein The plurality of feature points include a first feature point, and determining a probability distribution of the first feature point includes: If the first feature point is the first feature point among the multiple feature points, a probability distribution of the first feature point is determined based on a priori features of the first feature point.

9. A decoding device, characterized in that: The device comprises: A first determining module, configured to determine, based on a bitstream, a priori features of each feature point among a plurality of feature points of an image to be decoded; a grouping module, configured to divide the plurality of feature points into a plurality of groups based on a specified value; a second determination module, configured to sequentially determine, based on the prior features of the plurality of feature points, a first image feature of each group of feature points in the plurality of groups; wherein the step of determining the first image feature of any group of feature points comprises: determining a probability distribution of each feature point in the group in parallel, and parsing the first image feature of each feature point in the group from the bitstream based on the probability distribution of each feature point in the group; wherein the probability distribution of a feature point other than the first feature point in the plurality of feature points is determined using a context model; a reconstruction module, configured to reconstruct the image based on the first image feature of the plurality of feature points; The specified value is determined based on the size of the receptive field used by the context model; if the context model uses a receptive field of one size, the specified value is consistent with the size of the receptive field used by the context model; if the context model uses multiple receptive fields of different sizes, the specified value is consistent with the size of the largest receptive field among the multiple receptive fields of different sizes; the grouping module includes: A first determining submodule is configured to determine a slope based on the specified value, wherein the slope indicates a degree of inclination of a straight line containing feature points grouped into the same group; A division submodule is configured to divide the plurality of feature points into the plurality of groups based on the slope.

10. The device according to claim 9, wherein The plurality of feature points include a first feature point, and the second determining module includes: a first processing submodule, configured to determine, if the first feature point is not the first feature point among the plurality of feature points, surrounding information of the first feature point from first image features of each decoded feature point, the first feature point being one feature point in any one of the groups; a second processing submodule, configured to input surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model; The third processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point and context features of the first feature point.

11. The device according to claim 10, wherein The surrounding information of the first feature point includes the first image features of the decoded feature points in a neighborhood with the first feature point as the geometric center, the size of the neighborhood is determined based on the size of the receptive field used by the context model, and the surrounding information includes at least the first image features of n feature points surrounding the first feature point, where n is greater than or equal to 4.

12. The device according to claim 9, wherein The plurality of feature points include a first feature point, and the second determining module includes: The fourth processing submodule is configured to determine a probability distribution of the first feature point based on a priori features of the first feature point if the first feature point is the first feature point among the multiple feature points.

13. An encoding device, characterized in that: The device comprises: A first determination module is configured to determine, based on an image to be encoded, a first image feature, a probability distribution, and a first super-prior feature of each feature point among a plurality of feature points of the image; wherein the probability distribution of a feature point other than the first feature point among the plurality of feature points is determined using a context model; a grouping module, configured to divide the plurality of feature points into a plurality of groups based on a specified value; A first encoding module is configured to sequentially encode a first image feature of each of the plurality of groups of feature points into a bitstream based on a probability distribution of the plurality of feature points; A second encoding module, configured to encode the first super-prior features of the plurality of feature points into the bitstream; The specified value is determined based on the size of the receptive field used by the context model; if the context model uses a receptive field of one size, the specified value is consistent with the size of the receptive field used by the context model; if the context model uses multiple receptive fields of different sizes, the specified value is consistent with the size of the largest receptive field among the multiple receptive fields of different sizes; the grouping module includes: a third determining submodule, configured to determine a slope based on the specified value, wherein the slope is used to indicate a degree of inclination of a straight line containing feature points grouped into the same group; A division submodule is configured to divide the plurality of feature points into the plurality of groups based on the slope.

14. The device according to claim 13, wherein The first determining module includes: A first determining submodule, configured to determine a first image feature of the plurality of feature points based on the image; The second determining submodule is configured to determine first super-prior features of the plurality of feature points based on the first image features of the plurality of feature points, and to determine a probability distribution of each of the plurality of feature points in parallel.

15. The device according to claim 14, wherein The plurality of feature points include a first feature point, and the second determining submodule is configured to: If the first feature point is not the first feature point among the multiple feature points, determining a priori features of the first feature point based on a first image feature of the first feature point, the first feature point being one of the multiple feature points; determining peripheral information of the first feature point from the first image features of the plurality of feature points; Inputting surrounding information of the first feature point into the context model to obtain context features of the first feature point output by the context model; A probability distribution of the first feature point is determined based on the priori feature of the first feature point and the context feature of the first feature point.

16. The device according to claim 14, wherein The plurality of feature points include a first feature point, and the second determining submodule is configured to: If the first feature point is the first feature point among the multiple feature points, a probability distribution of the first feature point is determined based on a priori features of the first feature point.

17. A decoding terminal device, characterized in that: The decoding end device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 4.

18. A coding terminal device, characterized in that: The encoding end device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the steps of any one of the methods of claims 5-8.

19. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

20. A computer program product, characterized in that The computer program product comprises instructions, which, when executed, implement the method according to any one of claims 1 to 8.

Images