Coding method and electronic device

Abstract

Embodiments of the present application provide a coding method and an electronic device. The coding method comprises: first, obtaining an image to be encoded. Then, generating a first feature map matrix according to the image to be encoded; and generating an autoregressive weight matrix according to the first feature map matrix. Next, determining a probability distribution corresponding to the first feature map matrix according to the autoregressive weight matrix and the first feature map matrix; and then, encoding the first feature map matrix according to the probability distribution to obtain a code stream. In this way, the probability distribution of the determined first feature map matrix is closer to the real distribution of the first feature map matrix, thereby improving the accuracy of probability estimation, so as to reduce the length of the code stream corresponding to the image to be encoded and improve the rate-distortion performance of the sample.

Description

[0001] This application is a subsequent application based on priority to Chinese patent application filed on November 10, 2021, with application number 202111329486.X and entitled "Encoding / Decoding Method and Electronic Device". Technical Field

[0002] This application relates to the field of data processing, and more particularly to an encoding / decoding method and an electronic device. Background Technology

[0003] AI (Artificial Intelligence) image compression algorithms are based on deep learning and offer better compression results compared to traditional image compression technologies such as JPEG (Joint Photographic Experts Group) and BPG (Better Portable Graphics).

[0004] Currently, after each model of the AI ​​image compression algorithm is pre-trained, the model parameters of each model are fixed, while the images to be compressed are different. Using the same model parameters for feature extraction will inevitably limit the accuracy of entropy estimation, thus affecting the rate-distortion performance of image compression. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an encoding / decoding method and an electronic device. In this encoding method, an autoregressive weight matrix is ​​generated based on the feature map matrix of the image to be encoded. Then, the probability distribution of the feature map matrix is ​​determined based on the autoregressive weight matrix. This ensures that the determined probability distribution of the feature map matrix is ​​closer to the true distribution of the feature map matrix, thereby reducing the bitstream length corresponding to the image to be encoded and improving the rate-distortion performance of the samples.

[0006] In a first aspect, embodiments of this application provide an encoding method, which includes: first, acquiring an image to be encoded; then, generating a first feature map matrix based on the image to be encoded; and generating an autoregressive weight matrix based on the first feature map matrix. Next, based on the autoregressive weight matrix and the first feature map matrix, a probability distribution corresponding to the first feature map matrix is ​​determined. Subsequently, the first feature map matrix is ​​encoded according to the probability distribution to obtain a bitstream. In this way, the determined probability distribution of the first feature map matrix is ​​closer to the true distribution of the first feature map matrix, thereby improving the accuracy of probability estimation, which in turn reduces the bitstream length corresponding to the image to be encoded and improves the rate-distortion performance of the samples.

[0007] For example, the first feature map matrix is ​​a feature map matrix obtained by spatial transformation of the image to be encoded. Specifically, it is a feature map matrix obtained by quantizing the feature map matrix of the image to be encoded input to the output of the encoding network. For example, the first feature map matrix is ​​feature map matrix Y2.

[0008] For example, the first feature map matrix includes a feature map matrix obtained by spatial transformation of the image to be encoded, specifically, a feature map matrix obtained by quantizing the feature map matrix input to the output of the encoding network (e.g., the first feature map matrix is ​​feature map matrix Y2); and a feature map matrix obtained by feature extraction of the image to be encoded, specifically, a feature map matrix obtained by inputting the image to be encoded into other networks (including other encoding networks and non-encoding networks) (e.g., feature map matrix Y6). In this case, the first feature map matrix of the bitstream obtained by encoding according to the probability distribution is: a feature map matrix obtained by spatial transformation of the image to be encoded; specifically, a feature map matrix obtained by quantizing the feature map matrix input to the output of the encoding network.

[0009] For example, the feature map matrix Y2∈R c*h*w In this matrix, "c" refers to the number of channels in the feature map matrix, "h" represents the height of the feature map output by each channel, and "w" represents the width of the feature map output by each channel. In other words, the feature map matrix Y2 can include c feature maps of size h*w, and each feature map can include multiple feature points.

[0010] For example, the autoregressive weight matrix is ​​the autoregressive weight matrix WM.

[0011] For example, the size of the autoregressive convolution kernel is ks*ks, and the autoregressive weight matrix... Among them, “ks” 2 That is, "ks*ks", where "g" represents the number of convolutional kernel weight groups, and "ratio" represents the convolutional kernel channel scaling factor. 2 "*g*ratio" represents the number of channels in the autoregressive weight matrix. In other words, the autoregressive weight matrix WM corresponds to "ks". 2 There are *g*ratio* channels, each channel corresponds to h*w weight maps, and the size of each weight map is ks*ks. Among them, a feature point of a channel in the feature map matrix Y2 corresponds to the weight map of the corresponding position of the feature point in at least one channel of the autoregressive weight matrix WM; that is, each feature point of each feature map in the feature map matrix Y2 corresponds to one or more weight maps.

[0012] For example, the size of the autoregressive convolution kernel is ks1*ks2, and the autoregressive weight matrix WM∈R (c1*g*ratio)*h*wc1 = ks1 * ks2. Here, "g" represents the number of convolutional kernel weight groups, "ratio" represents the convolutional kernel channel scaling factor, and "c1 * g * ratio" represents the number of channels in the autoregressive weight matrix. That is, the autoregressive weight matrix WM corresponds to "c1 * g * ratio" channels, each channel includes h * w weight maps, and each weight map has a size of ks1 * ks2. Here, h * w is the size of the feature map corresponding to each channel in the feature map matrix Y2. A feature point in one channel of the feature map matrix Y2 corresponds to the weight map at the corresponding position in at least one channel of the autoregressive weight matrix WM; that is, each feature point in each feature map of the feature map matrix Y2 corresponds to one or more weight maps.

[0013] For example, the probability distribution is probability distribution PA1.

[0014] For example, the bitstream is bitstream SA.

[0015] According to the first aspect, generating an autoregressive weight matrix based on the first feature map matrix includes: firstly, extracting features from the first feature map matrix to obtain a second feature map matrix; then, generating a weight parameter matrix based on the second feature map matrix; and next, updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix.

[0016] For example, the second feature map matrix is ​​the feature map matrix obtained by extracting features from the feature map matrix Y2, such as feature map matrix Z2.

[0017] For example, the second feature map matrix is ​​the feature map matrix obtained by feature extraction from the feature map matrix Y6.

[0018] For example, the preset autoregressive weight matrix is ​​the preset autoregressive weight matrix RWM.

[0019] For example, the weight parameter matrix is ​​the weight parameter matrix WP.

[0020] According to the first aspect, or any implementation of the first aspect above, an autoregressive weight matrix is ​​generated based on the first feature map matrix, including: convolving the first feature map matrix to generate a weight parameter matrix; and updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix.

[0021] According to the first aspect, or any implementation of the first aspect above, the weight parameter matrix includes weight parameters of multiple channels, and the preset autoregressive weight matrix includes weight maps of multiple channels; updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix includes: filling the weight maps of multiple channels according to a preset filling relationship using weight parameters of multiple channels to obtain the autoregressive weight matrix.

[0022] For example, the weight parameters of one channel are used to fill the weight map of one channel.

[0023] For example, the weight parameter of one channel is used to fill the weight map of multiple channels.

[0024] For example, the weights of multiple channels are used to fill the weight map of a single channel.

[0025] For example, the size of the weight map of one channel of the preset autoregressive weight matrix is ​​ks*ks, where ks*ks is the size of the autoregressive convolution kernel. According to the first aspect, or any implementation of the first aspect above, the weight parameter matrix of one channel has N weight parameters, where ks and N are both positive integers, and N is less than or equal to ks*ks. Following a preset filling relationship, the weight parameters of multiple channels are used to fill the weight maps of multiple channels, resulting in the autoregressive weight matrix. This includes: according to the preset filling relationship, using the N weight parameters of one channel of the weight parameter matrix to fill N positions in the ks*ks weight map of one channel of the preset autoregressive weight matrix, thus obtaining the autoregressive weight matrix.

[0026] For example, a pre-defined autoregressive weight matrix Among them, “ks” 2 That is, "ks*ks", where "g" represents the number of convolutional kernel weight groups, and "ratio" represents the convolutional kernel channel scaling factor. 2 "*g*ratio" represents the number of channels in the preset autoregressive weight matrix RWM. In other words, the preset autoregressive weight matrix RWM corresponds to "ks". 2 There are *g*ratio* channels, each channel corresponding to h*w weight maps, and each weight map has a size of ks*ks. Here, h*w is the size of the feature map corresponding to each channel in the feature map matrix Y2. A feature point in one channel of the feature map matrix Y2 corresponds to the weight map at the corresponding position of that feature point in at least one channel of the autoregressive weight matrix WM; that is, each feature point of each feature map in the feature map matrix Y2 corresponds to one or more weight maps.

[0027] For example, the weight parameter matrix Among them, "(ks 2 / 2)” takes the value (ks) 2 / 2) The value obtained by rounding down, "(ks)" 2 " / 2)*g*ratio" represents the number of channels in the weight parameter matrix. Each channel corresponds to h*w weighted reassemblies, and each reassembly includes N weight parameters, where N is a positive integer and can be determined based on ks. Specifically, a feature point in one channel of the feature map matrix Y2 corresponds to a weighted reassembly at the corresponding position in at least one channel of the weight parameter matrix WP; that is, each feature point in the feature map matrix Y2 corresponds to one or more weighted reassemblies.

[0028] For example, autoregressive weight matrix

[0029] For example, the preset filling relationship can be set as needed, and this application does not limit it.

[0030] For example, when N is less than ks*ks, 0 is used to fill other positions in the weight graph.

[0031] For example, the size of the weight map of one channel of the preset autoregressive weight matrix is ​​ks1*ks2, where ks1*ks2 is the size of the autoregressive convolution kernel. According to the first aspect, or any implementation of the first aspect above, the weight parameter matrix of one channel has N weight parameters, where ks and N are both positive integers, and N is less than or equal to ks*ks. According to a preset filling relationship, the weight parameters of multiple channels are used to fill the weight map of multiple channels, including: according to the preset filling relationship, using the N weight parameters of one channel of the weight parameter matrix to fill N positions in the weight map of the preset autoregressive weight matrix of one channel in the ks*ks format. Here, ks1 and ks2 can be the same or different; this application does not impose any restrictions on this.

[0032] For example, the pre-defined autoregressive weight matrix RWM∈R (c1*g*ratio)*h*w c1 = ks1 * ks2. Here, "g" represents the number of convolutional kernel weight groups, "ratio" represents the convolutional kernel channel scaling factor, and "c1 * g * ratio" represents the number of channels in the preset autoregressive weight matrix RWM. That is, the preset autoregressive weight matrix RWM corresponds to "c1 * g * ratio" channels, each channel includes h * w weight maps, and each weight map has a size of ks1 * ks2. Here, h * w is the size of the feature map corresponding to each channel in the feature map matrix Y2. A feature point in one channel of the feature map matrix Y2 corresponds to the weight map at the corresponding position in at least one channel of the preset autoregressive weight matrix RWM. In other words, each feature point in each feature map of the feature map matrix Y2 corresponds to one or more weight maps.

[0033] For example, the weight parameter matrix WP∈R (c2*g*ratio)*h*w , Round down. Here, "c2*g*ratio" represents the number of channels in the weight parameter matrix. Each channel includes h*w weighted reassemblies, and each weighted reassembly includes N weight parameters, where N is a positive integer and can be determined based on ks1 and ks2. A feature point in one channel of the feature map matrix Y2 corresponds to a weighted reassembly at the corresponding position in at least one channel of the weight parameter matrix WP. In other words, each feature point in the feature map matrix Y2 corresponds to one or more weighted reassemblies.

[0034] For example, the autoregressive weight matrix WM∈R (c1*g*ratio)*h*w .

[0035] For example, the preset filling relationship can be set as needed, and this application does not limit it.

[0036] For example, when N is less than ks1*ks2, 0 is used to fill other positions in the weight graph.

[0037] According to the first aspect, or any implementation of the first aspect above, the probability distribution corresponding to the first feature map matrix is ​​determined based on the autoregressive weight matrix and the first feature map matrix, including: determining the entropy estimation feature based on the autoregressive weight matrix and the first feature map matrix; and determining the probability distribution corresponding to the first feature map matrix based on the entropy estimation feature.

[0038] For example, the entropy estimation feature is the entropy estimation feature phi.

[0039] For example, determining the probability distribution corresponding to the first feature map matrix based on the autoregressive weight matrix and the first feature map matrix includes: determining a first entropy estimation feature based on the autoregressive weight matrix and the first feature map matrix; extracting features from the first feature map matrix to obtain a second feature map matrix, and generating a second entropy estimation feature based on the second feature map matrix; and determining the probability distribution corresponding to the first feature map matrix based on the first entropy estimation feature and the second entropy estimation feature. Wherein, the first entropy estimation feature is the entropy estimation feature phi, and the second entropy estimation feature is the entropy estimation feature psi.

[0040] According to the first aspect, or any implementation thereof, the entropy estimation features are determined based on the autoregressive weight matrix and the first feature map matrix. This includes: performing dimensional transformations on the autoregressive weight matrix and the first feature map matrix, respectively, wherein the dimension of the dimensionally transformed autoregressive weight matrix corresponds to the dimension of the dimensionally transformed first feature map matrix; performing a dot product between the dimensionally transformed autoregressive weight matrix and the dimensionally transformed first feature map matrix to obtain intermediate features; and performing an inverse dimensional transformation on the intermediate features to obtain the entropy estimation features. This reduces the computational complexity of determining the entropy estimation features and improves coding efficiency.

[0041] For example, the autoregressive weight matrix after dimensional transformation is the autoregressive weight matrix WM′.

[0042] For example, performing a dimensionality transformation on the first feature map matrix may include: expanding the first feature map matrix, and then performing a dimensionality transformation on the expanded first feature map matrix to obtain a dimensionally transformed first feature map matrix. The dimensionally transformed first feature map matrix may be feature map matrix Y3.

[0043] For example, the intermediate feature is the intermediate feature phi′, and the entropy estimation feature is the entropy estimation feature phi.

[0044] For example, dimensional transformation and inverse dimensional transformation can both refer to adjusting the number of rows, columns, and dimensions of a matrix while keeping the number of elements unchanged; these can also be called matrix transformations.

[0045] According to the first aspect, or any implementation of the first aspect above, the entropy estimation features are determined based on the autoregressive weight matrix and the first feature map matrix, including: convolving the autoregressive weight matrix and the first feature map matrix to obtain the entropy estimation features.

[0046] According to the first aspect, or any implementation of the first aspect above, the method further includes: encoding the second feature map matrix to obtain a bitstream. In this way, the decoding end can directly obtain the features used to generate the autoregressive weight matrix without regenerating them, thus improving decoding efficiency.

[0047] For example, the bitstream is bitstream SB.

[0048] Secondly, embodiments of this application provide a decoding method, which includes: acquiring a bitstream; decoding feature values ​​corresponding to multiple feature points from the bitstream to obtain a first feature map matrix; wherein, for a current feature point to be decoded: firstly, generating an autoregressive weight matrix corresponding to the current feature point to be decoded based on previously decoded information; then, determining the probability distribution corresponding to the current feature point to be decoded based on the autoregressive weight matrix and previously decoded information; then, decoding the current feature point to be decoded according to the probability distribution to obtain the corresponding feature values; and finally, performing image reconstruction based on the first feature map matrix to output a reconstructed image.

[0049] For example, the bitstream may include: bitstream SA.

[0050] For example, the bitstream may include: bitstream SA and bitstream SB.

[0051] For example, the first feature map matrix is ​​feature map matrix Y2.

[0052] For example, the autoregressive weight matrix is ​​the autoregressive weight matrix WM.

[0053] For example, the probability distribution is probability distribution PA2.

[0054] According to the second aspect, when the bitstream includes bitstream SA and bitstream SB, the method further includes: decoding a second feature map matrix from the bitstream, the second feature map matrix including features obtained by feature extraction from the first feature map matrix; wherein the decoded information includes: the second feature map matrix and the feature values ​​corresponding to the decoded feature points.

[0055] For example, the second feature map matrix is ​​feature map matrix Z2.

[0056] According to the second aspect, or any implementation of the second aspect above, the decoded information includes: the second feature map matrix and the eigenvalues ​​corresponding to the decoded feature points. Based on the decoded information, generating the autoregressive weight matrix corresponding to the current feature point to be decoded includes: generating a weight parameter matrix based on the second feature map matrix; updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix.

[0057] According to the second aspect, or any implementation of the second aspect above, the decoded information includes: the eigenvalues ​​corresponding to the decoded feature points; based on the decoded information, generating the autoregressive weight matrix corresponding to the current feature point to be decoded includes: convolving the eigenvalues ​​corresponding to the decoded feature points to generate a weight parameter matrix; and updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix.

[0058] According to the second aspect, or any implementation of the second aspect above, the weight parameter matrix includes weight parameters of multiple channels, and the preset autoregressive weight matrix includes weight maps of multiple channels; updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix includes: filling the weight maps of multiple channels according to a preset filling relationship using weight parameters of multiple channels to obtain the autoregressive weight matrix.

[0059] For example, the size of the weight map of one channel of the preset autoregressive weight matrix is ​​ks*ks, where ks*ks is the size of the autoregressive convolution kernel. According to the second aspect, or any implementation of the second aspect above, the weight parameter matrix of one channel has N weight parameters, where ks and N are both positive integers, and N is less than or equal to ks*ks. Following a preset filling relationship, the weight parameters of multiple channels are used to fill the weight maps of multiple channels, resulting in the autoregressive weight matrix. This includes: according to the preset filling relationship, using the N weight parameters of one channel of the weight parameter matrix to fill N positions in the ks*ks weight map of one channel of the preset autoregressive weight matrix, thus obtaining the autoregressive weight matrix.

[0060] For example, the size of the weight map of one channel of the preset autoregressive weight matrix is ​​ks1*ks2, where ks1*ks2 is the size of the autoregressive convolution kernel. According to the second aspect, or any implementation of the second aspect above, the weight parameter matrix of one channel has N weight parameters, where ks1, ks2, and N are all positive integers, and N is less than or equal to ks1 and ks2. Following a preset filling relationship, the weight parameters of multiple channels are used to fill the weight maps of multiple channels, including: according to the preset filling relationship, using the N weight parameters of one channel of the weight parameter matrix to fill N positions in the weight map of ks1*ks2 of one channel of the preset autoregressive weight matrix.

[0061] According to the second aspect, or any implementation of the second aspect above, the probability distribution corresponding to the current feature point to be decoded is determined based on the autoregressive weight matrix and the decoded information, including: determining the entropy estimation feature based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points; and determining the probability distribution corresponding to the current feature point to be decoded based on the entropy estimation feature.

[0062] For example, determining the probability distribution corresponding to the current feature point to be decoded based on the autoregressive weight matrix and the decoded information includes: determining a first entropy estimation feature based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points; extracting features from the first feature map matrix to obtain a second feature map matrix, and generating a second entropy estimation feature based on the second feature map matrix; and determining the probability distribution corresponding to the current feature point to be decoded based on the first entropy estimation feature and the second entropy estimation feature. For example, the first entropy estimation feature is the entropy estimation feature phi, and the second entropy estimation feature is the entropy estimation feature psi.

[0063] According to the second aspect, or any implementation of the second aspect above, the entropy estimation features are determined based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points, including: performing dimensional transformations on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points, respectively, wherein the dimension of the dimensionally transformed autoregressive weight matrix corresponds to the dimension of the eigenvalues ​​corresponding to the dimensionally transformed decoded feature points; performing a dot product between the dimensionally transformed autoregressive weight matrix and the eigenvalues ​​corresponding to the dimensionally transformed decoded feature points to obtain intermediate features; and performing an inverse dimensional transformation on the intermediate features to obtain the entropy estimation features.

[0064] According to the second aspect, or any implementation of the second aspect above, the entropy estimation features are determined based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points, including: convolving the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points to obtain the entropy estimation features.

[0065] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0066] Thirdly, embodiments of this application provide an encoder for performing the encoding method in the first aspect or any possible implementation of the first aspect.

[0067] The third aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the third aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0068] Fourthly, embodiments of this application provide a decoder for performing the decoding method in the second aspect or any possible implementation of the second aspect.

[0069] The fourth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the fourth aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, as described above, and will not be repeated here.

[0070] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the electronic device to perform the encoding method in the first aspect or any possible implementation of the first aspect.

[0071] The fifth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the fifth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0072] In a sixth aspect, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, when executed by the processor, causing the electronic device to perform the decoding method in the second aspect or any possible implementation of the second aspect.

[0073] The sixth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the sixth aspect and any implementation thereof are similar to those of the second aspect and any implementation thereof, and will not be repeated here.

[0074] In a seventh aspect, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the encoding method in the first aspect or any possible implementation of the first aspect.

[0075] The seventh aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the seventh aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0076] Eighthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the decoding method in the second aspect or any possible implementation of the second aspect.

[0077] The eighth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the eighth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0078] Ninthly, embodiments of this application provide a computer storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the encoding method in the first aspect or any possible implementation thereof.

[0079] The ninth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the ninth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0080] In a tenth aspect, embodiments of this application provide a computer storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the decoding method in the second aspect or any possible implementation thereof.

[0081] The tenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the tenth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0082] Eleventhly, embodiments of this application provide a computer program product comprising a software program that, when executed by a computer or processor, causes the steps of the method in the first aspect or any possible implementation thereof to be performed.

[0083] The eleventh aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the eleventh aspect and any implementation thereof can be found in the technical effects corresponding to the first aspect and any implementation thereof, as described above, and will not be repeated here.

[0084] In a twelfth aspect, embodiments of this application provide a computer program product comprising a software program that, when executed by a computer or processor, causes the steps of the method in the second aspect or any possible implementation thereof to be performed.

[0085] The twelfth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the twelfth aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, as described above, and will not be repeated here. Attached Figure Description

[0086] Figure 1 A schematic diagram of the system framework structure is shown as an example;

[0087] Figure 2a This is a schematic diagram illustrating the encoding and decoding process as an example.

[0088] Figure 2b This is a schematic diagram illustrating the encoding and decoding process as an example.

[0089] Figure 3 This is a schematic diagram illustrating the decoding sequence as an example.

[0090] Figure 4 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0091] Figure 5 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0092] Figure 6 This is a schematic diagram illustrating the convolution process as an example.

[0093] Figure 7 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0094] Figure 8 This is an illustrative diagram illustrating the compression effect;

[0095] Figure 9 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0096] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0097] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0098] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0099] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0100] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0101] Figure 1 This is a schematic diagram illustrating the system framework structure. It should be understood that... Figure 1 The system shown is merely an example. The system of this application may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0102] Reference Figure 1 For example, the image compression process can be as follows: The image to be encoded is input into the AI ​​encoding unit. After processing by the AI ​​encoding unit, the feature values ​​and probability distributions corresponding to the feature points to be encoded are output. Then, the feature values ​​and probability distributions corresponding to the feature points to be encoded are input into the entropy encoding unit. The entropy encoding unit performs entropy encoding on the feature values ​​of the feature points to be encoded according to the probability distributions corresponding to the feature points to be encoded, and outputs the bitstream.

[0103] Continue to refer to Figure 1 For example, the image decompression process can be as follows: After the entropy decoding unit obtains the bitstream, it can perform entropy decoding on the feature points to be decoded according to the probability distribution predicted by the AI ​​encoding unit for the feature points to be decoded in the bitstream, and output the feature values ​​corresponding to the decoded feature points to the AI ​​decoding unit. The AI ​​decoding unit then performs image reconstruction based on the feature values ​​corresponding to the decoded feature points and outputs the reconstructed image.

[0104] For example, entropy coding refers to coding that does not lose any information during the coding process according to the entropy principle. Entropy coding includes various types, such as Shannon coding, Huffman coding, and arithmetic coding, etc. This application does not limit the types of coding.

[0105] For example, the image to be encoded input to the AI ​​encoding unit can be any one of RAW (raw) image, RGB (Red Green Blue) image, and YUV ("Y" represents luminance, "U" and "V" represent chrominance, "Chroma") image, and this application does not limit it.

[0106] For example, the compression and decompression processes can be performed in the same electronic device or in different electronic devices, and this application does not limit this.

[0107] For example, the AI ​​encoding unit and AI decoding unit can be located in an NPU (Neural Network Processing Unit) or a GPU (Graphics Processing Unit). For example, the entropy encoding unit and entropy decoding unit can be located in a CPU (Central Processing Unit).

[0108] For example, this application can be applied to compressing and decompressing a single image, or to compressing and decompressing multiple frames of images in a video sequence; this application does not limit the scope of the application.

[0109] For example, this application can be applied to a variety of scenarios, such as Huawei Cloud storage (or transmission) of images (or videos), video surveillance, live streaming, etc. This application does not limit these applications.

[0110] In the following description and accompanying figures, feature map matrix Y2 represents the first feature map matrix, and feature map matrix Z2 represents the second feature map matrix.

[0111] Figure 2a This is a schematic diagram illustrating the encoding and decoding process as an example.

[0112] Reference Figure 2a (1) For example, after the encoding end obtains the image to be encoded, it can process the image to be encoded to obtain the feature map matrix Y2; then, it can generate the autoregressive weight matrix WM based on the feature map matrix Y2. Subsequently, according to the feature map matrix Y2 and the autoregressive weight matrix WM, the probability distribution PA1 corresponding to the feature map matrix Y2 is predicted, and then the feature map matrix Y2 is encoded according to the probability distribution PA1 to obtain the bitstream.

[0113] Reference Figure 2a (2) For example, after the decoding end obtains the bitstream, it can decode the feature values ​​corresponding to multiple feature points from the bitstream. These feature values ​​can then form a feature map matrix Y2, which is then reconstructed to obtain the reconstructed image. The decoding process for the current feature point to be decoded in the bitstream can be as follows: Based on the decoded information obtained before the current feature point, an autoregressive weight matrix WM corresponding to the current feature point is generated. Then, based on the decoded information and the autoregressive weight matrix WM, the probability distribution PA2 corresponding to the current feature point is predicted. Then, based on the probability distribution PA2, the current feature point is decoded to obtain the corresponding feature value. By decoding each feature point in the bitstream in this way, the feature value corresponding to each feature point can be obtained.

[0114] in, Figure 2a The implementation process of the encoding and decoding can be found in the following sections. Figure 2b , Figure 4 , Figure 5 and Figure 7 The implementation process described in [the document / document].

[0115] Figure 2b This is a schematic diagram illustrating the encoding and decoding process as an example.

[0116] Reference Figure 2b For example, the AI ​​encoding unit may include: an encoding network, a quantization unit D1, a probability estimation unit C1, an autoregressive unit, an aggregation unit, a super-encoding network, a quantization unit D2, a super-decoding network, and a probability estimation unit C2.

[0117] For example, the AI ​​decoding unit includes: a decoding network, a probability estimation unit C1, an autoregressive unit, an aggregation unit, a super-decoding network, and a probability estimation unit C2.

[0118] For example, the entropy coding unit includes: entropy coding unit A1 and entropy coding unit B1.

[0119] For example, the entropy decoding unit includes: entropy decoding unit A2 and entropy decoding unit B2.

[0120] For example, the AI ​​encoding unit and the AI ​​decoding unit can be jointly trained so that each network and unit in the AI ​​encoding unit and the AI ​​decoding unit learns the corresponding parameters. For example, the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​encoding unit can be shared with the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​decoding unit. For example, the parameters learned by the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​encoding unit are the same as those learned by the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​decoding unit. For example, the computing power corresponding to the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​encoding unit is the same as that corresponding to the probability estimation unit C1, autoregressive unit, aggregation unit, super-decoding network, and probability estimation unit C2 in the AI ​​decoding unit.

[0121] For example, an encoding network can be used to perform spatial transformations on an image to be encoded, changing the image to another space. For example, the encoding network can be a convolutional neural network.

[0122] For example, a hypercoding network can be used to extract features. For example, a hypercoding network can be a convolutional neural network.

[0123] For example, quantization units (including quantization unit D1 and quantization unit D2) can be used to perform quantization processing.

[0124] For example, probability estimation unit C1 can be used for probability estimation and output a probability distribution. Optionally, probability estimation unit C1 can be a Gaussian probability estimation unit.

[0125] For example, probability estimation unit C2 can be used for probability estimation and output a probability distribution. Optionally, probability estimation unit C2 can be a discrete probability estimation unit.

[0126] For example, the entropy coding unit A1 can be used to encode according to the probability distribution determined by the probability estimation unit C1, thereby reducing the statistical redundancy of the output features.

[0127] For example, the entropy coding unit B1 can be used to encode according to the probability distribution determined by the probability estimation unit C2, thereby reducing the statistical redundancy of the output features.

[0128] For example, the entropy decoding unit A2 can be used to decode according to the probability distribution determined by the probability estimation unit C1.

[0129] For example, the entropy decoding unit B2 can be used to decode based on the probability distribution determined by the probability estimation unit C2.

[0130] For example, a decoding network can be used to perform an inverse spatial transformation on the information obtained from entropy decoding, outputting a reconstructed image. For example, the decoding network can be a convolutional neural network.

[0131] For example, a super-decoding network can be used to determine features associated with entropy estimation. For example, a super-decoding network can be a convolutional neural network.

[0132] For example, an autoregressive unit can be used to generate an autoregressive weight matrix based on sample association features; then, based on the autoregressive weight matrix, entropy estimation features can be determined. Wherein, if the autoregressive unit is a unit in an AI encoding unit, the sample association features can refer to any feature related to the image to be encoded. If the autoregressive unit is a unit in an AI decoding unit, the sample association features can include any feature related to the decoded information.

[0133] For example, an aggregation unit is used to convert entropy-estimated features into estimated parameters. For example, the aggregation unit may be a convolutional neural network. Optionally, the estimated parameters may be mean-variance estimated parameters.

[0134] Continue to refer to Figure 2b The encoding process can be as follows:

[0135] For example, the image to be encoded is input into an encoding network, which transforms the image to another space and outputs a feature map matrix Y1. The feature map matrix Y1 is then input into a quantization unit D1, which quantizes the feature map matrix Y1 and outputs a feature map matrix Y2.

[0136] For example, the feature map matrix Y1∈R c*h*w In this context, "c" refers to the number of channels in the feature map matrix, "h" represents the height of the feature map output by each channel, and "w" represents the width of the feature map output by each channel. In other words, the feature map matrix Y1 can include c feature maps of size h*w, and each feature map can include multiple feature points. For example, a feature point in the feature map matrix Y1 can be represented as E1(ci,hi,wi). Here, ci represents the channel where the feature point is located, with a value ranging from 1 to c (inclusive), hi ranges from 1 to h (inclusive), wi ranges from 1 to w (inclusive), and (hi,wi) represents the position of the feature point in the feature map.

[0137] For example, quantization unit D1 can quantize the eigenvalues ​​of each feature point in the feature map matrix Y1 according to a preset quantization step size to obtain the feature map matrix Y2∈R. c*h*w For example, the feature map matrix Y2 may include multiple feature points, and a feature point in the feature map matrix Y2 can be represented as E2(ci,hi,wi).

[0138] For example, on one hand, the feature map matrix Y2 is input into a super-coding network, where features are extracted from Y2 to obtain a feature map matrix Z1. Then, feature map matrix Z1 is input into a quantization unit D2. After quantization processing by D2, feature map matrix Z2 is output. Next, on one hand, feature map matrix Z2 is input into a probability estimation unit C2, where it is processed and outputs the probability distribution PB1 of each feature point in Z2 to an entropy encoding unit B1. On the other hand, feature map matrix Z2 is input into entropy encoding unit B1. Entropy encoding unit B1 encodes feature map matrix Z2 according to probability distribution PB1 and outputs a bitstream SB to an entropy decoding unit B2. Then, probability estimation unit C2 predicts the probability distribution PB2 of the feature points to be decoded in bitstream SB and inputs PB2 into entropy decoding unit B2. Subsequently, entropy decoding unit B2 decodes the feature points to be decoded in bitstream SB according to probability distribution PB2 and outputs feature map matrix Z2 to the super-decoding network. After the super-decoding network obtains the feature map matrix Z2, it can convert the feature map matrix Z2 into the entropy estimation feature psi, and input the entropy estimation feature psi into the aggregation unit.

[0139] For example, on the other hand, the feature map matrix Y2 can be input into the autoregressive unit, and the sample association features can be input into the autoregressive unit. The feature map matrix Y2 and the sample association features are processed by the autoregressive unit, and the entropy estimation feature phi is output to the aggregation unit.

[0140] For example, the aggregation unit can transform based on the entropy estimation features phi and psi, and output the estimated parameters to the probability estimation unit C1. Then, the probability estimation unit C1 can predict the probability distribution PA1 corresponding to each feature point in the feature map matrix Y2 based on the estimated parameters, and input the probability distribution PA1 to the entropy encoding unit A1.

[0141] For example, the entropy coding unit A1 can encode each feature point in the feature map matrix Y2 according to the probability distribution PA1 and output the bit stream SA. Thus, the encoding of the image to be encoded is completed.

[0142] It should be noted that after encoding the image to be encoded, both the bitstream SA obtained by encoding the feature map matrix Y2 and the bitstream SB obtained by encoding the feature map matrix Z2 can be sent to the entropy decoding unit A2. Alternatively, during decoding, the entropy decoding unit A2 can obtain both bitstream SA and bitstream SB.

[0143] Continue to refer to Figure 2b The decoding process can be as follows:

[0144] For example, the entropy decoding unit A2 first decodes the feature map matrix Z2 from the bitstream SB and sends the feature map matrix Z2 to the super decoding network. Then, the super decoding network converts the feature map matrix Z2 into entropy estimation features psi and outputs them to the aggregation unit.

[0145] For example, the bitstream SA contains the encoding information of each feature point in the feature map matrix Y2. The entropy decoding unit A2 decodes the encoding information of each feature point in the bitstream SA to obtain the feature value corresponding to each feature point, thereby obtaining the feature map matrix Y2.

[0146] For example, for each current feature point to be decoded: the entropy decoding unit A2 can input the feature value corresponding to the decoded feature point into the autoregressive unit, and input the sample association features into the autoregressive unit. The autoregressive unit processes the feature value corresponding to the decoded feature point and the sample association features, and outputs the entropy estimation feature phi to the aggregation unit. Then, the aggregation unit transforms the entropy estimation features phi and psi, and outputs the estimated parameters to the probability estimation unit C1. Next, the probability estimation unit C1 can predict the probability distribution PA2 corresponding to the current feature point to be decoded based on the estimated parameters, and input the probability distribution PA2 into the entropy decoding unit A2. Then, the entropy encoding unit A2 can decode the current feature point to be decoded according to the probability distribution PA2 corresponding to the feature point to be decoded, and obtain the corresponding feature value. In this way, by repeating the above steps, the entropy decoding unit A2 can decode the bitstream SA, output the feature map matrix Y2 to the decoding network, and perform an inverse transformation on the feature map matrix Y2 through the decoding network to obtain the reconstructed image.

[0147] For example, the entropy decoding unit A2 can perform parallel decoding or serial decoding of feature points from different channels. For example, the entropy decoding unit A2 can perform serial decoding or parallel decoding of feature points from the same channel; this application does not impose any limitations on this. This application uses the serial decoding of feature points from the same channel by the entropy decoding unit A2 as an example for illustrative purposes.

[0148] Figure 3 This is a schematic diagram illustrating the decoding sequence as an example.

[0149] Reference Figure 3 Exemplary Figure 3 Here is an example of a feature map for one channel in the feature map matrix Y2. The feature map is 10*10 in size, where each square represents a feature point. For example, the entropy decoding unit A2 can be configured according to... Figure 3The feature points in the feature map matrix Y2 are decoded sequentially, starting from the first row and proceeding from left to right. Once all feature points in the first row have been decoded, the feature points in the second row can be decoded from left to right, and so on, until all features in the feature map matrix Y2 have been decoded.

[0150] It should be noted that the entropy decoding unit A2 in this application can also be derived from... Figure 3 Other decoding sequences besides those for each feature point can be set sequentially according to requirements, and this application does not impose any restrictions on them.

[0151] It should be noted that during the encoding process, the feature map matrix Y1 can also be input into the supercoding network, and the feature map matrix Z2 can be obtained through the supercoding network and the quantization unit D2. This application does not impose any restrictions on this.

[0152] It should be noted that, Figure 2b The network or unit in the dashed box on the right can also be other networks or other units, and can be set according to requirements. This application does not impose any restrictions on this.

[0153] It should be noted that the AI ​​encoding unit, AI decoding unit, entropy encoding unit, and entropy decoding unit of this application may also include other networks and units for generating other entropy estimation features, and then input the other entropy estimation features into the aggregation unit, which generates estimation parameters based on the entropy estimation feature phi, the entropy estimation feature psi, and other entropy estimation features. This application does not impose any restrictions on this.

[0154] It should be noted that the AI ​​encoding unit, AI decoding unit, entropy encoding unit, and entropy decoding unit in this application may also exclude [certain components]. Figure 2b The network and cells in the dashed box on the right can be set according to requirements, and this application embodiment does not limit them.

[0155] Figure 4 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0156] Reference Figure 4 For example, relative to Figure 2b In other words, Figure 4 Not included Figure 2b The network and units are shown in the dashed box on the right. This way, during encoding and decoding, there is no need to generate the entropy estimation feature psi, and the aggregation unit only needs to generate the estimated parameters based on the entropy estimation feature phi.

[0157] This application is based on Figure 2b The system framework shown is used as an example for illustration.

[0158] The following example illustrates this concept using sample association features as the output features of the super-decoding network.

[0159] Figure 5 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0160] Reference Figure 5 For example, an autoregressive unit may include: a feature deformation unit, a weight generation unit, and an aggregation deformation unit. It should be understood that... Figure 5 The autoregressive unit shown is just one example of an autoregressive unit, and an autoregressive unit may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. Figure 5 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0161] For example, a feature deformation unit can be used to transform the dimensions of a feature. Specifically, it can be used to expand and transform the dimensions of a feature matrix. For example, dimension transformation can refer to adjusting the number of rows, columns, and dimensions of a matrix while keeping the number of elements unchanged.

[0162] For example, a weight generation unit can be used to generate an autoregressive weight matrix of an autoregressive convolution kernel.

[0163] For example, aggregate deformation units can be used to transform the dimensions of features.

[0164] For example, the channel splitting unit can be used to split the features output by each output channel of the super decoding network at the channel level.

[0165] Reference Figure 5 For example, the encoding process can be as follows:

[0166] For example, after the image to be encoded is processed by the encoding network and quantization unit D1, the output feature map matrix Y2 is sent to the supercoding network. Then, after the feature map matrix Y2 is processed by the supercoding network, quantization unit D2, probability estimation unit C2, entropy encoding unit B1, entropy decoding unit B2, and super decoding network, the entropy estimation associated features can be output. This can be referred to the description of the encoding process above, and will not be repeated here.

[0167] For example, the number of output channels of the super-decoding network can be represented by cp, where cp is a positive integer. Entropy-estimated associated features can be input to the channel splitting unit, which splits the entropy-estimated associated features into channels and outputs the entropy-estimated features psi and the weight parameter matrix WP. The entropy-estimated features psi correspond to cp1 channels, and the weight parameter matrix WP corresponds to cp2 channels, where cp1 + cp2 = cp, and cp1 and cp2 are both positive integers.

[0168] For example, cp1 can be determined based on c, where cp1 = n * c, and n can be a positive number, set according to requirements. This application does not impose any restrictions on this. For instance, if c = 192 and n can be set to 2, then cp1 = 384.

[0169] For example, the autoregressive convolution kernel size is ks*ks. After training the autoregressive unit, the preset autoregressive weight matrix of the autoregressive unit can be obtained. Among them, “ks” 2 That is, "ks*ks", where "g" represents the number of convolutional kernel weight groups, and "ratio" represents the convolutional kernel channel scaling factor. 2 "*g*ratio" represents the number of channels in the preset autoregressive weight matrix. In other words, the preset autoregressive weight matrix RWM corresponds to "ks". 2 There are *g*ratio* channels, each channel corresponds to h*w weight maps, and the size of each weight map is ks*ks. Among them, a feature point of a channel in the feature map matrix Y2 corresponds to the weight map of the corresponding position of the feature point in at least one channel of the preset autoregressive weight matrix RWM; that is, each feature point of each feature map in the feature map matrix Y2 corresponds to one or more weight maps.

[0170] For example, the values ​​of ks, g, and ratio can be preset before training the autoregressive unit; then the autoregressive unit can be trained according to the preset values ​​of "ks", "g", and "ratio".

[0171] For example, the autoregressive unit can update the preset autoregressive weight matrix RWM based on the weight parameter matrix WP to obtain the autoregressive weight matrix WM; wherein the preset autoregressive weight matrix RWM and the autoregressive weight matrix WM have the same size. Then, the entropy estimation feature phi can be generated by convolving the feature map matrix Y2 with the autoregressive weight matrix WM.

[0172] For example, the weight parameter matrix Among them, "(ks 2" / 2)*g*ratio" represents the number of output channels used by the supercoding network to output the weight parameter matrix WP. Each channel corresponds to h*w weight reassemblies, and each weight reassembly includes N weight parameters, where N is a positive integer and can be determined by ks. Specifically, a feature point in one channel of the feature map matrix Y2 corresponds to a weight reassembly at the corresponding position in at least one channel of the weight parameter matrix WP; that is, each feature point in each feature map of the feature map matrix Y2 corresponds to one or more weight reassemblies.

[0173] It should be noted that the number of channels in the weight parameter matrix WP can also be other values ​​less than or equal to ks. 2 The value of *g*ratio can be set according to requirements. This application does not limit the number of channels in the weight parameter matrix WP. The smaller cp2 is, the fewer parameters and the lower the computational power of the super-decoding network.

[0174] For example, the autoregressive convolution kernel size is ks1*ks2. After training the autoregressive unit, the preset autoregressive weight matrix RWM∈R can be obtained. (c1*g*ratio)*h*w c1 = ks1 * ks2. Here, "g" represents the number of convolutional kernel weight groups, "ratio" represents the convolutional kernel channel scaling factor, and "c1 * g * ratio" represents the number of channels in the preset autoregressive weight matrix RWM. That is, the preset autoregressive weight matrix RWM corresponds to "c1 * g * ratio" channels, each channel includes h * w weight maps, and each weight map has a size of ks1 * ks2. In this case, a feature point in one channel of the feature map matrix Y2 corresponds to the weight map at the corresponding position in at least one channel of the preset autoregressive weight matrix RWM; that is, each feature point in each feature map of the feature map matrix Y2 corresponds to one or more weight maps.

[0175] For example, the values ​​of ks1, ks2, g, and ratio can be preset before training the autoregressive unit; then the autoregressive unit can be trained according to the preset values ​​of "ks1", "ks2", "g", and "ratio".

[0176] For example, the autoregressive unit can update the preset autoregressive weight matrix RWM based on the weight parameter matrix WP to obtain the autoregressive weight matrix WM; wherein the preset autoregressive weight matrix RWM and the autoregressive weight matrix WM have the same size. Then, the entropy estimation feature phi can be generated by convolving the feature map matrix Y2 with the autoregressive weight matrix WM.

[0177] For example, the weight parameter matrix Round down. Here, "c2*g*ratio" represents the number of channels in the weight parameter matrix WP, i.e., c2*g*ratio = cp2. Each channel includes h*w weighted reassemblies, and each weighted reassembly includes N weight parameters, where N is a positive integer and can be determined based on ks1 and ks2. A feature point in one channel of the feature map matrix Y2 corresponds to a weighted reassembly at the corresponding position in at least one channel of the weight parameter matrix WP; that is, each feature point in each feature map of the feature map matrix Y2 corresponds to one or more weighted reassemblies.

[0178] It should be noted that the number of channels in the weight parameter matrix WP can be any number less than c1*g*ratio, or it can be equal to c1*g*ratio. The specific number can be set according to requirements; this application does not impose a limit on the number of channels in the weight parameter matrix WP. The smaller cp2 is, the smaller the number of parameters and the lower the computational power of the super-decoding network.

[0179] Figure 6 This is a schematic diagram illustrating the convolution process as an example.

[0180] Reference Figure 6 Exemplary Figure 6 (1) is the feature map of one channel in the feature map matrix Y2. The size of the feature map is 10*10 (that is, h=10, w=10). Figure 6 (2) is a weighted graph with a size of 5*5 (that is, ks1=ks2=ks=5).

[0181] For example, the autoregressive unit will Figure 6 (1) Feature point A (e.g.) Figure 6 (1) The 24 feature points surrounding the black square (as shown in the 24 squares in region B) are... Figure 6 (2) Except for the center point C (e.g.) Figure 6 (2) The weights at the 24 positions outside the black square are convolved to determine the entropy estimation feature phi of feature point A. However, during the decoding process, among the 24 feature points surrounding feature point A, some feature points may be decoded before feature point A, while others may be decoded after feature point A. Therefore, during convolution, the decoded feature points among the 24 feature points surrounding feature point A are convolved with the weights at the corresponding positions in the weight map.

[0182] For example, if the size of the autoregressive convolution kernel is ks*ks, then the number N of weight parameters in a weighted array corresponding to each feature point in the feature map matrix Y2 can be less than or equal to ks*ks. For instance, N can be the number of partially or completely decoded feature points within the ks*ks region centered on that feature point.

[0183] For example, if the size of the autoregressive convolution kernel is ks1*ks2, then the number N of weight parameters in a weighted array corresponding to each feature point in the feature map matrix Y2 can be less than or equal to ks1*ks2. For instance, N can be the number of partially or completely decoded feature points within the ks1*ks2 region centered on that feature point.

[0184] For example, after the weight parameter matrix WP split by the channel splitting unit is input into the weight generation unit, the weight generation unit can update the preset autoregressive weight matrix RWM according to the weight parameter matrix WP.

[0185] In one possible approach, a feature point in one channel of the feature map matrix Y2 corresponds to a weight map of the corresponding position in one channel of the preset autoregressive weight matrix RWM, and a weighted array of the corresponding position in one channel of the weight parameter matrix WP (that is, a feature point in the feature map matrix Y2 corresponds to a weight map and a weighted array). In this case, for a feature point in the feature map matrix Y2, according to a preset filling relationship, N weight parameters from the weighted array corresponding to that feature point can be used to fill N positions in the weight map corresponding to that feature point. If the size of the autoregressive convolution kernel is ks*ks, then when N is less than ks*ks, 0 is used to fill the other positions in the weight map. If the size of the autoregressive convolution kernel is ks1*ks2, then when N is less than ks1*ks2, 0 is used to fill the other positions in the weight map.

[0186] For example, the preset filling relationship can be set as needed, and this application does not limit it.

[0187] For example, if the size of the autoregressive convolution kernel is ks*ks, the preset padding relationship can include: it can be pre-set according to requirements, for a feature point in the feature map matrix Y2, using the k-th (k is 1 to ks) weighted combination corresponding to that feature point. 2 Integers between 2 and 1 (inclusive) 2 The p-th weight parameter, rounded down to the nearest integer, is used to fill the p-th (p is 1 to ks) of the weight map corresponding to that feature point. 2 Integers between 2 and 1 (inclusive) 2 The number of positions is not limited to the integers ( / 2 rounded down).

[0188] For example, the size of the autoregressive convolution kernel is ks1*ks2, and the preset filling relationship may include: for a feature point in the feature map matrix Y2, the kth weight parameter (k is an integer between 1 and c2, including 1 and c2) in a weighted group corresponding to the feature point is used to fill the pth position (p is an integer between 1 and c1, including 1 and c1) of a weighted map corresponding to the feature point. This application does not limit this.

[0189] For example, suppose that entropy decoding unit A2 is according to Figure 3 If the feature points are decoded in the order shown, then for a feature point in the feature map matrix Y2, the method of filling the N positions in the weight map corresponding to that feature point in the preset autoregressive weight matrix RWM with N weight parameters from a weighted reassembly corresponding to that feature point in the weight parameter matrix can be as follows: Since the entropy decoding unit A2 is based on... Figure 3 The feature points are decoded in the order shown, therefore region B contains only the 12 feature points preceding feature point A (e.g., ...). Figure 6 (1) The gray squares shown in the middle are the decoded feature points. Therefore, when estimating the feature phi of feature point A by predicting its entropy, the 12 feature points preceding feature point A in region B (such as...) can be used. Figure 6 (1) As shown in the gray square), the weights of the 12 positions before the center position C of the weight map (as shown in the gray square). Figure 6 (2) As shown by the gray square, convolution can be performed. In this case, if the size of the autoregressive convolution kernel is ks*ks, each weight set can include ks. 2 / 2 weight parameters, where when ks 2 When / 2 is not an integer, it can be rounded down. If the size of the autoregressive convolution kernel is ks1*ks2, each weighted group can include c2 weight parameters.

[0190] In other words, when the entropy decoding unit A2 is according to Figure 3 When decoding feature points in the order shown, the N weight parameters in a weighted array corresponding to a feature point in the feature map matrix Y2 can be used to fill the N positions before the center position of the weight map corresponding to that feature point, and 0 can be used to fill the N positions after the center position of the weight map corresponding to that feature point.

[0191] In one possible approach, a feature point in one channel of the feature map matrix Y2 corresponds to a weight map of the corresponding position in multiple channels of a preset autoregressive weight matrix RWM, and a weight set of the corresponding position in one channel of the weight parameter matrix WP (that is, a feature point in the feature map matrix Y2 corresponds to multiple weight maps and one weight set). In this case, for a feature point in the feature map matrix Y2, multiple weight maps corresponding to that feature point can be filled according to a preset filling relationship using a weight set corresponding to that feature point. For example, for each weight map corresponding to that feature point, N weight parameters from a weight group corresponding to that feature point can be used for filling. If the size of the autoregressive convolution kernel is ks*ks, then when N is less than ks*ks, 0 is used to fill other positions in the weight map. If the size of the autoregressive convolution kernel is ks1*ks2, then when N is less than ks1*ks2, 0 is used to fill other positions in the weight map.

[0192] In one possible approach, a feature point in one channel of the feature map matrix Y2 corresponds to a weight map of the corresponding position in one channel of the preset autoregressive weight matrix RWM, and a weighted array of the corresponding positions in multiple channels of the weight parameter matrix WP (that is, one feature point in the feature map matrix Y2 corresponds to one weight map and multiple weighted arrays). In this case, for a feature point in the feature map matrix Y2, multiple weighted arrays corresponding to that feature point can be used to fill a weight map corresponding to that feature point according to a preset filling relationship. For example, multiple weighted arrays corresponding to that feature point can be merged into one weighted array (including N weight parameters); for instance, the weight parameters at corresponding positions in multiple weighted arrays can be weighted to obtain the weight parameter for each position. Then, the N weight parameters from the merged weighted array are used to fill N positions in the weight map corresponding to that feature point. If the size of the autoregressive convolution kernel is ks*ks, then when N is less than ks*ks, 0 is used to fill other positions in the weight map. If the size of the autoregressive convolution kernel is ks1*ks2, then when N is less than ks1*ks2, 0 is used to fill other positions in the weight graph.

[0193] For example, to effectively control the computational cost and parameter count of the autoregressive unit, the autoregressive weight matrix WM can be dimensionally transformed to obtain the autoregressive weight matrix WM′; and the feature map matrix Y2 can be dimensionally transformed to obtain the feature image matrix Y3. The feature image matrix Y3 and the autoregressive weight matrix WM′ have corresponding dimensions. Then, the autoregressive weight matrix WM′ and the feature image matrix Y3 are multiplied by a dot to obtain the intermediate feature phi′, and then the intermediate feature phi′ is inversely dimensionally transformed to obtain the entropy estimation feature phi.

[0194] For example, the size of the autoregressive convolution kernel is ks*ks. For instance, the feature map matrix Y2 can be input into the feature deformation unit of the autoregressive unit, which can perform dimensionality transformation on the feature map matrix Y2 to obtain the feature map matrix. The feature map matrix Y3 is a five-dimensional matrix. Specifically, the feature deformation unit expands the feature map matrix Y2 to obtain the feature map matrix Y3. Then, a matrix transformation is performed on the feature map matrix Y4 to obtain the feature map matrix Y3. In feature map matrix Y3, "g" corresponds to one dimension. "1" corresponds to one dimension, "ks" corresponds to one dimension. 2 "h" corresponds to one dimension, "w" corresponds to one dimension, and "h" corresponds to one dimension. Furthermore, the autoregressive weight matrix WM can be dimensionally transformed to obtain the autoregressive weight matrix WM′. The autoregressive weight matrix WM′ is a five-dimensional matrix. In the autoregressive weight matrix WM′, "g" corresponds to one dimension, "1" corresponds to one dimension, "ratio" corresponds to one dimension, and "ks" corresponds to another dimension. 2 "h" corresponds to one dimension, "w" corresponds to one dimension.

[0195] For example, the "g" dimension of the feature map matrix Y3 corresponds to the "g" dimension of the autoregressive weight matrix WM′. The feature map matrix Y3... This dimension corresponds to the "1" dimension of the autoregressive weight matrix WM′, that is to say, Each feature map matrix Y3 corresponds to an autoregressive weight matrix WM′. The "1" dimension of the feature map matrix Y3 corresponds to the "ratio" dimension of the autoregressive weight matrix WM′. 2 "This dimension, along with the autoregressive weight matrix WM′'s "ks" 2 This dimension corresponds to the "h" dimension of the feature map matrix Y3, which corresponds to the "h" dimension of the autoregressive weight matrix WM′. Similarly, the "w" dimension of the feature map matrix Y3 corresponds to the "w" dimension of the autoregressive weight matrix WM′.

[0196] For example, the feature map matrix Y3 can be multiplied by the autoregressive weight matrix WM′ to obtain the intermediate feature phi′. The intermediate feature phi′ is a five-dimensional matrix. Since the correlation between feature points in the Y2 channel of the feature map matrix is ​​weak, this removes the coupling between channels, reducing computational power and the number of parameters, while not affecting the model's entropy estimation feature prediction ability.

[0197] For example, the size of the autoregressive convolution kernel is ks1*ks2. For instance, the feature map matrix Y2 can be input into the feature deformation unit of the autoregressive unit, which can perform dimensionality transformation on the feature map matrix Y2 to obtain the feature map matrix Y3. The feature map matrix Y3 is a five-dimensional matrix. Specifically, the feature deformation unit can expand the feature map matrix Y2 to obtain the feature map matrix Y4∈R. c*c1*h*w Then, the feature map matrix Y4 is transformed to obtain the feature map matrix Y3. In feature map matrix Y3, "g" corresponds to one dimension. Each dimension corresponds to a different dimension: "1" corresponds to a different dimension, "c1" corresponds to a different dimension, "h" corresponds to a different dimension, and "w" corresponds to a different dimension.

[0198] For example, the autoregressive weight matrix WM can be dimensionally transformed to obtain the autoregressive weight matrix WM′, where WM′∈R. g*1*ratio*c1*h*w The autoregressive weight matrix WM′ is a five-dimensional matrix. In the autoregressive weight matrix WM′, “g” corresponds to one dimension, “1” corresponds to one dimension, “ratio” corresponds to one dimension, “c1” corresponds to one dimension, “h” corresponds to one dimension, and “w” corresponds to one dimension.

[0199] For example, the "g" dimension of the feature map matrix Y3 corresponds to the "g" dimension of the autoregressive weight matrix WM′. The feature map matrix Y3... This dimension corresponds to the "1" dimension of the autoregressive weight matrix WM′, that is to say, Each feature map matrix Y3 corresponds to an autoregressive weight matrix WM′. The dimension "1" of feature map matrix Y3 corresponds to the "ratio" dimension of autoregressive weight matrix WM′. The dimension "c1" of feature map matrix Y3 corresponds to the "c1" dimension of autoregressive weight matrix WM′. The dimension "h" of feature map matrix Y3 corresponds to the "h" dimension of autoregressive weight matrix WM′. The dimension "w" of feature map matrix Y3 corresponds to the "w" dimension of autoregressive weight matrix WM′.

[0200] For example, the feature map matrix Y3 can be multiplied by the autoregressive weight matrix WM′ to obtain intermediate features. The intermediate feature phi′ is a five-dimensional matrix. Since the correlation between feature points in the Y2 channel of the feature map matrix is ​​weak, this removes the coupling between channels, reducing computational power and the number of parameters, while not affecting the model's entropy estimation feature prediction ability.

[0201] For example, the intermediate feature phi′ can be input into the aggregation deformation unit for aggregation deformation, that is, matrix transformation, to obtain the entropy estimation feature phi∈R. (c*ratio)*h*w The entropy estimation feature is a 3D matrix ("c*ratio" is the number of channels) to facilitate the subsequent processing of the aggregation unit and probability estimation unit C1, generating the probability distribution P1, which is a 3D matrix.

[0202] In this way, for different images to be encoded, an autoregressive weight matrix can be adaptively generated to extract entropy estimation features, which makes the probability distribution predicted for each feature point in the feature map matrix Y2 closer to the true distribution, thereby reducing the bitstream length of the sample and improving the rate-distortion performance of the sample.

[0203] For example, entropy estimation features *phi* and *psi* can be input into an aggregation unit. The aggregation unit performs channel concatenation on the entropy estimation features *phi* and *psi* to obtain concatenated features. Channel concatenation refers to concatenating the entropy estimation features *phi* and *psi* of the same channel; each channel has corresponding entropy estimation features *phi* and *psi*. Then, the aggregation unit performs a convolution operation on the concatenated features corresponding to a channel, outputting the estimated parameters for that channel, such as the mean-variance distribution parameters (μ, σ). For example, (μ, σ) ∈ R... (2*c)*h*w , where the mean μ corresponds to the number of channels c, and the variance σ corresponds to the number of channels c.

[0204] For example, the estimated parameters can be input into the probability estimation unit C1, processed by the probability estimation model C1, and output to the entropy coding unit A1 the probability distribution PA1 corresponding to each feature point in the feature map matrix Y2. The entropy coding unit A1 encodes the feature map matrix Y2 according to the probability distribution PA1 corresponding to each feature point in the feature map matrix Y2 to obtain the code stream SA.

[0205] Reference Figure 5 For example, the decoding process is illustrated using a feature point Q(cj,hj,wj) in the feature map matrix Y2. Here, (cj,hj,wj) represents the position of feature point Q in the feature map matrix Y2. cj is an integer between 1 and c, inclusive. hj is an integer between 1 and h, inclusive. wj is an integer between 1 and w, inclusive.

[0206] For example, the entropy decoding unit A2 first decodes the feature map matrix Z2 from the bitstream SB, and then sends the feature map matrix Z2 to the super decoding network. The super decoding network converts the feature map matrix Z2 into entropy estimation associated features and outputs it to the channel splitting unit. Through the splitting by the channel splitting unit, the entropy estimation feature psi is output to the aggregation unit, and the weight parameter matrix WP is output to the weight generation unit of the autoregressive unit. Specifically, the entropy estimation feature psi output by the channel splitting unit includes the entropy estimation features corresponding to all feature points in the feature map matrix Y2, and the weight parameter matrix WP output by the channel splitting unit includes the weighted reassemblies corresponding to all feature points in the feature map matrix Y2.

[0207] For example, the weight generation unit can select the weight set corresponding to the feature point Q(cj,hj,wj) from the weight parameter matrix WP, update the preset autoregressive weight matrix RWM of the autoregressive unit, and obtain the autoregressive weight matrix WM; and perform dimensional transformation on the autoregressive weight matrix WM, which can be referred to the above description and will not be repeated here. It should be noted that when using the weight parameters of a weight set to fill the corresponding weight map, the weight parameters are used to fill the positions corresponding to the decoded feature points in the weight map.

[0208] For example, before decoding a current feature point Q(cj,hj,wj) to be decoded in the feature map matrix Y2, the eigenvalues ​​corresponding to the already decoded feature points can be input into the feature deformation unit of the autoregressive unit. For example, a feature map matrix Y5, identical to the feature map matrix Y2 but with eigenvalues ​​of 0 for each feature point, can be generated, where Y5∈R. c*h*w Then, the eigenvalues ​​corresponding to the decoded feature points can be filled into the corresponding positions of the feature map matrix Y5 to update the feature map matrix Y5. The updated feature map matrix Y5 is then input into the feature deformation unit of the autoregressive unit. After dimensional transformation by the feature deformation unit, the feature map matrix Y3 is obtained. The dimensionally transformed feature map matrix Y3 is then multiplied by the autoregressive weight matrix WM′ to obtain the intermediate feature phi′. The intermediate feature phi′ is then aggregated and deformed to obtain the entropy estimation feature phi. Here, the entropy estimation feature phi is the entropy estimation feature phi corresponding to feature point Q(cj,hj,wj).

[0209] For example, the entropy estimation features phi and psi are input into the aggregation unit. The aggregation unit can select the entropy estimation unit psi corresponding to feature point Q(cj,hj,wj), and then concatenate the entropy estimation features phi and psi corresponding to feature point Q(cj,hj,wj) to obtain concatenated features. Then, for a single channel, the aggregation unit performs a convolution operation on the concatenated features corresponding to that channel and outputs the estimated parameters corresponding to that channel to the probability estimation unit C1.

[0210] For example, the estimated parameters are processed by the probability estimation unit C1, and the output probability distribution PA2 is sent to the entropy decoding unit A2. Then, the entropy decoding unit A2 can decode the feature point Q(cj,hj,wj) according to the probability distribution PA2. After obtaining the eigenvalues ​​corresponding to Q(cj,hj,wj), the eigenvalues ​​of the feature point Q(ci,hi,wi) can be added to the corresponding positions in the feature map matrix Y5, updating the feature map matrix Y5. Then, the next feature point of feature point Q(cj,hj,wj) is decoded in the same way until all feature points are decoded. In this way, after decoding the last current feature point to be decoded, the eigenvalues ​​corresponding to the last decoded feature point can be used to fill the corresponding positions in the feature map matrix Y5, updating the feature map matrix Y5, thus completing the decoding of the bitstream SA, and the updated feature map matrix Y5 can be stopped from being input to the autoregressive unit. At this point, the updated feature map matrix Y5 is the same as the feature map matrix Y2. This means that after the entropy decoding unit A2 decodes all the features, it can obtain the feature map matrix Y2. Then, the feature map matrix Y2 is input into the decoding network, which performs image reconstruction and outputs the reconstructed image.

[0211] The following example uses the sample association features as the feature map matrix Y2 for illustration.

[0212] Figure 7 This is a schematic diagram illustrating the encoding and decoding process as an example.

[0213] Reference Figure 7 Exemplary Figure 7 The encoding and decoding process can be followed as described above. Figure 5 The description of [the item] will not be repeated here. Among them, Figure 7 and Figure 5 The difference is that, Figure 7 The Chinese Super League decoding network only outputs the entropy estimation feature phi, but does not output the weight parameter matrix WP.

[0214] also, Figure 7 and Figure 5The difference also lies in the different sample association features input to the autoregressive unit, and the corresponding processing of these features also differs, as follows:

[0215] During the encoding process:

[0216] For example, after the image to be encoded is processed by the encoding network and quantization module D1, the feature map matrix Y2 can be split into two inputs to the autoregressive unit. For example, one input of the feature map matrix Y2 can be to the feature deformation unit of the autoregressive unit, and the other input can be to the convolution unit. The feature deformation unit processes the feature map matrix Y2 to obtain the feature map matrix Y3, which can be referenced to the processing of the feature map matrix Y2. Figure 4 The description will not be repeated here.

[0217] For example, the size of the autoregressive convolution kernel is ks*ks. Exemplary convolutional units can be used for MaskConv...

[0218] (Mask Convolution) Operation. After the feature map matrix Y2 is input into the convolutional unit, the convolutional unit can perform Mask Convolution on the feature map matrix Y2 and output the weight parameter matrix. Among them, the weight parameter matrix WP output by the convolutional unit is the same as described above. Figure 4 The weight parameter matrix WP generated by the mid-channel splitting unit has the same dimension. Then, the convolutional unit inputs the weight parameter matrix WP into the weight generation unit, which generates the autoregressive weight matrix WM based on WP. Figure 7 The processing procedure for generating medium-weighted elements and aggregated deformation elements can be referred to the above section on... Figure 5 The description will not be repeated here.

[0219] For example, the size of the autoregressive convolution kernel is ks1*ks2. Exemplarily, the convolutional unit can be used to perform Mask Conv (mask convolution) operations. After inputting the feature map matrix Y2 into the convolutional unit, the convolutional unit can perform Mask Conv on the feature map matrix Y2, outputting a weight parameter matrix WP, WP∈R. (c2*g*ratio)*h*w The weight parameter matrix WP output by the convolutional unit is similar to that described above. Figure 4 The weight parameter matrix WP generated by the mid-channel splitting unit has the same dimension. Then, the convolutional unit inputs the weight parameter matrix WP into the weight generation unit, which generates the autoregressive weight matrix WM based on WP. Figure 7 The processing procedure for generating medium-weighted elements and aggregated deformation elements can be referred to the above section on... Figure 5 The description will not be repeated here.

[0220] During decoding:

[0221] For example, before decoding a current feature point Q(ci,hi,wi) to be decoded in the feature map matrix Y2, the eigenvalues ​​corresponding to the already decoded feature points can be input into the feature deformation unit of the autoregressive unit. For example, a feature map matrix Y5, identical to the feature map matrix Y2, can be generated, where each feature point has an eigenvalue of 0. c *h*w Then, the eigenvalues ​​corresponding to the decoded feature points can be filled into the corresponding positions of the feature map matrix Y5 to update the feature map matrix Y5. The updated feature map matrix Y5 is then input into the feature deformation unit and the convolution unit of the autoregressive unit. After the feature deformation unit performs a dimensionality transformation on the updated feature map matrix Y5, the feature map matrix Y3 is obtained; and after the convolution unit performs a Mask Conv operation on the updated feature map matrix Y5, the weight parameter matrix WP is output to the weight generation unit, which generates the autoregressive weight matrix WM based on the weight parameter matrix WP. The dimensionally transformed feature map matrix 3 is then multiplied by the autoregressive weight matrix WM′ to obtain the intermediate feature phi′. Finally, the intermediate feature phi′ is subjected to an inverse dimensionality transformation to obtain the entropy estimation feature phi.

[0222] It should be noted that during the encoding process, the sample association feature can be the feature map matrix Y6 obtained by feature extraction of the image to be encoded. Specifically, it can be the feature map matrix Y6 obtained by inputting the image to be encoded into other networks (including other encoding networks and non-encoding networks). Alternatively, the sample association feature can be obtained by inputting the feature map matrix Y6 into... Figure 2b Behind the dashed box on the right, the weight parameter matrix WP is the output of the super-decoding network. Sample association features can also be achieved by inputting the feature map matrix Y6 into other networks (except...). Figure 2b The network within the dashed box on the right (the weight parameter matrix WP output by other networks); this application does not impose any restrictions on this.

[0223] Figure 8 This is a schematic diagram illustrating the compression effect as an example.

[0224] Reference Figure 8 Exemplary Figure 8 The vertical axis represents PSNR (Peak Signal to Noise Ratio), measured in dB (decibels), which characterizes image reconstruction quality; a higher PSNR indicates better image reconstruction quality. The horizontal axis represents Bits per pixel (the number of bits used to store each pixel; a smaller value indicates a lower compression rate), measured in BPP (bits per pixel). Figure 8In the figure, the dashed curve represents the relationship between image reconstruction quality and bitstream size in this application, while the solid curve represents the relationship between image reconstruction quality and bitstream size in the prior art. Comparing the two curves, it can be seen that when the bitstream size is the same, the image reconstruction quality of the compression / decompression scheme in this application is higher.

[0225] In one example, Figure 9 A schematic block diagram illustrating an embodiment of the present application shows an apparatus 900. The apparatus 900 may include a processor 901 and a transceiver / transceiver pin 902, and optionally, a memory 903.

[0226] The various components of device 900 are coupled together via bus 904, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 904 in the figure.

[0227] Optionally, the memory 903 can be used for the instructions in the foregoing method embodiments. The processor 901 can be used to execute the instructions in the memory 903, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0228] The device 900 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0229] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0230] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the encoding and decoding method in the above embodiment.

[0231] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the encoding and decoding method in the above embodiment.

[0232] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the encoding and decoding methods in the above-described method embodiments.

[0233] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0234] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0235] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0236] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0237] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0238] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0239] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0240] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0241] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0242] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0243] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An encoding method, characterized in that, The method includes: Obtain the image to be encoded; Generate a first feature map matrix based on the image to be encoded; Generate an autoregressive weight matrix based on the first feature map matrix; Based on the autoregressive weight matrix and the first feature map matrix, the entropy estimation features are determined; Based on the entropy estimation features, determine the probability distribution corresponding to the first feature map matrix; The first feature map matrix is ​​encoded according to the probability distribution to obtain a bitstream.

2. The method according to claim 1, characterized in that, The step of generating an autoregressive weight matrix based on the first feature map matrix includes: Feature extraction is performed on the first feature map matrix to obtain the second feature map matrix; Based on the second feature map matrix, a weight parameter matrix is ​​generated; The preset autoregressive weight matrix is ​​updated using the weight parameter matrix to obtain the autoregressive weight matrix.

3. The method according to claim 1, characterized in that, The step of generating an autoregressive weight matrix based on the first feature map matrix includes: Convolve the first feature map matrix to generate a weight parameter matrix; The preset autoregressive weight matrix is ​​updated using the weight parameter matrix to obtain the autoregressive weight matrix.

4. The method according to claim 2 or 3, characterized in that, The weight parameter matrix includes weight parameters for multiple channels, and the preset autoregressive weight matrix includes weight graphs for multiple channels. The step of updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix includes: According to the preset filling relationship, the weight map of the multiple channels is filled using the weight parameters of the multiple channels to obtain the autoregressive weight matrix.

5. The method according to claim 4, characterized in that, The weight parameter matrix has N weight parameters per channel, and the weight map of the preset autoregressive weight matrix has a size of ks×ks per channel, where ks×ks is the size of the autoregressive convolution kernel, ks and N are both positive integers, and N is less than or equal to ks×ks. The step of filling the weight map of the multiple channels according to a preset filling relationship, using the weight parameters of the multiple channels, to obtain the autoregressive weight matrix includes: According to the preset filling relationship, the N weight parameters of one channel of the weight parameter matrix are used to fill the N positions in the weight graph of one channel of the preset autoregressive weight matrix to obtain the autoregressive weight matrix.

6. The method according to any one of claims 1 to 3 and 5, characterized in that, The step of determining the entropy estimation features based on the autoregressive weight matrix and the first feature map matrix includes: The dimensions of the autoregressive weight matrix and the first feature map matrix are transformed respectively, wherein the dimension of the transformed autoregressive weight matrix corresponds to the dimension of the transformed first feature map matrix. The intermediate feature is obtained by multiplying the autoregressive weight matrix after dimensional transformation and the first feature map matrix after dimensional transformation by a dot product. The intermediate features are subjected to an inverse dimensional transformation to obtain the entropy estimation features.

7. The method according to any one of claims 1 to 3 and 5, characterized in that, The step of determining the entropy estimation features based on the autoregressive weight matrix and the first feature map matrix includes: The entropy estimation features are obtained by convolving the autoregressive weight matrix and the first feature map matrix.

8. The method according to claim 2, characterized in that, The method further includes: The second feature map matrix is ​​encoded to obtain the bitstream.

9. A decoding method, characterized in that, The method includes: Obtain the bitstream, decode the feature values ​​corresponding to multiple feature points from the bitstream, and obtain the first feature map matrix; Specifically, for the current feature point to be decoded: based on the decoded information, an autoregressive weight matrix corresponding to the current feature point to be decoded is generated; based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points, an entropy estimation feature is determined; based on the entropy estimation feature, a probability distribution corresponding to the current feature point to be decoded is determined; based on the probability distribution, the current feature point to be decoded is decoded to obtain the corresponding eigenvalues. Image reconstruction is performed based on the first feature map matrix, and the reconstructed image is output.

10. The method according to claim 9, characterized in that, The method further includes: A second feature map matrix is ​​decoded from the bitstream, the second feature map matrix including features obtained by feature extraction from the first feature map matrix; The decoded information includes the second feature map matrix and the feature values ​​corresponding to the decoded feature points.

11. The method according to claim 10, characterized in that, The step of generating the autoregressive weight matrix corresponding to the currently undecoded feature point based on the decoded information includes: Based on the second feature map matrix, a weight parameter matrix is ​​generated; The preset autoregressive weight matrix is ​​updated using the weight parameter matrix to obtain the autoregressive weight matrix.

12. The method according to claim 9, characterized in that, The decoded information includes: the feature values ​​corresponding to the decoded feature points; The step of generating the autoregressive weight matrix corresponding to the currently undecoded feature point based on the decoded information includes: Convolve the feature values ​​corresponding to the decoded feature points to generate a weight parameter matrix; The preset autoregressive weight matrix is ​​updated using the weight parameter matrix to obtain the autoregressive weight matrix.

13. The method according to claim 11 or 12, characterized in that, The weight parameter matrix includes weight parameters for multiple channels, and the preset autoregressive weight matrix includes weight graphs for multiple channels. The step of updating the preset autoregressive weight matrix using the weight parameter matrix to obtain the autoregressive weight matrix includes: According to the preset filling relationship, the weight map of the multiple channels is filled using the weight parameters of the multiple channels to obtain the autoregressive weight matrix.

14. The method according to claim 13, characterized in that, The weight parameter matrix has N weight parameters for each channel, and the weight map of each channel of the preset autoregressive weight matrix has a size of ks×ks, where ks×ks is the size of the autoregressive convolution kernel, ks and N are both positive integers, and N is less than or equal to ks×ks. The step of filling the weight map of the multiple channels according to a preset filling relationship, using the weight parameters of the multiple channels, to obtain the autoregressive weight matrix includes: According to the preset filling relationship, the N weight parameters of one channel of the weight parameter matrix are used to fill the N positions in the weight graph of one channel of the preset autoregressive weight matrix to obtain the autoregressive weight matrix.

15. The method according to any one of claims 9 to 12, 14, characterized in that, The step of determining the entropy estimation features based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points includes: The dimensions of the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points are transformed respectively, wherein the dimension of the autoregressive weight matrix after the dimension transformation corresponds to the dimension of the eigenvalues ​​corresponding to the decoded feature points after the dimension transformation. The intermediate feature is obtained by multiplying the autoregressive weight matrix after dimensional transformation and the eigenvalues ​​corresponding to the decoded feature points after dimensional transformation. The intermediate features are subjected to an inverse dimensional transformation to obtain the entropy estimation features.

16. The method according to any one of claims 9 to 12, 14, characterized in that, The step of determining the entropy estimation features based on the autoregressive weight matrix and the eigenvalues ​​corresponding to the decoded feature points includes: The entropy estimation features are obtained by convolving the autoregressive weight matrix with the eigenvalues ​​corresponding to the decoded feature points.

17. An encoder, characterized in that, The encoder includes an autoregressive unit and an entropy coding unit; The autoregressive unit is used to determine entropy estimation features based on the autoregressive weight matrix and the first feature map matrix; wherein the first feature map matrix is ​​generated based on the image to be encoded, and the autoregressive weight matrix is ​​generated based on the first feature map matrix; The entropy coding unit is used to encode the first feature map matrix according to the probability distribution corresponding to the first feature map matrix to obtain a bitstream, wherein the probability distribution is determined based on the entropy estimation features.

18. A decoder, characterized in that, The decoder includes an autoregressive unit, an entropy decoding unit, and a decoding network; The autoregressive unit is used to determine the entropy estimation features based on the autoregressive weight matrix corresponding to the current feature point to be decoded and the feature values ​​corresponding to the decoded feature points, wherein the autoregressive weight matrix is ​​generated based on the decoded information; The entropy decoding unit is used to decode the current feature point to be decoded according to the probability distribution corresponding to the current feature point to be decoded, so as to obtain the feature value corresponding to the current feature point to be decoded; wherein, the probability distribution is determined based on the entropy estimation feature; The decoding network is used to reconstruct an image based on a first feature map matrix and output a reconstructed image; wherein the first feature map matrix is ​​obtained based on the feature values ​​corresponding to multiple current feature points to be decoded.

19. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the encoding method according to any one of claims 1 to 8.

20. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the decoding method according to any one of claims 9 to 16.

21. A chip, characterized in that, It includes a connected processor and a memory, the memory being used to store computer instructions; when the processor executes the computer instructions, it causes the chip to perform the encoding method of any one of claims 1 to 8.

22. A chip, characterized in that, It includes a connected processor and a memory, the memory being used to store computer instructions; when the processor executes the computer instructions, it causes the chip to perform the decoding method according to any one of claims 9 to 16.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 16.

24. A computer program product, characterized in that, The computer program product includes a software program that, when executed by a computer or processor, causes the steps of the method described in any one of claims 1 to 16 to be performed.

Images