Encoding and decoding method and electronic equipment

Abstract

The embodiment of the invention provides a coding and decoding method and electronic equipment. The coding method comprises the following steps: acquiring a to-be-coded image; performing feature extraction on the to-be-coded image to obtain a first feature map; a probability distribution parameter corresponding to the first feature map is determined, and the probability distribution parameter comprises a first index of variance; performing first adjustment processing on the first feature map according to the first gain vector to obtain a second feature map; performing second adjustment processing on the first index according to a second gain vector to obtain a second index, the second gain vector being obtained by converting the first gain vector to a logarithm domain, and the second adjustment processing being additive operation; performing entropy coding on the second feature map according to a second index to obtain a code stream; in this way, the calculated amount of variance adjustment can be reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of coding and decoding, and in particular to a coding and decoding method and an electronic device. Background Art

[0002] The AI ​​(Artificial Intelligence) image compression algorithm is based on deep learning and has better compression effect than traditional image compression technologies (such as JPEG (Joint Photographic Experts Group), BPG (Better Portable Graphics), etc.).

[0003] In the end-to-end AI image compression framework, after extracting the features of the original image to obtain the feature map, the probability distribution of the feature map, such as the variance, is determined; then, the variance is adjusted and quantized; then, the probability distribution of each feature point in the feature map is determined based on the quantized variance, and then the feature value of each feature point is entropy encoded based on the probability distribution of each feature point to obtain the bitstream. However, in the prior art, variance adjustment requires a certain amount of computation and is inefficient. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a coding and decoding method and an electronic device. The method can reduce the amount of calculation of variance adjustment and improve the efficiency of variance adjustment.

[0005] In a first aspect, an embodiment of the present application provides a coding method, the method comprising: first, obtaining an image to be coded; then, extracting features of the image to be coded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, the first feature map includes N feature points, N is the product of c, h and w, and c, h and w are positive integers; and determining probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points; then, according to the first A gain vector performs a first adjustment process on a first feature map to obtain a second feature map, wherein the size of the first gain vector is c*1, and the second feature map includes N feature points; and a second adjustment process is performed on the N first indexes according to the second gain vector to obtain N second indexes, wherein the second gain vector is obtained by converting the first gain vector to a logarithmic domain, the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond one-to-one to the N feature points; then, the second feature map is entropy encoded according to the N second indexes to obtain a bit stream.

[0006] That is to say, in the variance adjustment process, the present application converts the multiplication operation between the variance and the gain into an addition operation between the index of the variance and the gain converted to the logarithmic domain; thus, in some cases, the amount of calculation of the variance adjustment can be reduced, the efficiency of the variance adjustment can be improved, and the coding efficiency can be improved while ensuring the compression performance. In other cases, the bit rate overhead can be reduced while reducing the amount of calculation of the variance adjustment and improving the efficiency of the variance adjustment (because the training and testing processes of the entropy estimation network for outputting the variance (i.e., the entropy estimation network of the prior art) are inconsistent, and the training and testing processes of the entropy estimation network for outputting the first index of the variance (i.e., the entropy estimation network of the present application) are consistent, so in some cases, the bit rate overhead can be reduced).

[0007] In addition, the probability distribution table of the present application corresponds to the index of the variance one by one, and then, in the entropy coding process, the present application can directly determine the corresponding probability distribution table according to the quantized second index, and then, the eigenvalues ​​of the feature points in the second feature map can be entropy coded according to the probability distribution table. After the prior art adjusts and quantizes the variance, it is necessary to find the nearest preset variance from multiple preset variances (the probability distribution table corresponds to the preset variance one by one); then, the probability distribution table corresponding to the nearest preset variance is used to entropy code the eigenvalues ​​of the feature points in the second feature map. Among them, the process of finding the nearest preset variance in the prior art requires a certain amount of calculation, so the present application can also reduce the amount of calculation for entropy coding and improve the efficiency of entropy coding.

[0008] For example, the first feature map ∈ R c*h*w , where c*h*w can also be written as c×h×w. "*" and "×" do not represent multiplication calculations, but represent the dimension of R, that is, the dimension of the first feature map. "c" refers to the number of channels of the first feature map, "h" represents the height of the feature map output by each channel, and "w" represents the width of the feature map output by each channel; that is, the first feature map includes feature maps of c channels, and the feature map of each channel may include h*w feature points, each feature point has an eigenvalue, and the first feature map may include eigenvalues ​​of N feature points, where N is the product of c, h and w, and c, h and w are all positive integers. For the convenience of subsequent description, the eigenvalues ​​of N feature points in the first feature map are referred to as first eigenvalues; that is, the first feature map may include the first eigenvalues ​​of N feature points.

[0009] For example, the second feature map ∈ R c*h*w , the second feature map may include second feature values ​​of N feature points.

[0010] Exemplarily, the probability distribution parameters corresponding to the first feature map may include an index (index, an index is used to uniquely identify a variance; for ease of explanation, hereinafter referred to as the first index) of the variance corresponding to each feature point in the N feature points of the first feature map. It should be understood that the probability distribution parameters may also include other parameters, which are not limited in this application.

[0011] Exemplarily, the first adjustment process may be a multiplication operation. It should be noted that the multiplication operation may include two types of calculations, "multiplication" and "division". This application takes the first adjustment process as "multiplication" as an example for explanation.

[0012] It should be noted that the addition operation can include two calculations: "addition" and "subtraction"; when the first adjustment process is 'multiplication', the second adjustment process is "addition"; when the first adjustment process is "division", the second adjustment process is "subtraction"; this application takes the second adjustment process as "addition" as an example for explanation.

[0013] Exemplarily, the first gain vector can be determined according to the target bit rate. Wherein, "c*1" can also be written as "c×1", "*" and "×" do not represent multiplication calculations, but represent the dimension of the first gain vector, and the first gain vector may include c gains, one gain corresponding to a channel in the first feature map; that is, all feature points of the same channel correspond to the same gain in the first gain vector.

[0014] Exemplarily, the first gain vector may be converted to a logarithmic domain according to a first index of the variance to obtain a second gain vector.

[0015] In addition, the estimation information for determining the probability distribution parameters can also be written into the bitstream and transmitted to the decoding end, so that the entropy estimation network at the decoding end can determine the probability distribution parameters corresponding to the first feature map according to the estimation information. The estimation information can be determined according to the first feature map, or according to the feature map obtained by processing the first feature map, and this application does not limit this.

[0016] According to the first aspect, before entropy encoding the second feature map according to N second indexes to obtain the code stream, the method also includes: performing a third adjustment process on the second feature map according to the first step length; performing a fourth adjustment process on the N second indexes according to the second step length, wherein the second step length is obtained by converting the first step length to the logarithmic domain, and the fourth adjustment process is an addition operation. In this way, the feature map (i.e., the second feature map) and the index (i.e., the second index) used for entropy encoding can be adjusted again, so that the second eigenvalue of the feature point and the second index of the variance are more concentrated, thereby reducing the code rate overhead to a certain extent. And in the variance adjustment process, the present application converts the multiplication operation of the variance and the second step length into the addition operation of the index of the variance and the second step length, which reduces the amount of calculation of the variance adjustment, improves the efficiency of the variance adjustment, and thus improves the coding efficiency.

[0017] Exemplarily, the first step length may be determined based on factors such as a target bit rate, a target image quality, and a channel quality.

[0018] Exemplarily, the first step length may be converted to a logarithmic domain according to the first index of the variance to obtain a second step length.

[0019] It should be understood that, before performing the first adjustment process on the first feature map according to the first gain vector to obtain the second feature map, the third adjustment process may be performed on the first feature map according to the first step length; and before performing the second adjustment process on the N first indexes according to the second gain vector to obtain the N second indexes, the fourth adjustment process may be performed on the N first indexes according to the second step length. That is, the present application does not limit the execution order of the first adjustment process, the second adjustment process, the third adjustment process, and the fourth adjustment process.

[0020] According to the first aspect, or any implementation of the first aspect above, the method further includes: generating a mask image corresponding to the first feature map, wherein the mask image is used for the third adjustment process and / or the fourth adjustment process.

[0021] The mask map may also be referred to as a binary mask map, which is used to extract the region of interest and shield the region of no interest; the size of the mask map corresponding to the first feature map is the same as the size of the first feature map, and the mask map corresponding to the first feature map may include a mask value corresponding to each of the N feature points of the first feature map, and the mask value corresponding to a feature point is either 1 or 0. The first feature map is multiplied by the mask map corresponding to the first feature map, and the feature points corresponding to the mask value of 1 are extracted, and the third adjustment process and the fourth adjustment process are performed; the feature points corresponding to the mask value of 0 are shielded.

[0022] Exemplarily, the mask image is generated according to N second indices.

[0023] In this way, the second feature map and the second index can be adaptively adjusted in step size to reduce the difference between the eigenvalues ​​of each feature point and the difference between the indexes of the variances corresponding to each feature point, thereby reducing the quantization loss and the bit rate overhead to a certain extent.

[0024] According to the first aspect, or any implementation method of the first aspect above, a mask map of a first feature map is generated, including: dividing the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; pooling is performed on the Z second indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; based on the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, a mask map corresponding to the first feature map is generated.

[0025] According to the first aspect, or any implementation of the first aspect above, the probability distribution parameter also includes N means, the N means correspond to the N feature points one by one, and before performing a first adjustment process on the first feature map according to the first gain vector to obtain the second feature map, the method also includes: subtracting the corresponding mean from the feature value of each feature point in the first feature map. In this way, the probability distribution of each feature point can be adjusted to approximate a Gaussian distribution, thereby reducing the number of probability distribution tables used for entropy coding and reducing memory usage.

[0026] According to the first aspect, or any implementation of the first aspect above, the pooling processing includes at least one of the following: average pooling processing, maximum pooling processing, or minimum pooling processing.

[0027] According to the first aspect, or any implementation of the first aspect above, before entropy encoding the second feature map according to the N second indexes to obtain the bitstream, the method further includes: rounding the N second indexes. Since a corresponding probability distribution table needs to be generated for each index in advance, after rounding the second index, it is only necessary to pre-set the probability distribution table corresponding to the integer, which can reduce the number of probability distribution tables used for entropy encoding and reduce memory usage.

[0028] For example, rounding may be referred to as quantization / quantization processing.

[0029] In a second aspect, an embodiment of the present application provides a decoding method, the method comprising: first, receiving a code stream, wherein the code stream comprises encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the first feature map comprises N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, the width is w, N is the product of c, h and w, c, h and w are positive integers; then, determining probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters comprise N first indices, the N first indices correspond one-to-one to N variances, and the N variances correspond to N The feature points correspond one to one; then, a second adjustment process is performed on the N first indexes according to the second gain vector to obtain N second indexes, wherein the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond one to one with the N feature points; thereafter, entropy decoding is performed on the encoded data of the N feature points according to the N second indexes to obtain a second feature map; then, a fifth adjustment process is performed on the second feature map according to the first gain vector to obtain a first feature map, wherein the second gain vector is obtained by converting the first gain vector to the logarithmic domain, and the fifth adjustment process is the inverse process of the first adjustment process; the first feature map is reconstructed to obtain a reconstructed image of the image.

[0030] In the variance adjustment process, the present application converts the multiplication operation between the variance and the gain into an addition operation between the index of the variance and the gain converted to the logarithmic domain; thus, in some cases, the amount of calculation of the variance adjustment can be reduced while ensuring the compression performance, thereby improving the efficiency of the variance adjustment and thus improving the decoding efficiency.

[0031] In addition, the probability distribution table of the present application corresponds to the index of the variance one-to-one, and then, in the entropy decoding process, the present application can directly determine the corresponding probability distribution table according to the quantized second index, and then, the characteristic value of the feature point in the second feature map can be entropy decoded according to the probability distribution table. After the prior art adjusts and quantizes the variance, it is necessary to find the nearest preset variance from multiple preset variances (the probability distribution table corresponds to the preset variance one-to-one); then the probability distribution table corresponding to the nearest preset variance is used to entropy decode the characteristic value of the feature point in the second feature map. Among them, the process of finding the nearest preset variance in the prior art requires a certain amount of calculation, so the present application can also reduce the amount of calculation for entropy decoding and improve the efficiency of entropy decoding.

[0032] According to the second aspect, the code stream includes the encoded data of N feature points in the second feature map after the third adjustment processing; before reconstructing the first feature map to obtain the reconstructed image of the image, the method also includes: performing a sixth adjustment processing on the first feature map according to the first step size, and the sixth adjustment processing is the inverse process of the third adjustment processing; before entropy decoding the encoded data of the N feature points according to the N second indexes to obtain the second feature map, the method also includes: performing a fourth adjustment processing on the N second indexes according to the second step size, wherein the second step size is obtained by converting the first step size to the logarithmic domain, and the fourth adjustment processing is an addition operation.

[0033] According to the second aspect, or any implementation of the second aspect above, the method further includes: generating a mask image corresponding to the first feature map, wherein the mask image is used for the sixth adjustment process and / or the fourth adjustment process.

[0034] According to the second aspect, or any implementation method of the second aspect above, a mask map of the first feature map is generated, including: dividing the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; pooling is performed on the Z second indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; according to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, the mask map corresponding to the first feature map is determined.

[0035] According to the second aspect, or any implementation of the second aspect above, the probability distribution parameters also include N means, and the N means correspond one-to-one to the N feature points; before reconstructing the first feature map to obtain a reconstructed image of the image, the method also includes: adding the corresponding mean to the feature value of each feature point in the first feature map.

[0036] According to the second aspect, or any implementation of the second aspect, the pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

[0037] According to the second aspect, or any implementation of the second aspect above, before entropy decoding the encoded data of N feature points according to N second indexes to obtain the second feature map, the method also includes: rounding the N second indexes.

[0038] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0039] In a third aspect, an embodiment of the present application also provides a coding method, which includes: first, obtaining an image to be encoded; then, performing feature extraction on the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, c, the first feature map includes N feature points, N is the product of c, h and w, and c, h and w are positive integers; and determining the probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points; then, performing a first adjustment processing on the first feature map according to the first step length to obtain a second feature map, wherein the second feature map includes N feature points; and performing a second adjustment processing on the first indexes corresponding to the N feature points according to the second step length to obtain N second indexes, wherein the second step length is obtained by converting the first step length to the logarithmic domain, the second adjustment processing is an addition operation, and the N second indexes correspond one-to-one to the N feature points; then, entropy encoding is performed on the second feature map according to the N second indexes to obtain a code stream.

[0040] In the variance adjustment process, the present application converts the multiplication operation of the variance and the second step length into the addition operation of the index of the variance and the second step length. In this way, in some cases, the amount of calculation of the variance adjustment is reduced, the efficiency of the variance adjustment is improved, and the coding efficiency is improved while ensuring the compression performance. In other cases, the bit rate overhead can be reduced while reducing the amount of calculation of the variance adjustment and improving the efficiency of the variance adjustment (because the entropy estimation network used to output the variance (i.e., the entropy estimation network of the prior art) has inconsistent processes during training and testing, and the entropy estimation network used to determine the first index of the variance (i.e., the entropy estimation network of the present application) has different processes during training and testing, so in some cases, the bit rate overhead can be reduced).

[0041] According to the third aspect, the method further includes: generating a mask image corresponding to the first feature image, wherein the mask image is used for the first adjustment process and / or the second adjustment process.

[0042] According to the third aspect, or any implementation method of the third aspect above, a mask map of the first feature map is generated, including: dividing the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; pooling is performed on the Z first indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; according to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, the mask map corresponding to the first feature map is determined.

[0043] According to the third aspect, or any implementation of the third aspect above, the probability distribution parameters also include N means, and the N means correspond one-to-one to the N feature points; before performing a first adjustment processing on the first feature map according to the first step length to obtain the second feature map, the method also includes: subtracting the corresponding mean from the feature value of each feature point in the first feature map.

[0044] According to the third aspect, or any implementation of the third aspect, the pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

[0045] According to the third aspect, or any implementation of the third aspect above, before entropy encoding the second feature map according to the N second indexes to obtain the code stream, the method also includes: rounding the N second indexes.

[0046] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0047] In a fourth aspect, an embodiment of the present application further provides a decoding method, which comprises: first, receiving a code stream, wherein the code stream comprises encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the first feature map comprises N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, the width is w, N is the product of c, h and w, c, h and w are positive integers; then, determining a probability distribution parameter corresponding to the first feature map, wherein the probability distribution parameter is N first indexes, and the N first indexes are related to the N squares. The differences correspond one to one, and the N variances correspond one to one to the N feature points; then, the N first indexes are subjected to a second adjustment process according to the second step length to obtain N second indexes, and the second adjustment process is an addition operation, and the N second indexes correspond one to one to the N feature points; subsequently, the encoded data of the N feature points are entropy decoded according to the N second indexes to obtain a second feature map; then, the second feature map is subjected to a third adjustment process according to the first step length to obtain a first feature map, wherein the second step length is obtained by converting the first step length to the logarithmic domain, and the third adjustment process is the inverse process of the first adjustment process; the first feature map is reconstructed to obtain a reconstructed image of the image.

[0048] According to a fourth aspect, the method further includes: generating a mask image corresponding to the first feature image, wherein the mask image is used for the third adjustment process and / or the second adjustment process.

[0049] According to the fourth aspect, or any implementation method of the fourth aspect above, a mask map of the first feature map is generated, including: dividing the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; pooling is performed on the Z first indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; according to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, the mask map corresponding to the first feature map is determined.

[0050] According to the fourth aspect, or any implementation of the fourth aspect above, the probability distribution parameters also include N means, and the N means correspond one-to-one to the N feature points; before reconstructing the first feature map to obtain a reconstructed image of the image, the method also includes: adding the corresponding mean to the feature value of each feature point in the first feature map.

[0051] According to the fourth aspect, or any implementation of the fourth aspect, the pooling processing includes at least one of the following: average pooling processing, maximum pooling processing, or minimum pooling processing.

[0052] According to the fourth aspect, or any implementation of the fourth aspect above, before entropy decoding the encoded data of N feature points according to N second indexes to obtain the second feature map, the method also includes: rounding the N second indexes.

[0053] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0054] In a fifth aspect, an embodiment of the present application further provides an encoding device, the device comprising:

[0055] A first acquisition module, used for acquiring an image to be encoded;

[0056] A first encoding network is used to extract features of the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w. The first feature map includes N feature points, N is the product of c, h, and w, and c, h, and w are positive integers;

[0057] A first entropy estimation network is used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0058] A first adjustment module, configured to perform a first adjustment process on the first feature map according to the first gain vector to obtain a second feature map, wherein the size of the first gain vector is c*1, and the second feature map includes N feature points;

[0059] The first adjustment module is further used to perform a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0060] The first entropy coding network is further used to perform entropy coding on the second feature map according to the N second indexes to obtain a code stream.

[0061] It should be noted that the encoding device of the fifth aspect may also include a communication module, and may also be used to execute any implementation method of the first aspect, which will not be repeated here.

[0062] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0063] In a sixth aspect, an embodiment of the present application further provides a decoding device, the device comprising:

[0064] A first receiving module is used to receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the first feature map includes N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, and the width is w, N is the product of c, h and w, and c, h and w are positive integers;

[0065] A second entropy estimation network is used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0066] A second adjustment module, used for performing a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0067] A first entropy decoding network is used to perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map;

[0068] The second adjustment module is further used to perform a fifth adjustment process on the second feature map according to the first gain vector to obtain the first feature map, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, and the fifth adjustment process is an inverse process of the first adjustment process;

[0069] The first decoding network is used to reconstruct the first feature map to obtain a reconstructed image of the image.

[0070] It should be noted that the decoding device of the sixth aspect may also include a communication module, and may also be used to execute any implementation method of the second aspect, which will not be repeated here.

[0071] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0072] In a seventh aspect, an embodiment of the present application further provides an encoding device, the device comprising:

[0073] A second acquisition module, used for acquiring an image to be encoded;

[0074] The second encoding network is used to extract features of the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, c, the first feature map includes N feature points, N is the product of c, h and w, and c, h and w are positive integers;

[0075] A third entropy estimation network is used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0076] A third adjustment module is used to perform a first adjustment process on the first feature map according to the first step length to obtain a second feature map, wherein the second feature map includes N feature points;

[0077] The third adjustment module is further used to perform a second adjustment process on the first indexes corresponding to the N feature points according to the second step length to obtain N second indexes, wherein the second step length is obtained by converting the first step length into the logarithmic domain, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0078] The second entropy coding module is also used to perform entropy coding on the second feature map according to the N second indexes to obtain a code stream.

[0079] It should be noted that the encoding device of the seventh aspect may also include a communication module, and may also be used to execute any implementation method of the first aspect, which will not be repeated here.

[0080] The seventh aspect and any implementation of the seventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

[0081] In an eighth aspect, an embodiment of the present application further provides a decoding device, the device comprising:

[0082] A second receiving module is used to receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the first feature map includes N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, and the width is w, N is the product of c, h and w, and c, h and w are positive integers;

[0083] A fourth entropy estimation network, used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0084] A fourth adjustment module, configured to perform a second adjustment process on the N first indexes according to the second step length to obtain N second indexes, wherein the second adjustment process is an addition operation, and the N second indexes correspond one-to-one to the N feature points;

[0085] A second entropy decoding network is used to perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map;

[0086] The fourth adjustment module is further used to perform a third adjustment process on the second feature map according to the first step length to obtain the first feature map, wherein the second step length is obtained by converting the first step length into a logarithmic domain, and the third adjustment process is an inverse process of the first adjustment process;

[0087] The second decoding network is used to reconstruct the first feature map to obtain a reconstructed image of the image.

[0088] It should be noted that the decoding device of the eighth aspect may also include a communication module, and may also be used to execute any implementation method of the second aspect, which will not be repeated here.

[0089] The eighth aspect and any implementation of the eighth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the eighth aspect and any implementation of the eighth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

[0090] In the ninth aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enables the electronic device to execute the encoding method in the first aspect or any possible implementation of the first aspect; or, enables the electronic device to execute the encoding method in the third aspect or any possible implementation of the third aspect.

[0091] The ninth aspect and any implementation of the ninth aspect correspond to the first aspect and any implementation of the first aspect, respectively, or correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the ninth aspect and any implementation of the ninth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, or can refer to the technical effects corresponding to the above-mentioned third aspect and any implementation of the third aspect, which will not be repeated here.

[0092] In the tenth aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enables the electronic device to execute the decoding method in the second aspect or any possible implementation of the second aspect; or, enables the electronic device to execute the decoding method in the fourth aspect or any possible implementation of the fourth aspect.

[0093] The tenth aspect and any implementation of the tenth aspect correspond to the second aspect and any implementation of the second aspect, respectively, or correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the tenth aspect and any implementation of the tenth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, or can refer to the above-mentioned fourth aspect and the technical effects corresponding to any implementation of the fourth aspect, which will not be repeated here.

[0094] In the eleventh aspect, an embodiment of the present application provides a chip, comprising one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the method in the first aspect or any possible implementation of the first aspect is executed; or, the method in the third aspect or any possible implementation of the third aspect is executed.

[0095] The eleventh aspect and any implementation of the eleventh aspect correspond to the first aspect and any implementation of the first aspect, or correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the above-mentioned third aspect and any implementation of the third aspect, which will not be repeated here.

[0096] In the twelfth aspect, an embodiment of the present application provides a chip, comprising one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the method in the second aspect or any possible implementation of the second aspect is executed; or, the method in the fourth aspect or any possible implementation of the fourth aspect is executed.

[0097] The twelfth aspect and any implementation of the twelfth aspect correspond to the second aspect and any implementation of the second aspect, respectively, or correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, or can refer to the above-mentioned fourth aspect and the technical effects corresponding to any implementation of the fourth aspect, which will not be repeated here.

[0098] In the thirteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, it enables the computer or the processor to execute the encoding method in the first aspect or any possible implementation of the first aspect; or, it enables the computer or the processor to execute the encoding method in the third aspect or any possible implementation of the third aspect.

[0099] The thirteenth aspect and any implementation of the thirteenth aspect correspond to the first aspect and any implementation of the first aspect, or correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the thirteenth aspect and any implementation of the thirteenth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the above-mentioned third aspect and any implementation of the third aspect, which will not be repeated here.

[0100] In the fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, it enables the computer or the processor to execute the decoding method in the second aspect or any possible implementation of the second aspect; or, it enables the computer or the processor to execute the decoding method in the fourth aspect or any possible implementation of the fourth aspect.

[0101] The fourteenth aspect and any implementation of the fourteenth aspect correspond to the second aspect and any implementation of the second aspect, respectively, or correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the fourteenth aspect and any implementation of the fourteenth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, or can refer to the above-mentioned fourth aspect and the technical effects corresponding to any implementation of the fourth aspect, which will not be repeated here.

[0102] In a fifteenth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the encoding method in the first aspect or any possible implementation of the first aspect, or the computer or the processor executes the encoding method in the third aspect or any possible implementation of the third aspect.

[0103] The fifteenth aspect and any implementation of the fifteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively, or correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the fifteenth aspect and any implementation of the fifteenth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, or can refer to the above-mentioned third aspect and the technical effects corresponding to any implementation of the third aspect, which will not be repeated here.

[0104] In the sixteenth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the encoding method in the second aspect or any possible implementation of the second aspect, or the computer or the processor executes the encoding method in the fourth aspect or any possible implementation of the fourth aspect.

[0105] The sixteenth aspect and any implementation of the sixteenth aspect correspond to the second aspect and any implementation of the second aspect, or correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the sixteenth aspect and any implementation of the sixteenth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, or can refer to the above-mentioned fourth aspect and the technical effects corresponding to any implementation of the fourth aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1A is a schematic diagram of a system framework shown as an example;

[0107] Figure 1B is a schematic diagram of a system framework shown as an example;

[0108] Figure 2A is a schematic diagram of a compression framework shown as an example;

[0109] Figure 2B is a schematic diagram of an exemplary encoding process;

[0110] Figure 3 is a schematic diagram of an exemplary decoding process;

[0111] Figure 4A is a schematic diagram of an exemplary encoding process;

[0112] Figure 4B is a schematic diagram of an MSAK diagram shown as an example;

[0113] Figure 5 is a schematic diagram of an exemplary decoding process;

[0114] Figure 6 is a schematic diagram of an exemplary encoding process;

[0115] Figure 7 is a schematic diagram of an exemplary decoding process;

[0116] Figure 8 is a schematic diagram of an exemplary encoding device;

[0117] Fig. 9is a schematic diagram of an exemplary decoding device;

[0118] Fig.10 is a schematic diagram of an exemplary encoding device;

[0119] Fig.11 is a schematic diagram of an exemplary decoding device;

[0120] Fig.12 Schematic diagram of the structure of the device shown as an example. DETAILED DESCRIPTION

[0121] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0122] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0123] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0124] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0125] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0126] Figure 1A The system framework diagram is shown as an example.

[0127] Reference Figure 1A, exemplarily, the first electronic device may include a first acquisition module, a first encoding module, a first channel encoding module, a first channel decoding module, a first decoding module and a first playback module. It should be understood that the first electronic device may include Figure 1A The present application does not limit the number of modules shown. In addition, the first encoding module may also be referred to as a first encoder, and the first decoding module may also be referred to as a first decoder.

[0128] Reference Figure 1A , exemplarily, the second electronic device may include a second acquisition module, a second encoding module, a second channel encoding module, a second channel decoding module, a second decoding module, and a second playback module. It should be understood that the second electronic device may include Figure 1A The present application does not limit the number of modules shown. In addition, the second encoding module may also be referred to as a second encoder, and the second decoding module may also be referred to as a second decoder.

[0129] Exemplarily, the process of encoding and transmitting video data to the second electronic device by the first electronic device, and decoding and playing by the second electronic device can be as follows: the first acquisition module can perform video acquisition and output video data to the first encoding module. Then, the first encoding module can encode the video data and output the code stream to the first channel encoding module. After that, the first channel encoding module can channel encode the code stream and transmit the channel-encoded code stream to the second electronic device through a wireless or wired network communication device. Then, the second channel decoding module of the second electronic device can channel decode the received data to obtain the code stream and output the code stream to the second decoding module. Then, the second decoding module can decode the code stream to obtain reconstructed video data; then the reconstructed video data is output to the second playback module, and the second playback module plays it.

[0130] It should be understood that the process of the second electronic device encoding and transmitting video data to the first electronic device, which is decoded and played by the first electronic device, is similar to the process of the first electronic device transmitting video data to the second electronic device, which is played by the second electronic device, and will not be repeated here.

[0131] It should be understood that the first electronic device and the second electronic device can be directly connected without communicating through a wireless or wired network communication device, and this application does not limit this.

[0132] Exemplarily, the first electronic device and the second electronic device may include, but are not limited to: personal computers, computer workstations, smart phones, tablet computers, servers, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.

[0133] For example, Figure 1A The system framework can be applied to any scenario that requires encoding and decoding, such as VR (Virtual Reality) / AR (Augmented Reality) scenarios. In VR and AR scenarios: In one possible way, the first electronic device is a server and the second electronic device is a VR / AR device; in one possible way, the second electronic device is a server and the first electronic device is a VR / AR device.

[0134] Exemplarily, when the first electronic device encodes the video data and the second electronic device decodes to obtain the reconstructed video data, the first electronic device may be referred to as the encoding end and the second electronic device may be referred to as the decoding end. When the second electronic device encodes the video data and the first electronic device decodes to obtain the reconstructed video data, the second electronic device may be referred to as the encoding end and the first electronic device may be referred to as the decoding end.

[0135] Figure 1B The system framework diagram is shown as an example.

[0136] Reference Figure 1B (1) Exemplarily, a wireless or core network device may include: a channel decoding module, other decoding modules, a coding module (that is, the coding module involved in this application) and a channel coding module. The wireless or core network device may be used for transcoding.

[0137] For example, Figure 1B The specific application scenario of (1) may be: when the first electronic device is not provided with an encoding module but only with other encoding modules, and the second electronic device is only provided with a decoding module but not with other decoding modules, in order to enable the second electronic device to decode and play the video data encoded by the first electronic device using other encoding modules, wireless or core network equipment may be used for transcoding.

[0138] Specifically, the first electronic device uses other encoding modules to encode the video data to obtain code stream 1; and performs channel encoding on code stream 1 and sends it to the wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the code stream 1 obtained by channel decoding to other decoding modules. After that, other decoding modules decode code stream 1 to obtain video data and output the video data to the encoding module. Then, the encoding module can encode the video data to obtain code stream 2 and output code stream 2 to the channel encoding module, and the channel encoding module performs channel encoding on code stream 2 and sends it to the second electronic device. In this way, the second electronic device can call the decoding module to decode the code stream 2 obtained by channel decoding to obtain reconstructed video data; the reconstructed video data can be played later.

[0139] Reference Figure 1B (2) Exemplarily, the wireless or core network device may include: a channel decoding module, a decoding module (that is, the encoding module involved in this application), other encoding modules and a channel encoding module. The wireless or core network device may be used for transcoding.

[0140] For example, Figure 1B The specific application scenario of (2) may be: when the first electronic device is only provided with an encoding module but not other encoding modules, and the second electronic device is not provided with a decoding module but only other decoding modules, in order to enable the second electronic device to decode and play the video data encoded by the encoding module of the first electronic device, wireless or core network equipment may be used for transcoding.

[0141] Specifically, the first electronic device uses an encoding module to encode the video data to obtain code stream 1; and performs channel encoding on code stream 1 and sends it to a wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the code stream 1 obtained by channel decoding to the decoding module. After that, the decoding module decodes code stream 1 to obtain video data and outputs the video data to other encoding modules. Then, other encoding modules can encode the video data to obtain code stream 2 and output code stream 2 to the channel encoding module, and the channel encoding module performs channel encoding on code stream 2 and sends it to the second electronic device. In this way, the second electronic device can call other decoding modules to decode code stream 2 obtained by channel decoding to obtain reconstructed video data; the reconstructed video data can be played later.

[0142] Figure 2A FIG. 1 is a schematic diagram of a compression framework shown as an example. Figure 2A In the embodiment of FIG. 1 , an end-to-end AI compression framework is shown; in Figure 2A The encoding and decoding of an image is taken as an example to illustrate.

[0143] Reference Figure 2A , exemplary, an end-to-end AI compression framework may include an encoding network, a quantization module, an entropy encoding module, an entropy estimation network, an entropy decoding module, a decoding network, a first adjustment module, a second adjustment module, a third adjustment module, and a fourth adjustment module.

[0144] Exemplarily, the coding network can be used to perform spatial transformation (also called feature extraction) on the image to be coded, and transform the image to be coded into another space. Exemplarily, the coding network can be a convolutional neural network.

[0145] Exemplarily, a quantization module can be used to perform quantization processing.

[0146] Exemplarily, the entropy estimation network can be used to determine the probability distribution parameters corresponding to the feature map, such as the mean, variance, etc., which is not limited in this application.

[0147] Exemplarily, the entropy coding module can be used to perform entropy coding on the feature map according to the probability distribution parameters.

[0148] Exemplarily, the entropy decoding module can be used to perform entropy decoding on the feature map according to the probability distribution parameters.

[0149] Exemplarily, the decoding network can be used to perform inverse spatial transformation (also called reconstruction or feature recovery) on the feature map and output a reconstructed image. Exemplarily, the decoding network can be a convolutional neural network.

[0150] Exemplarily, entropy coding refers to coding that does not lose any information according to the entropy principle during the coding process. Entropy coding may include multiple types, such as Shannon coding, Huffman coding, and arithmetic coding, etc. This application does not impose any restrictions on this.

[0151] Exemplarily, the image to be encoded input to the encoding network can be any one of a RAW (unprocessed) image, an RGB (RedGreenBlue) image, and a YUV ("Y" represents brightness (Luminance, Luma), and "U" and "V" represent chrominance, concentration (Chrominance, Chroma)) image, and the present application does not impose any restrictions on this.

[0152] Exemplarily, the first adjustment module and the third adjustment module are used to perform adjustment processing on the feature map, such as multiplication operation.

[0153] Exemplarily, the second adjustment module and the fourth adjustment module are used to perform adjustment processing, such as addition operation, on the index of the variance.

[0154] It should be noted that the present application does not limit the implementation method of the entropy estimation network, for example, it can be a hyper-prior network, etc.

[0155] Among them, the feature graphs involved in the embodiments of the present application include: a first feature graph, a second feature graph, a third feature graph, a fourth feature graph and a fifth feature graph, all ∈R c*h*w, where "c" refers to the number of channels of the feature map, "h" represents the height of the feature map output by each channel, and "w" represents the width of the feature map output by each channel; the feature map of each channel may include the eigenvalues ​​of h*w feature points; that is, the feature map includes N feature points, where N is the product of c, w, and h. For ease of distinction, the eigenvalue of the feature point in the first feature map may be referred to as the first eigenvalue, the eigenvalue of the feature point in the second feature map may be referred to as the second eigenvalue, the eigenvalue of the feature point in the third feature map may be referred to as the third eigenvalue, the eigenvalue of the feature point in the fourth feature map may be referred to as the fourth eigenvalue, and the eigenvalue of the feature point in the fifth feature map may be referred to as the fifth eigenvalue.

[0156] The following Figure 2A Based on, refer to Figure 2B and Figure 3 In the embodiment, a coding and decoding process of an image (which may be an image in video data or an independent image) is described.

[0157] Figure 2B FIG. 1 is a schematic diagram of an exemplary encoding process.

[0158] S201, obtaining an image to be encoded.

[0159] Exemplarily, the encoding end may obtain the image to be encoded, and then may refer to S202 to S206 to encode the image to be encoded to obtain a code stream.

[0160] S202, extracting features from the image to be encoded to obtain a first feature map, where the first feature map includes N feature points.

[0161] Exemplarily, the encoding network can perform feature extraction on the image to be encoded to transform the image to be encoded into another space, thereby reducing the temporal redundancy and spatial redundancy of the image to be encoded, and obtaining a first feature map.

[0162] For example, the first feature map ∈ R c*h*w , that is, the size of the first feature map is c*h*w; where "c" refers to the number of channels of the first feature map, "h" refers to the height of the feature map output by each channel, and "w" refers to the width of the feature map output by each channel; that is, the first feature map includes feature maps of c channels, and the feature map of each channel may include h*w feature points. The first feature map may include N feature points, and each feature point has an eigenvalue, where N is the product of c, h, and w, and c, h, and w are all positive integers. Among them, the eigenvalues ​​of the N feature points in the first feature map can be called the first eigenvalues.

[0163] S203, determining probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points.

[0164] In one possible approach, refer to Figure 2A , the first feature map can be input into the entropy estimation network to obtain the probability distribution parameters corresponding to the first feature map output by the entropy estimation network.

[0165] In one possible approach, feature extraction can be performed on the first feature map to obtain a third feature map, and the third feature map is input into an entropy estimation network to obtain probability distribution parameters corresponding to the first feature map output by the entropy estimation network, wherein the size of the third feature map is the same as that of the first feature map.

[0166] In one possible manner, the feature map obtained by adjusting and quantizing the first feature map can be input into an entropy estimation network to obtain probability distribution parameters corresponding to the first feature map output by the entropy estimation network, wherein the size of the feature map obtained by adjusting and quantizing is the same as the size of the first feature map.

[0167] That is to say, the present application can directly input the first feature map into the entropy estimation network, or can input the feature map obtained after processing the first feature map into the entropy estimation network to obtain the probability distribution parameters corresponding to the first feature map; the present application does not impose any restrictions on this.

[0168] Exemplarily, the entropy estimation network can process the input feature map (such as the first feature map and the third feature map) to obtain estimation information; then, perform probability estimation based on the estimation information to determine the probability distribution parameter corresponding to the first feature map. Exemplarily, when the entropy estimation network is a super priori network, the estimation information can be super priori information.

[0169] Exemplarily, the probability distribution parameters corresponding to the first feature map may include an index (index, an index is used to uniquely identify a variance; for ease of explanation, hereinafter referred to as the first index) of the variance corresponding to each feature point in the N feature points of the first feature map. It should be understood that the probability distribution parameters may also include other parameters, which are not limited in this application.

[0170] S204: Perform a first adjustment process on the first feature map according to the first gain vector to obtain a second feature map.

[0171] For example, different gain vectors may be set in advance for different target bit rates to obtain a gain vector set M = [M1, M2, ..., M g]; wherein the gain vector set M may include g gain vectors, the size of each gain vector is c*1, and g is a positive integer.

[0172] Exemplarily, the first adjustment process is a multiplication operation. It should be noted that the multiplication operation may include two calculations, "multiplication" and "division". This application takes the first adjustment process as "multiplication" as an example for explanation.

[0173] Exemplarily, a corresponding gain vector may be selected from the gain vector set M as the first gain vector according to the current target bit rate; then, a first adjustment process is performed on the first feature map according to the first gain vector to obtain a second feature map.

[0174] Specifically, for each channel of the first feature map, the first eigenvalue of each feature point among the h*w feature points included in the channel is multiplied by the gain corresponding to the channel in the first gain vector to obtain the second eigenvalue of the feature point; in this way, the second eigenvalues ​​of N feature points can be obtained, and the second eigenvalues ​​of these N feature points can form the second feature map. Exemplarily, the second feature map ∈ R c*h*w , the second feature map may include second feature values ​​of N feature points.

[0175] For example, in Figure 2B In the embodiment, taking a feature point in a first feature map as an example, a first adjustment process is performed on the first eigenvalue of the feature point according to the first gain vector to obtain the second eigenvalue, which can refer to the following formula (1):

[0176]

[0177] In formula (1), y2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue; is the channel c in the first gain vector i The corresponding gain, y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) first eigenvalue. Among them, c i is a positive integer less than or equal to c, h j is a positive integer less than or equal to h, w k is a positive integer less than or equal to w, and b is a positive integer less than or equal to g.

[0178] According to formula (1), the larger the vector in the first gain vector, the larger the step size; in this way, the first eigenvalues ​​of more feature points will be adjusted to the same eigenvalue, thereby achieving the effect of different bit rates in the same AI compression framework.

[0179] Exemplarily, the second eigenvalues ​​of the N feature points may be rounded (that is, quantized); in this case, formula (1) may be converted to formula (2):

[0180]

[0181] In formula (2), [] indicates rounding.

[0182] It should be understood that the gain vector set M = [M1 -1 ,M2 -1 ,......,M g -1 ], thus, in the process of performing the first adjustment process on the first feature map according to the first gain vector, c i The first eigenvalue of the feature point in the channel is divided by c in the first gain vector i The gain corresponding to the channel is not limited in this application. This application is M = [M1, M2, ..., M g ] as an example to illustrate.

[0183] In a possible manner, the probability distribution parameter may further include N means, and the N means correspond to the N feature points one by one. In this case, the first eigenvalue of each feature point in the N feature points in the first feature map may be subtracted from the corresponding mean to obtain the fourth eigenvalue of each feature point. The fourth feature group of the N feature points may form a fourth feature map. The following formula (3) may be referred to:

[0184] r4(c i ,h j ,w k )=y1(c i ,h j ,w k ) -μ(c i ,h j ,w k ) (3)

[0185] In formula (3), y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), r4(c i ,h j ,w k) represents the feature point (c i ,h j ,w k )’s fourth eigenvalue; μ(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponding to the mean.

[0186] Afterwards, the fourth feature map may be subjected to a first adjustment process according to the first gain vector to obtain a second feature map and the second feature values ​​of the N feature points in the second feature map may be rounded; this may refer to formula (4):

[0187]

[0188] In formula (4), y2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue, r4(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s fourth eigenvalue; is the channel c in the first gain vector i The corresponding gain.

[0189] Wherein, S204 may be executed by the first adjustment module.

[0190] It should be noted that the present application does not limit the execution order of S203 and S204.

[0191] S205, performing a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the second gain vector is obtained by converting the first gain vector to the logarithmic domain, the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond one-to-one to the N feature points.

[0192] Exemplarily, the gain vector set M can be converted to the log domain (logarithmic domain) in advance to obtain the gain vector set M l =[m1 l ,m2 l ,......,m g l]; wherein, the method of converting the gain in the gain vector of the gain vector set into the log domain can refer to the following formula (5):

[0193]

[0194] In formula (5), is the c of the dth gain vector in the gain vector set M i The gain corresponding to the channel, is the gain vector set M l The c of the d-th gain vector in i The gain corresponding to the channel, σ max is the maximum possible variance, such as 0.11, σ min is the minimum value of the possible variance, such as 100; d is a positive integer less than or equal to g; L is the total number of indexes of possible variances.

[0195] It should be noted that addition operations can include both "addition" and "subtraction" calculations; Figure 2B In the embodiment, when the first adjustment process is 'multiplication', the second adjustment process is 'addition'; when the first adjustment process is 'division', the second adjustment process is 'subtraction'; this application takes the second adjustment process 'addition' as an example for explanation.

[0196] For example, in Figure 2B In the embodiment, taking a feature point as an example, a second adjustment process is performed on the first index of the variance corresponding to the feature point according to the second gain vector to obtain the second index of the variance corresponding to the feature point, which can be referred to the following formula (6):

[0197]

[0198] In formula (6), index2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the second index of the variance; index1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the first index of the variance; is the second gain vector c i The gain corresponding to the channel.

[0199] Afterwards, the second index can be rounded. At this time, formula (6) can be converted to formula (7):

[0200]

[0201] In formula (7), [] indicates rounding.

[0202] Wherein, S205 may be executed by the second adjustment module. The present application does not limit the execution order of S204 and S205.

[0203] S206: Perform entropy encoding on the second feature map according to the N second indexes to obtain a code stream.

[0204] Afterwards, the entropy coding module can determine the probability distribution table corresponding to the N feature points according to the N second indexes; then, the second feature values ​​of the N feature points in the second feature map can be entropy coded (that is, the second feature map is entropy coded) according to the probability distribution table corresponding to the N feature points to obtain a bitstream. In this way, the obtained bitstream can include the coded data of the N feature points in the second feature map.

[0205] In addition, the estimation information can also be written into the bitstream and transmitted to the decoding end, so that the entropy estimation network at the decoding end can determine the probability distribution parameters corresponding to the first feature map based on the estimation information.

[0206] Exemplarily, the encoding end may further include a device for transmitting a code stream; the device for transmitting a code stream may include: a transmitter and at least one storage medium, wherein the at least one storage medium is used to store the above Figure 2B The code stream generated by the embodiment; the transmitter is used to obtain the code stream from the storage medium and send the code stream to the terminal side device through the transmission medium.

[0207] Exemplarily, after the encoding end generates the bitstream, it can send the bitstream to the system for distributing the bitstream. The system for distributing the bitstream can include: at least one storage medium and a streaming media device; the storage medium is used to store the above Figure 2B At least one code stream generated by the embodiment; a streaming media device, used to obtain a target code stream from at least one storage medium and send the target code stream to a terminal side device, wherein the streaming media device includes a content server or a content distribution server.

[0208] Figure 3 FIG. 4 is a schematic diagram showing an exemplary decoding process. Figure 3 The decoding process and Figure 2B The encoding process corresponds to .

[0209] Exemplarily, the decoding end may include a device for storing a code stream. The device for storing a code stream may include: a receiver and at least one storage medium, the receiver is used to receive the code stream; and the at least one storage medium is used to store the code stream. Then, the code stream may be decoded with reference to S302 to S306.

[0210] S301, receiving a code stream.

[0211] Exemplarily, the code stream may include encoded data of N feature points in the second feature map and encoded data of the estimation information.

[0212] S302, determining probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points.

[0213] Exemplarily, the entropy decoding module may first perform entropy decoding on the coded data of the estimation information in the bitstream to obtain the estimation information. Then, the entropy estimation network may perform probability estimation based on the estimation information to determine the probability distribution parameters corresponding to the first feature map; wherein the probability distribution parameters include the first index of the variance corresponding to each feature point in the N feature points of the first feature map.

[0214] S303 , performing a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, where the N second indexes correspond one-to-one to the N feature points.

[0215] For example, S303 may refer to the description of S205 above, which will not be described in detail herein. S303 may be executed by the fourth adjustment module.

[0216] S304: Perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map.

[0217] Exemplarily, the entropy decoding module can determine the probability distribution table corresponding to N feature points based on N second indexes; then, according to the probability distribution table corresponding to the N feature points, the encoded data of the N feature points in the second feature map in the bitstream can be entropy decoded to obtain the second feature map.

[0218] S305: Perform a third adjustment process on the second feature map according to the first gain vector to obtain a first feature map.

[0219] Exemplarily, S305 may be executed by a third adjustment module.

[0220] It should be noted that Figure 3 The third adjustment process in the embodiment is Figure 2A The reverse process of the first adjustment process in the embodiment.

[0221] Exemplarily, when the first adjustment process in S204 is "multiplication", the third adjustment process in S305 is "division"; when the first adjustment process in S204 is "division", the third adjustment process in S305 is "multiplication". This application takes the third adjustment process as "division" as an example for explanation.

[0222] Exemplarily, a corresponding gain vector can be selected from the gain vector set M as the first gain vector according to the current target bit rate; then, a third adjustment process is performed on the second feature map according to the first gain vector to obtain the first feature map.

[0223] Specifically, for each channel of the second feature map, the second eigenvalue of each feature point among the h*w feature points included in the channel is divided by the gain corresponding to the channel in the first gain vector to obtain the first eigenvalue of the feature point; in this way, the first eigenvalues ​​of N feature points can be obtained, and the first eigenvalues ​​of these N feature points can constitute the first feature map.

[0224] For example, in Figure 3 In the embodiment, taking a feature point as an example, a method of performing a third adjustment process on the second eigenvalue of the feature point according to the first gain vector to obtain the first eigenvalue of the feature point can refer to the following formula (8):

[0225]

[0226] In formula (8), y2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue; is the first gain vector, y1(c i ,h j ,w k ) represents the feature point [c i ,h j ,w k ] is the first eigenvalue of ].

[0227] When the encoder performs the first adjustment processing on the fourth feature map according to the first gain vector to obtain the second feature map, the decoder performs the third adjustment processing on the second feature map according to the first gain vector to obtain the fourth feature map; then, the fourth feature value of each feature point in the N features of the fourth feature map can be added with the corresponding mean value to obtain the first feature map; the following formula (9) can be referred to:

[0228] y1(c i ,h j ,w k )=r4(c i ,h j ,w k ) +μ(c i ,h j ,w k ) (9)

[0229] In formula (9), y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), r4(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s fourth eigenvalue; μ(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponding to the mean.

[0230] S306: reconstruct the first feature map to obtain a reconstructed image of the image.

[0231] Afterwards, the first feature map can be input into the decoding network, and the decoding network reconstructs the first feature map and outputs the reconstructed image.

[0232] Based on the above Figure 2B The encoding process of the embodiment and Figure 3 It can be seen from the decoding process of the embodiment that in the process of adjusting the variance before quantization, the present application converts the multiplication operation between the variance and the gain into an addition operation between the index of the variance and the gain converted to the logarithmic domain; in this way, in some cases, the amount of calculation of the variance adjustment can be reduced, the efficiency of the variance adjustment can be improved, and the encoding and decoding efficiency can be improved while ensuring the compression performance. In other cases, the bit rate overhead can be reduced while reducing the amount of calculation of the variance adjustment and improving the variance adjustment efficiency (because the entropy estimation network for outputting the variance (i.e., the entropy estimation network of the prior art) has inconsistent training and testing processes, while the entropy estimation network for determining the first index of the variance (i.e., the entropy estimation network of the present application) has consistent training and testing processes, so in some cases, the bit rate overhead can be reduced).

[0233] In addition, the probability distribution table of the present application corresponds one to one with the index of the variance, and then, in the process of entropy coding and decoding, the present application can directly determine the corresponding probability distribution table according to the second index, and then, the characteristic value of the feature point in the second feature map can be entropy coded and decoded according to the probability distribution table. After the prior art adjusts the variance, it is necessary to find the nearest preset variance from multiple preset variances (the probability distribution table corresponds one to one with the preset variance); then, the probability distribution table corresponding to the nearest preset variance is used to entropy code and decode the characteristic value of the feature point in the second feature map. Among them, the process of finding the nearest preset variance in the prior art requires a certain amount of calculation, so the present application can also reduce the amount of calculation of entropy coding and decoding, and improve the efficiency of entropy coding and decoding.

[0234] The following Figure 2A Based on, refer to Figure 4A and Figure 5 In the embodiment, another encoding and decoding process of an image (which can be a frame of image in video data or an independent image) is described.

[0235] Figure 4A FIG. 1 is a schematic diagram of an exemplary encoding process.

[0236] S401, obtaining an image to be encoded.

[0237] S402, extracting features from the image to be encoded to obtain a first feature map.

[0238] S403, determining probability distribution parameters corresponding to the first feature map, where the probability distribution parameters include a first index of the variance.

[0239] Illustratively, S401 to S403 may refer to the description of S201 to S203 above, which will not be repeated here.

[0240] S404: Perform a first adjustment process on the first feature map according to the first step length to obtain a second feature map.

[0241] Exemplarily, S404 may be executed by a first adjustment module.

[0242] Exemplarily, the first step length may be pre-set according to factors such as image quality requirements, bit rate requirements, and channel quality. Thus, after obtaining the first feature map, the first adjustment module may perform a first adjustment process on the first feature map according to the first step length to obtain the second feature map.

[0243] Exemplarily, a mask image corresponding to the first feature map may be generated first, and then the first feature map may be subjected to a first adjustment process according to the mask image corresponding to the first feature map and the first step length to obtain a second feature map. The mask image may also be referred to as a binary mask image, which is used to extract regions of interest and shield regions of no interest; the size of the mask image corresponding to the first feature map is the same as the size of the first feature map, and the mask image corresponding to the first feature map may include a mask value corresponding to each of the N feature points of the first feature map, and the mask value corresponding to a feature point is either 1 or 0. The first feature map is multiplied by the mask image corresponding to the first feature map, and the feature points corresponding to the mask value of 1 are extracted, and the first adjustment process and the second adjustment process are performed; the feature points corresponding to the mask value of 0 are shielded.

[0244] Exemplarily, a mask image corresponding to the first feature map can be generated according to the N first indices corresponding to the N feature points in the first feature map. Exemplarily, the first feature map can be divided into S feature blocks, where S is a positive integer; then, the Z first indices corresponding to the Z feature points contained in the S feature blocks are pooled to obtain S pooled values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; then, a mask image corresponding to the first feature map is generated according to the S pooled values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain.

[0245] Specifically, for each channel of the first feature map, the feature map can be divided into p feature blocks of size L1*L2, so that the first feature map can be divided into S (S=p*c) feature blocks. Among them, L1 and L2 can be equal or unequal, and this application does not limit this. Then, for a feature block, the first index of the variance corresponding to the Y feature points contained in the feature block can be pooled to obtain the pooling value corresponding to the feature block; the specific pooling process can refer to the following formula (10):

[0246]

[0247] In formula (10), index q Represents the pooling value corresponding to the qth feature block, where q is a positive integer less than or equal to M.

[0248] In formula (10), MaxPool represents the maximum pooling process, pooling_block_size represents the size of the feature block, i.e., L1*L2; index represents the first index of the variance, and the meaning of MaxPool(pooling_block_size,index) is: perform maximum pooling process on the first index of the variance corresponding to Y feature points (Y equals L1 times L2) contained in the feature block of size L1*L2, that is, select the maximum value of the first index of the variance among the Y feature points as the pooling value corresponding to the feature block.

[0249] In formula (10), AvgPool represents average pooling processing, and the meaning of AvgPool(pooling_block_size, index) is: average pooling processing is performed on the first index of the variance corresponding to the Y feature points contained in the feature block of size L1*L2, that is, the average value of the first index of the variance corresponding to the Y feature points is calculated as the pooling value corresponding to the feature block.

[0250] In formula (10), MinPool represents minimum pooling processing, and the meaning of MinPool(pooling_block_size, index) is: perform minimum pooling processing on the first index of the variance corresponding to the Y feature points contained in the feature block of size L1*L2, that is, select the minimum value of the first index of the variance among the Y feature points as the pooling value corresponding to the feature block.

[0251] In formula (10), pooling_mode=0, pooling_mode=1 and pooling_mode=2 correspond to three pooling processing strategies; the specific ones can be determined according to the target bit rate, and this application does not impose any limitation on this.

[0252] In this way, S pooling values ​​can be obtained; the S pooling values ​​correspond one-to-one to the S feature blocks.

[0253] Next, a mask image corresponding to the first feature map is generated according to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain. Specifically, the threshold can be set in advance and the threshold can be converted to the logarithmic domain.

[0254] Specifically, the threshold can be converted into the logarithmic domain by referring to formula (11):

[0255]

[0256] In formula (11), threshold is the threshold set according to the variance, log is the threshold for conversion to the logarithmic domain, σ max is the maximum possible variance, σ min is the minimum possible variance.

[0257] It should be noted that when the above pooling value is obtained by average pooling, since the average value of the variance after conversion to the log domain is less than the average value of σ, η can be a positive number less than 1, such as 0.85, so that the mask map determined by the first index of the variance can be guaranteed to be the same or close to the mask map determined by the variance, thereby ensuring that the compression performance remains unchanged. When the above pooling value is obtained by maximum pooling or minimum pooling, η can be 1.

[0258] Afterwards, the S pooling values ​​corresponding to the S feature blocks can be compared with the threshold converted to the logarithmic domain to generate a mask image corresponding to the first feature map; specifically, refer to the following formula (12):

[0259]

[0260] In formula (12), mask q (c i ,h j ,w k ) represents the feature points (c i ,h j ,w k ) The mask value corresponding to greater_flag can be set as required, such as "0" or "1". index q (c i ,h j ,w k ) represents the feature points (c i ,h j ,w k ) corresponds to the first index pooling value of the variance, that is, the pooling value corresponding to the qth feature block.

[0261] For example, when greater_flag = 1, if index q (c i ,h j ,w k ) is greater than threshold log , then the feature points (c i ,h j ,w k ) corresponds to a mask value of 1; otherwise, the feature point (c i ,h j ,w k )The corresponding mask value is 0; In this way, the mask image corresponding to the first feature map can be obtained as follows Figure 4B (1) as shown. Figure 4B(1) shows the mask image corresponding to the feature map (size is 8*8) of a channel of the first feature map, and the size of the feature block is 4*4; the mask value corresponding to each feature point contained in feature block 1 is 1, the mask value corresponding to each feature point contained in feature block 2 is 0, the mask value corresponding to each feature point contained in feature block 3 is 0, and the mask value corresponding to each feature point contained in feature block 4 is 1.

[0262] For example, when greater_flag = 0, if index q (c i ,h j ,w k ) is less than threshold log , then the feature points (c i ,h j ,w k ) corresponds to a mask value of 1; otherwise, the feature point (c i ,h j ,w k ) The corresponding mask value is 0. In this way, the mask image corresponding to the first feature map can be obtained as follows Figure 4B (2) as shown. Figure 4B (2) shows the mask image corresponding to the feature map of a channel of the first feature map (size is 8*8), and the size of the feature block is 4*4; the mask value corresponding to each feature point contained in feature block 1 is 0, the mask value corresponding to each feature point contained in feature block 2 is 1, the mask value corresponding to each feature point contained in feature block 3 is 1, and the mask value corresponding to each feature point contained in feature block 4 is 0.

[0263] Next, the first feature map can be adjusted according to the mask map corresponding to the first feature map and the first step length to obtain the second feature map. Specifically, for each feature point in the N feature points of the first feature map, when the mask value corresponding to the feature point is 1, the first feature value of the feature point can be adjusted according to the first step length; when the mask value corresponding to the feature point is 0, the first feature value of the feature point does not need to be adjusted.

[0264] Among them, Figure 4A In the embodiment of the invention, the first adjustment process is a multiplication operation. It should be noted that the multiplication operation may include two types of calculations, "multiplication" and "division". The present application takes the first adjustment process as "multiplication" as an example for explanation. Specifically, taking a feature point as an example, the process of performing the first adjustment process on the first eigenvalue of the feature point according to the first step length to obtain the second eigenvalue of the feature point can refer to the following formula (13):

[0265] y2(c i ,h j ,w k )=y1(c i ,h j ,w k )*scale (13)

[0266] In formula (13), y2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue; scale is the first step length, y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) is the first eigenvalue of .

[0267] Exemplarily, the second eigenvalues ​​of the N feature points may also be rounded; in this case, formula (14) may be converted to formula (14):

[0268] y2(c i ,h j ,w k )=[y1(c i ,h j ,w k )*scale] (14)

[0269] In formula (14), [] indicates rounding.

[0270] It should be understood that in Figure 4A In the embodiment, the probability distribution parameter may also include N means in the first feature map, and the N means correspond to the N feature points one by one. In this case, the first feature value of each feature point in the N feature points of the first feature map may be subtracted from the corresponding mean to obtain a fourth feature map. Then, the fourth feature map may be subjected to a first adjustment process according to the mask map corresponding to the first feature map and the first step length to obtain a second feature map; the details may refer to the above description, which will not be repeated here.

[0271] S405, performing a second adjustment process on the N first indexes according to the second step length to obtain N second indexes, wherein the second step length is obtained by converting the first step length into the logarithmic domain, and the N second indexes correspond one-to-one to the N feature points.

[0272] Exemplarily, S405 may be executed by the second adjustment module.

[0273] For example, in Figure 4A In the embodiment of , the second adjustment process is an addition operation. It should be noted that the addition operation may include two types of calculations: "addition" and "subtraction". Figure 4A In the embodiment, when the first adjustment process is 'multiplication', the second adjustment process is 'addition'; when the first adjustment process is 'division', the second adjustment process is 'subtraction'; this application takes the second adjustment process 'addition' as an example for explanation.

[0274] For example, the first step length may be converted to the logarithmic domain to obtain the second step length; the following formula (15) may be referred to:

[0275]

[0276] In formula (15), scale is the first step length, scale log is the second step length, σ max is the maximum possible variance, σ min is the minimum possible variance.

[0277] Then, the N first indexes can be subjected to a second adjustment process according to the mask image corresponding to the first feature map and the second step length to obtain N second indexes. Specifically, for each feature point in the N feature points of the first feature map, when the mask value corresponding to the feature point is 1, the first index of the variance corresponding to the feature point can be subjected to a second adjustment process according to the second step length; when the mask value corresponding to the feature point is 0, the first eigenvalue of the variance corresponding to the feature point does not need to be subjected to a second adjustment process.

[0278] Specifically, taking a feature point as an example, a second adjustment process is performed on the first index of the variance corresponding to the feature point according to the second step length to obtain the second index of the variance corresponding to the feature point, which can be referred to the following formula (16):

[0279] index2(c i ,h j ,w k )=index1(c i ,h j ,w k )+scale log (16)

[0280] In formula (16), index2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the second index of the variance; scale logis the second step length, index1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the first index of the variance.

[0281] Afterwards, the second index can be rounded. At this time, formula (16) can be converted to formula (17):

[0282] index2(c i ,h j ,w k )=[index1(c i ,h j ,w k )+scale log ] (17)

[0283] In formula (17), [] indicates rounding.

[0284] S406: Perform entropy encoding on the second feature map according to the N second indexes to obtain a code stream.

[0285] Exemplarily, S406 may refer to the description of S206 above, which will not be repeated here.

[0286] Figure 5 FIG. 4 is a schematic diagram showing an exemplary decoding process. Figure 5 The decoding process and Figure 4A The encoding process corresponds to .

[0287] S501, receiving a code stream.

[0288] S502, determining probability distribution parameters corresponding to the first feature map, the probability distribution parameters including N first indexes, the N first indexes corresponding one-to-one to N variances, and the N variances corresponding one-to-one to N feature points.

[0289] Illustratively, S501 to S502 may refer to the description of S301 to S302 above, which will not be repeated here.

[0290] S503 , performing a second adjustment process on the N first indexes according to the second step length to obtain N second indexes, where the N second indexes correspond one-to-one to the N feature points.

[0291] For example, S503 may refer to the description of S405 above, which will not be described in detail herein. S503 may be executed by the fourth adjustment module.

[0292] S504: Perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map.

[0293] Exemplarily, S504 may refer to the description of 304 above, which will not be repeated here.

[0294] S505: Perform a third adjustment process on the second feature map according to the first step length to obtain a first feature map.

[0295] Exemplarily, S505 may be executed by a third adjustment module.

[0296] For example, Figure 5 The third adjustment process in the embodiment is: Figure 4A The reverse process of the first adjustment process in the embodiment.

[0297] Exemplarily, when the first adjustment process in S404 is "multiplication", the third adjustment process in S505 is "division"; when the first adjustment process in S404 is "division", the fifth adjustment process in S505 is "multiplication". This application takes the third adjustment process as "division" as an example for explanation.

[0298] Exemplarily, taking a feature point as an example, the third adjustment processing method for the feature point according to the first step can refer to the following formula (18):

[0299]

[0300] In formula (18), y2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue; scale is the first step length, y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) is the first eigenvalue of .

[0301] When the encoding end performs the first adjustment processing on the fourth feature map according to the first step length to obtain the second feature map, the decoding end performs the third adjustment processing on the second feature map according to the first step length to obtain the fourth feature map; then, the fourth eigenvalue of each feature point in the N feature points of the fourth feature map can be added to the corresponding mean value to obtain the first feature map; this can be referred to as shown in the above formula (9), which will not be repeated here.

[0302] S506: reconstruct the first feature map to obtain a reconstructed image of the image.

[0303] Exemplarily, S506 may refer to the description of S306 above, which will not be repeated here.

[0304] The following Figure 2A Based on, refer to Figure 6 and Figure 7 In the embodiment, another encoding and decoding process of an image (which can be a frame of image in video data or an independent image) is described.

[0305] Figure 6 FIG. 1 is a schematic diagram of an exemplary encoding process.

[0306] S601, obtaining an image to be encoded.

[0307] S602: Extract features from the image to be encoded to obtain a first feature map.

[0308] S603, determine the probability distribution parameters corresponding to the first feature map, the probability distribution parameters include N first indexes and N means, the N means correspond one-to-one to the N feature points, the N first indexes correspond one-to-one to the N variances, and the N variances correspond one-to-one to the N feature points.

[0309] Illustratively, S601 to S603 may refer to the description of S201 to S203 above, which will not be repeated here.

[0310] S604: Subtract the corresponding mean from the first eigenvalue of each feature point in the N feature points of the first feature map to obtain a fourth feature map.

[0311] Exemplarily, the fourth feature map includes fourth eigenvalues ​​of N feature points, and the fourth eigenvalue is as r4[c i ,h j ,w k ] is shown in the figure and will not be repeated here.

[0312] S605: Perform a first adjustment process on the fourth feature map according to the first gain vector to obtain a second feature map.

[0313] Exemplarily, the first adjustment process in S605 is the same as the first adjustment process in S204 described above, and will not be described in detail herein.

[0314] Exemplarily, the fourth feature map includes fourth eigenvalues ​​of N feature points, and the calculation method of the fourth eigenvalues ​​can be shown in the following formula (19):

[0315]

[0316] In formula (19), r2(c i ,hj ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue, r5(c i ,h j ,w k ) represents the feature point [c i ,h j ,w k ] the fourth eigenvalue, m b is the first gain vector.

[0317] S606: Perform a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes.

[0318] Exemplarily, the second adjustment process in S606 is the same as the second adjustment process in S205 described above, and will not be described in detail herein.

[0319] S607, performing a third adjustment process on the second feature map according to the first step length to obtain a fifth feature map.

[0320] Exemplarily, the third adjustment process in S607 is the same as the first adjustment process in S404 above, and will not be described in detail herein.

[0321] Exemplarily, the fifth feature map includes fifth eigenvalues ​​of N feature points, and the calculation method of the fifth eigenvalue can be shown in the following formula (20):

[0322] r5(c i ,h j ,w k )=r2(c i ,h j ,w k )*scale (20)

[0323] In formula (20), r2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue, r5(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), where scale is the first step length.

[0324] Exemplarily, the fifth eigenvalue of the feature point may also be rounded; in this case, formula (20) may be converted to formula (21):

[0325] r5(c i ,h j ,w k )=[r2(c i ,h j ,w k )*scale] (21)

[0326] In formula (21), [] indicates rounding.

[0327] S608, performing a fourth adjustment process on the N second indexes according to the second step length to obtain N third indexes.

[0328] Exemplarily, the fourth adjustment process in S608 is the same as the second adjustment process in S405 above, and will not be described in detail herein.

[0329] Exemplarily, the third index may be calculated as shown in the following formula (22):

[0330] index3(c i ,h j ,w k )=index2(c i ,h j ,w k )+scale log (twenty two)

[0331] In formula (22), index3(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the third index of the variance; scale log is the second step length, index2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ) corresponds to the second index of the variance.

[0332] Afterwards, the third index can be rounded. At this time, formula (22) can be converted to formula (23):

[0333] index3(c i ,h j ,w k )=[index2(ci ,h j ,w k )+scale log ] (twenty three)

[0334] In formula (23), [] indicates rounding.

[0335] Among them, before executing S607 and S608, a mask image corresponding to the first feature map can be generated according to the N second indexes; specifically, the first feature map can be divided into S feature blocks, where S is a positive integer; the Z second indexes corresponding to the Z feature points contained in the S feature blocks are pooled to obtain S pooling values ​​corresponding to the S feature blocks; based on the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, a mask image corresponding to the first feature map is generated.

[0336] It should be understood that the present application does not limit the execution order of S605 , S606 , S607 , and S608 .

[0337] S609: Perform entropy encoding on the fifth feature map according to the N third indexes to obtain a code stream.

[0338] Exemplarily, S609 may refer to the description of S306 above, which will not be repeated here.

[0339] Figure 7 FIG. 4 is a schematic diagram showing an exemplary decoding process. Figure 7 The decoding process and Figure 6 The encoding process corresponds to .

[0340] S701, receiving a code stream.

[0341] Exemplarily, the code stream may include encoded data of the fifth feature map (obtained by entropy encoding the fourth feature map (ie, the fifth feature map) that has undergone the first adjustment process, the third adjustment process, and rounding) and encoded data of the estimation information.

[0342] S702, determine probability distribution parameters corresponding to the first feature map, the probability distribution parameters include N first indexes and N means, the N first indexes correspond one-to-one to N variances, the N variances correspond one-to-one to N feature points, and the N means correspond one-to-one to N feature points.

[0343] Illustratively, S701 to S702 may refer to the description of S301 to S302 above, which will not be repeated here.

[0344] S703 , performing a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, where the N second indexes correspond one-to-one to the N feature points.

[0345] S704 , performing a fourth adjustment process on the N second indexes according to the second step length to obtain N third indexes, where the N third indexes correspond one-to-one to the N feature points.

[0346] Illustratively, S703 to S704 may refer to the description of S604 and S608 above, which will not be repeated here.

[0347] It should be noted that this application does not limit the execution order of S703 and S704.

[0348] S705 , entropy decoding is performed on the encoded data of the fifth feature map in the code stream according to the N third indexes to obtain the fifth feature map.

[0349] Exemplarily, S705 may refer to the description of S304 above, which will not be repeated here.

[0350] S706: Perform a fifth adjustment process on the fifth feature map according to the first gain vector to obtain a second feature map.

[0351] Exemplarily, the fifth adjustment process of S706 is the same as the third adjustment process of S305, and will not be described in detail herein.

[0352] Exemplarily, the second feature map includes the second eigenvalues ​​of N feature points, and the fourth eigenvalue can be calculated as shown in the following formula (24):

[0353]

[0354] In formula (24), r2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s second eigenvalue, r5(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), is the first gain vector.

[0355] S707, performing a sixth adjustment process on the second feature map according to the first step length to obtain a fourth feature map.

[0356] Illustratively, the sixth adjustment process of S707 is the same as the third adjustment process of S505, and will not be described in detail herein.

[0357] Exemplarily, the fourth feature map includes fourth eigenvalues ​​of N feature points, and the calculation method of the fourth eigenvalue can be shown in the following formula (25):

[0358]

[0359] In formula (25), r4(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s fourth eigenvalue, r2(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), and scale is the first step length.

[0360] It should be noted that this application does not limit the execution order of S706 and S707.

[0361] S708 , adding the corresponding mean value to the fourth eigenvalue of each feature point in the N feature points of the fourth feature map to obtain the first feature map.

[0362] Exemplarily, the first feature map includes first eigenvalues ​​of N feature points, and the calculation method of the first eigenvalues ​​can be shown in the following formula (26):

[0363] y1(c i ,h j ,w k )=r4(c i ,h j ,w k )+μ(c i ,h j ,w k ) (26)

[0364] In formula (26), r4(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k )’s fourth eigenvalue, y1(c i ,h j ,w k ) represents the feature point (c i ,h j ,w k ), μ(c i ,h j ,w k ) is the feature point (c i ,h j ,wk ) corresponding to the mean.

[0365] S709: reconstruct the first feature map to obtain a reconstructed image of the image.

[0366] Exemplarily, S709 may refer to the description of S306 above, which will not be repeated here.

[0367] Exemplary, tested, under the same quality, using Figure 6 The bit rate obtained by the encoding method is -0.26% compared with the bit rate obtained by the encoding method using the prior art; that is, under the same quality, the bit rate loss of the present application is lower.

[0368] exist Figure 2B On the basis of the embodiment, the first index can be multiplied by 2 to the power of f (f is a positive integer) to obtain a fourth index; and the second gain vector can be multiplied by 2 to the power of f to obtain a third gain; then, the fourth index can be subtracted from the third gain to obtain a fifth index; then, the second feature map is entropy encoded according to the fifth index to obtain a code stream. In other words, the present application performs operations at the order of 2 to the power of f. The prior art first multiplies the variance 1 output by the entropy estimation network by 2 to the power of f1 (f1 is a positive integer) to obtain variance 2, and multiplies the first gain vector by 2 to the power of f2 (f2 is a positive integer) to obtain a fourth gain; then, the variance 2 is multiplied by the fourth gain to adjust the variance; that is, the prior art performs operations at the order of 2 to the power of (f1+f2). It can be seen that the order of magnitude of the variance adjustment of the present application is small, which can reduce the computing memory; and the quantization loss after the variance adjustment of the present application is small, and the bit rate loss is less.

[0369] Figure 8 FIG. 1 is a schematic diagram of an exemplary encoding device. Figure 8 The encoding device in can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0370] Reference Figure 8 , the encoding device may include:

[0371] A first acquisition module 801 is used to acquire an image to be encoded;

[0372] A first encoding network 802 is used to extract features from the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w. The first feature map includes N feature points, N is the product of c, h, and w, and c, h, and w are positive integers;

[0373] A first entropy estimation network 803, used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0374] A first adjustment module 804 is used to perform a first adjustment process on the first feature map according to the first gain vector to obtain a second feature map, wherein the size of the first gain vector is c*1, and the second feature map includes N feature points;

[0375] The first adjustment module 805 is further used to perform a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0376] The first entropy coding network 806 is further used to perform entropy coding on the second feature map according to the N second indexes to obtain a code stream.

[0377] Fig. 9 FIG. 1 is a schematic diagram of an exemplary encoding device. Fig. 9 The decoding device in can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0378] Reference Fig. 9 , the decoding device may include:

[0379] A first receiving module 901 is used to receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the number of channels of the first feature map is c, the height is h, and the width is w, N is the product of c, h, and w, and c, h, and w are positive integers;

[0380] A second entropy estimation network 902, used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0381] A second adjustment module 903 is used to perform a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0382] A first entropy decoding network 904 is used to perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map;

[0383] The second adjustment module 905 is further used to perform a fifth adjustment process on the second feature map according to the first gain vector to obtain the first feature map, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, and the fifth adjustment process is an inverse process of the first adjustment process;

[0384] The first decoding network 906 is used to reconstruct the first feature map to obtain a reconstructed image of the image.

[0385] Fig.10 FIG. 1 is a schematic diagram of an exemplary encoding device. Fig.10 The encoding device in can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0386] Reference Fig.10 , the encoding device may include:

[0387] The second acquisition module 1001 is used to acquire the image to be encoded;

[0388] The second encoding network 1002 is used to extract features of the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, c, the first feature map includes N feature points, N is the product of c, h and w, and c, h and w are positive integers;

[0389] A third entropy estimation network 1003 is used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0390] A third adjustment module 1004 is used to perform a first adjustment process on the first feature map according to the first step length to obtain a second feature map, wherein the second feature map includes N feature points;

[0391] The third adjustment module 1005 is further used to perform a second adjustment process on the first indexes corresponding to the N feature points according to the second step length to obtain N second indexes, wherein the second step length is obtained by converting the first step length into the logarithmic domain, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0392] The second entropy coding module 1005 is further used to perform entropy coding on the second feature map according to the N second indexes to obtain a code stream.

[0393] Fig.11FIG. 1 is a schematic diagram of an exemplary encoding device. Fig.11 The decoding device in can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0394] Reference Fig.11 , the decoding device may include:

[0395] A second receiving module 1101 is used to receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on the image, the number of channels of the first feature map is c, the height is h, and the width is w, N is the product of c, h, and w, and c, h, and w are positive integers;

[0396] A fourth entropy estimation network 1102, used to determine probability distribution parameters corresponding to the first feature map, wherein the probability distribution parameters include N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to N feature points;

[0397] A fourth adjustment module 1103 is used to perform a second adjustment process on the N first indexes according to the second step length to obtain N second indexes, where the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one;

[0398] A second entropy decoding network 1104 is used to perform entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain a second feature map;

[0399] The fourth adjustment module 1105 is further used to perform a third adjustment process on the second feature map according to the first step length to obtain the first feature map, wherein the second step length is obtained by converting the first step length into the logarithmic domain, and the third adjustment process is an inverse process of the first adjustment process;

[0400] The second decoding network 1106 is used to reconstruct the first feature map to obtain a reconstructed image of the image.

[0401] In one example, Fig.12 A schematic block diagram of a device 1200 according to an embodiment of the present application is shown. The device 1200 may include: a processor 1201 and a transceiver / transceiver pin 1202 , and optionally, a memory 1203 .

[0402] The various components of the device 1200 are coupled together via a bus 1204, wherein the bus 1204 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus 1204 in the figure.

[0403] Optionally, the memory 1203 may be used to store instructions in the aforementioned method embodiment. The processor 1201 may be used to execute instructions in the memory 1203, and control the receiving pin to receive a signal, and control the sending pin to send a signal.

[0404] The apparatus 1200 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0405] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0406] The embodiment of the present application also provides a chip, including one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the above-mentioned related method steps implement the steps of the method in the above-mentioned embodiment. The interface circuit is a transceiver / transceiver pin 1202.

[0407] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0408] This embodiment further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer executes the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0409] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0410] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0411] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0412] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0413] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0414] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0415] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0416] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0417] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0418] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (ReadOnly Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0419] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0420] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A coding method, characterized in that: The method comprises: Obtaining an image to be encoded; Performing feature extraction on the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, the first feature map includes N feature points, N is the product of c, h, and w, and c, h, and w are positive integers; Determine a probability distribution parameter corresponding to the first feature map, wherein the probability distribution parameter includes N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to the N feature points; Performing a first adjustment process on the first feature map according to the first gain vector to obtain a second feature map, wherein the size of the first gain vector is c*1, and the second feature map includes the N feature points; Performing a second adjustment process on the N first indexes according to a second gain vector to obtain N second indexes, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond one-to-one to the N feature points; The second feature map is entropy encoded according to the N second indexes to obtain a code stream.

2. The method according to claim 1, characterized in that Before entropy encoding the second feature map according to the N second indexes to obtain a bitstream, the method further includes: Performing a third adjustment process on the second feature map according to the first step length; A fourth adjustment process is performed on the N second indexes according to a second step length, wherein the second step length is obtained by converting the first step length into a logarithmic domain, and the fourth adjustment process is an addition operation.

3. The method according to claim 2, characterized in that The method further comprises: Generate a mask image corresponding to the first feature image, wherein the mask image is used for the third adjustment process and / or the fourth adjustment process.

4. The method according to claim 3, characterized in that The step of generating a mask map of the first feature map includes: Divide the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; Performing pooling processing on the Z second indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; A mask image corresponding to the first feature map is generated according to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain.

5. The method according to any one of claims 1 to 4, characterized in that: The probability distribution parameters further include N means, and the N means correspond one-to-one to the N feature points; Before performing a first adjustment process on the first characteristic map according to the first gain vector to obtain a second characteristic map, the method further includes: Subtract the corresponding mean from the feature value of each feature point in the first feature map.

6. The method according to claim 4, characterized in that The pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

7. The method according to any one of claims 1 to 6, characterized in that: Before entropy encoding the second feature map according to the N second indexes to obtain a bitstream, the method further includes: The N second indexes are rounded.

8. A decoding method, characterized in that: The method comprises: Receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on an image, the first feature map includes the N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, and the width is w, N is the product of c, h, and w, and c, h, and w are positive integers; Determine a probability distribution parameter corresponding to the first feature map, wherein the probability distribution parameter includes N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to the N feature points; Performing a second adjustment process on the N first indexes according to the second gain vector to obtain N second indexes, wherein the size of the second gain vector is c*1, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one; Performing entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain the second feature map; Performing a fifth adjustment process on the second feature map according to the first gain vector to obtain the first feature map, wherein the second gain vector is obtained by converting the first gain vector into a logarithmic domain, and the fifth adjustment process is an inverse process of the first adjustment process; The first feature map is reconstructed to obtain a reconstructed image of the image.

9. The method according to claim 8, characterized in that The bitstream includes the encoded data of N feature points in the second feature map after the third adjustment process; Before reconstructing the first feature map to obtain a reconstructed image of the image, the method further includes: Performing a sixth adjustment process on the first feature map according to the first step length, wherein the sixth adjustment process is a reverse process of the third adjustment process; Before entropy decoding the encoded data of the N feature points according to the N second indexes to obtain the second feature map, the method further includes: A fourth adjustment process is performed on the N second indexes according to a second step length, wherein the second step length is obtained by converting the first step length into a logarithmic domain, and the fourth adjustment process is an addition operation.

10. The method according to claim 9, characterized in that The method further comprises: Generate a mask image corresponding to the first feature image, wherein the mask image is used for the sixth adjustment process and / or the fourth adjustment process.

11. The method according to claim 10, characterized in that The step of generating a mask map of the first feature map includes: Divide the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; Performing pooling processing on the Z second indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; According to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, a mask image corresponding to the first feature map is determined.

12. The method according to any one of claims 8 to 11, characterized in that The probability distribution parameters further include N means, and the N means correspond one-to-one to the N feature points; Before reconstructing the first feature map to obtain a reconstructed image of the image, the method further includes: The feature value of each feature point in the first feature map is added with the corresponding mean value.

13. The method according to claim 11, characterized in that The pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

14. The method according to any one of claims 8 to 13, characterized in that Before entropy decoding the encoded data of the N feature points according to the N second indexes to obtain the second feature map, the method further includes: The N second indexes are rounded.

15. A coding method, characterized in that: The encoding method comprises: Obtaining an image to be encoded; Performing feature extraction on the image to be encoded to obtain a first feature map, wherein the number of channels of the first feature map is c, the height is h, and the width is w, the first feature map includes N feature points, N is the product of c, h, and w, and c, h, and w are positive integers; Determine a probability distribution parameter corresponding to the first feature map, wherein the probability distribution parameter includes N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to the N feature points; Performing a first adjustment process on the first feature map according to the first step length to obtain a second feature map, wherein the second feature map includes the N feature points; Performing a second adjustment process on the first indexes corresponding to the N feature points according to a second step length to obtain N second indexes, wherein the second step length is obtained by converting the first step length into a logarithmic domain, the second adjustment process is an addition operation, and the N second indexes correspond to the N feature points one by one; The second feature map is entropy encoded according to the N second indexes to obtain a code stream.

16. The method according to claim 15, characterized in that The method further comprises: Generate a mask image corresponding to the first feature image, wherein the mask image is used for the first adjustment process and / or the second adjustment process.

17. The method according to claim 16, characterized in that The step of generating a mask map of the first feature map includes: Divide the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; Performing pooling processing on the Z first indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; According to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, a mask image corresponding to the first feature map is determined.

18. The method according to any one of claims 15 to 17, characterized in that The probability distribution parameters further include N means, and the N means correspond one-to-one to the N feature points; Before performing a first adjustment process on the first feature map according to the first step length to obtain a second feature map, the method further includes: Subtract the corresponding mean from the feature value of each feature point in the first feature map.

19. The method according to claim 17, characterized in that The pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

20. The method according to any one of claims 15 to 19, characterized in that Before entropy encoding the second feature map according to the N second indexes to obtain a bitstream, the method further includes: The N second indexes are rounded.

21. A decoding method, characterized in that: The decoding method comprises: Receive a code stream, wherein the code stream includes encoded data of N feature points in a second feature map, the second feature map is obtained by performing a first adjustment process on the first feature map, the first feature map is obtained by performing feature extraction on an image, the first feature map includes the N feature points, the number of channels of the first feature map and the second feature map is c, the height is h, and the width is w, N is the product of c, h, and w, and c, h, and w are positive integers; Determine a probability distribution parameter corresponding to the first feature map, wherein the probability distribution parameter includes N first indexes, the N first indexes correspond one-to-one to N variances, and the N variances correspond one-to-one to the N feature points; Performing a second adjustment process on the N first indexes according to a second step length to obtain N second indexes, wherein the second adjustment process is an addition operation, and the N second indexes correspond one-to-one to the N feature points; Performing entropy decoding on the encoded data of the N feature points according to the N second indexes to obtain the second feature map; Performing a third adjustment process on the second feature map according to the first step length to obtain the first feature map, wherein the second step length is obtained by converting the first step length into a logarithmic domain, and the third adjustment process is an inverse process of the first adjustment process; The first feature map is reconstructed to obtain a reconstructed image of the image.

22. The method according to claim 21, characterized in that The method further comprises: Generate a mask image corresponding to the first feature image, wherein the mask image is used for the third adjustment process and / or the second adjustment process.

23. The method according to claim 22, characterized in that The step of generating a mask map of the first feature map includes: Divide the first feature map into S feature blocks, wherein the height of the feature block is L1 and the width is L2, and L1, L2 and S are positive integers; Performing pooling processing on the Z first indexes corresponding to the Z feature points contained in the S feature blocks to obtain S pooling values ​​corresponding to the S feature blocks, where Z is the product of L1, L2 and S; According to the S pooling values ​​corresponding to the S feature blocks and the threshold converted to the logarithmic domain, a mask image corresponding to the first feature map is determined.

24. The method according to any one of claims 21 to 23, characterized in that The probability distribution parameters further include N means, and the N means correspond one-to-one to the N feature points; Before reconstructing the first feature map to obtain a reconstructed image of the image, the method further includes: The feature value of each feature point in the first feature map is added with the corresponding mean value.

25. The method according to claim 23, characterized in that The pooling process includes at least one of the following: average pooling process, maximum pooling process or minimum pooling process.

26. The method according to any one of claims 21 to 25, characterized in that Before entropy decoding the encoded data of the N feature points according to the N second indexes to obtain the second feature map, the method further includes: The N second indexes are rounded.

27. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the encoding method according to any one of claims 1 to 7, or executes the encoding method according to any one of claims 15 to 20.

28. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the decoding method according to any one of claims 8 to 14, or executes the encoding method according to any one of claims 21 to 26.

29. A chip, characterized in that: The method comprises one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the method according to any one of claims 1 to 7 are executed, or the steps of the method according to any one of claims 15 to 20 are executed.

30. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the method as described in any one of claims 8 to 14 are executed, or the steps of the method as described in any one of claims 21 to 26 are executed.

31. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the method according to any one of claims 1 to 26.

32. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method according to any one of claims 1 to 26 are performed.

Images