Video image encoding and decoding method and device

Abstract

The present application provides a video image-based encoding and decoding method and device and a video or image compression technology field based on artificial intelligence (AI), specifically relating to a neural network-based video compression technology field. The encoding method includes: obtaining a first image, which is an image to be encoded or a decoded image, performing probability estimation based on first context information to obtain a first probability estimation result; the first context information is obtained from the first image; and writing the first probability estimation result into a compressed bitstream. The decoding end performs sampling based on the probability estimation result to obtain an estimation coefficient, and obtains a reconstructed image based on the estimation coefficient obtained by sampling. High-quality images can be obtained by using the present application.

Description

Technical Field

[0001] The present application relates to the field of video encoding and decoding, and in particular to a method and device for encoding and decoding video images. Background Art

[0002] A digital image is image information recorded as digital signals. A digital image (hereafter referred to as an image) can be viewed as a two-dimensional array with M rows and N columns, consisting of M x N samples. The location of each sample is called a sampling position, and the value of each sample is called a sampling value.

[0003] In applications such as image storage and transmission, image encoding is often required to reduce storage capacity and transmission bandwidth. Image encoding involves two steps: encoding and decoding. A typical encoding process generally includes three steps: transform, quantization, and entropy coding. For an image to be encoded, the first step is to decorrelate the image through transforms to obtain transform coefficients with a more concentrated energy distribution; the second step is to quantize the transform coefficients to obtain quantized coefficients; and the third step is to entropy encode the quantized coefficients to obtain a compressed bitstream. Corresponding to the encoding process, a typical decoding process involves the decoder, after receiving the compressed bitstream, performing three steps: entropy decoding, inverse quantization, and inverse transform to obtain a reconstructed image.

[0004] In the decoding methods of the prior art, entropy decoding, inverse quantization and inverse transformation are generally deterministic processes, that is, decoding a compressed code stream will obtain a unique reconstructed image, and the quality of the reconstructed image is not high under certain evaluation indicators. Summary of the Invention

[0005] The embodiments of the present application provide a video image encoding and decoding method and related equipment, which can improve the quality of the image.

[0006] These and other objects are achieved by the subject matter of the independent claims. Further implementations are apparent from the dependent claims, the detailed description and the accompanying drawings.

[0007] Particular embodiments are outlined in the accompanying independent claims, further embodiments are outlined in the dependent claims.

[0008] Based on the first aspect, the present application relates to a method for encoding a video image. The method is performed by an encoding device, and the method includes:

[0009] A first image is obtained, where the first image is an image to be encoded or a decoded image, and a probability estimation is performed based on first context information to obtain a first probability estimation result; the first context information is obtained from the first image; and the first probability estimation result is written into a compressed code stream.

[0010] The first context information may be pixels in the first image or coefficients in a first transformed image obtained by transforming the first image.

[0011] Probability estimation is performed at the encoding end to obtain a probability estimation result, and the probability estimation result is transmitted to the decoding end, so that the decoding end can obtain a high-quality image by sampling based on the probability estimation result.

[0012] In one possible design, the method of this embodiment further includes:

[0013] Acquiring a second image, where the second image is an image to be encoded or a decoded image, and the second image is different from the first image; and performing probability estimation based on the first context information to obtain a first probability estimation result, including:

[0014] The first probability estimation result is obtained by performing probability estimation based on the first context information and the second context information, wherein the second context information is obtained from the second image.

[0015] By introducing the second context information, a probability estimation result with higher accuracy can be obtained, so that the decoding end can obtain an image with better quality by sampling based on the probability estimation result.

[0016] In one possible design, performing probability estimation based on the first context information to obtain a first probability estimation result includes:

[0017] A probability estimation is performed based on the context information of the first data to obtain a probability estimation result of the first data; a probability estimation is performed based on the context information of the second data to obtain a probability estimation result of the second data; wherein the first data and the second data are obtained based on the first image; the first context information includes the context information of the first data and the context information of the second data.

[0018] The encoding end calculates the probability estimation results of each data in the first image one by one, and transmits the probability estimation results of each data to the decoding end, so that the decoding end can accurately sample based on the probability estimation results of each data, thereby obtaining a reconstructed image with higher quality.

[0019] In one possible design, the first probability estimation result includes a probability estimation result of a first preset area, the first preset area includes first data and second data, the first preset area is located in the first image, or in an image obtained by transforming the first image, and performing probability estimation based on the first context information to obtain the first probability estimation result includes:

[0020] A probability estimation is performed based on the context information of the first data to obtain a probability estimation result of the first data; a probability estimation is performed based on the context information of the second data to obtain a probability estimation result of the second data, wherein the first context information includes the context information of the first data and the context information of the second data; a probability estimation result of a first preset area is obtained based on the probability estimation result of the first data and the probability estimation result of the second data, wherein the first probability estimation result includes the probability estimation result of the first preset area.

[0021] The first preset area is an image block in the first image, or a subband obtained by performing a wavelet transform on the first image, or a frequency band obtained by performing a discrete cosine transform (DCT) on the first image, or a transform block obtained by performing a DCT on the first image, or a channel in a three-dimensional feature map obtained by performing feature extraction on the first image.

[0022] Wherein, performing DCT transformation on the first image in units of one or more image blocks can obtain one or more transformation blocks.

[0023] For data in a preset area, the encoder uses one probability estimation result as the probability estimation result for all data in the preset area, so that only one probability estimation result needs to be transmitted during transmission, thereby reducing the number of transmission code streams and the resources required to transmit the code streams.

[0024] In one possible design, the first probability estimation result includes a probability estimation result of a second preset area, the second preset area is located in the first image, or in an image obtained by transforming the first image, the first context information includes the context information of the second preset area, and the first probability estimation result is obtained by performing probability estimation based on the first context information, including: performing probability estimation based on the context information of the second preset area to obtain a probability estimation result of the second preset area, and the first probability estimation result includes the probability estimation result of the second preset area.

[0025] The second preset area is an image block in the first image, or a subband obtained by performing wavelet transform on the first image, or a frequency band obtained by performing DCT on the first image, or a transform block obtained by performing DCT on the first image, or a channel in a three-dimensional feature map obtained by performing feature extraction on the first image.

[0026] Wherein, performing DCT transformation on the first image in units of one or more image blocks can obtain one or more transformation blocks.

[0027] For data in a preset area, the encoder uses one probability estimation result as the probability estimation result for all data in the preset area, so that only one probability estimation result needs to be transmitted during transmission, thereby reducing the number of transmission code streams and the resources required to transmit the code streams.

[0028] In one possible design, the encoding method further includes:

[0029] The value of the first identifier of the first preset area is set to the first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; the probability estimation result of the first preset area is saved in a probability estimation result set, and the index of the probability estimation result of the first preset area in the probability estimation result set is recorded; the probability estimation result is written into the compressed code stream, including: writing the probability estimation result set, the index, the size information of the first preset area and the first identifier into the compressed code stream.

[0030] For multiple probability estimation results for multiple preset regions, the encoder saves the probability estimation results for the multiple preset regions into a probability estimation result set and records the position (i.e., index) of the probability estimation result for each preset region in the probability estimation result set. This allows the decoder to accurately determine the probability estimation result for each preset region from the probability estimation result set obtained based on bitstream decoding based on the index, thereby ensuring decoding accuracy. By introducing size information to indicate the number of times sampling is required based on the probability estimation result of the first preset region when sampling to obtain the estimation coefficients in the first preset region, all estimation coefficients in the first preset region are obtained.

[0031] In one possible design, the encoding method further includes:

[0032] The value of the first identifier of the first preset area is set to the first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; the probability estimation result of the first preset area is preprocessed according to the scaling factor of the first preset area to obtain a processed probability estimation result, the processed probability estimation result is saved in a probability estimation result set, and the index of the processed probability estimation result in the probability estimation result set is recorded; the probability estimation result is written into the compressed code stream, including: writing the probability estimation result set, the index, the size information of the first preset area and the first identifier into the compressed code stream.

[0033] The encoder preprocesses the probability estimation results of the first preset region to obtain a processed probability estimation result. The decoder then samples the processed probability estimation result to obtain a reconstructed image. By setting different preprocessing methods, reconstructed images of different qualities can be obtained, such as images with high subjective quality or images with high objective quality.

[0034] In one possible design, the encoding method further includes:

[0035] The value of the first identifier of the first preset area is set to a first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; the first probability estimation result is written into the compressed code stream, including: writing the probability estimation result of the first preset area, the size information of the first preset area and the first identifier into the code stream.

[0036] By setting the value of the first identifier of the first preset area to the first value, it is indicated that the decoding end uses the probability estimation result of the first preset area when sampling to obtain the estimation coefficients in the first preset area after decoding the probability estimation result of the first preset area; by introducing size information to indicate the number of times that sampling needs to be performed based on the probability estimation result of the first preset area when sampling to obtain the estimation coefficients in the first preset area, thereby obtaining all the estimation coefficients in the first preset area.

[0037] In one possible design, the encoding method further includes:

[0038] The probability estimation result of the first data is preprocessed to obtain a processed probability estimation result.

[0039] In one possible design, the probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes:

[0040] The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

[0041] In one possible design, the probability estimation result of the first data includes the mean and variance of the Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes: preprocessing the variance of the Gaussian distribution according to a scaling factor of the first data to obtain a processed variance, wherein the processed probability estimation result includes the mean and the processed variance of the Gaussian distribution; then

[0042] The scaling factor of the first data and the scaling factor of the second data are the same; or, the scaling factor of the first data and the scaling factor of the second data are different; or,

[0043] Preprocessing a probability estimation result of the first data based on content information of a preset region to which the first data belongs to obtain a processed probability estimation result includes: determining a scaling factor of the first data based on the content information of the preset region to which the first data belongs, and preprocessing a variance of a Gaussian distribution based on the scaling factor to obtain a processed variance. The content information of the preset region includes a texture resolution level or texture complexity of the preset region.

[0044] As an example, the complexity of the texture can be calculated, and the resolution level is considered to be high for the preset area with complex texture, and the resolution level is considered to be low for the preset area with smooth texture. For the first data and the second data that belong to the preset area with high resolution level, the contraction factor of the first data and the contraction factor of the second data are different. For the first data and the second data that belong to the preset area with low resolution level, the contraction factor of the first data and the contraction factor of the second data are the same. As another example, for the first data and the second data that belong to the preset area with high texture complexity, the contraction factor of the first data and the contraction factor of the second data are different. For the first data and the second data that belong to the preset area with low texture complexity, the contraction factor of the first data and the contraction factor of the second data are the same.

[0045] The preset area may be an image block, a sub-band, a frequency band, or a channel mentioned below.

[0046] If the first data and the second data belong to the same image block in the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different image blocks, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the image block to which the first data belongs;

[0047] or,

[0048] If the first data and the second data belong to one of the plurality of sub-bands obtained by performing a wavelet transform on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different sub-bands, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the sub-band to which the first data belongs;

[0049] or,

[0050] If the first data and the second data belong to one of a plurality of frequency bands or one of a plurality of transform blocks obtained by performing DCT on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different frequency bands or transform blocks, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the frequency band or transform block to which the first data belongs;

[0051] or,

[0052] If the first data and the second data belong to the same channel of the three-dimensional feature map obtained by feature extraction of the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different channels, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the channel to which the first data belongs.

[0053] It should be noted that the texture complexity of the image block to which the first data belongs can be determined based on the content of the corresponding image block in the image to be encoded or the decoded image; the texture complexity of the subband to which the first data belongs can be determined based on the content of the corresponding part of the subband in the image to be encoded or the decoded image; the texture complexity of the frequency band to which the first data belongs can be determined based on the content of the corresponding part of the frequency band in the image to be encoded or the decoded image; and the texture complexity of the channel to which the first data belongs can be determined based on the content of the corresponding part of the channel in the image to be encoded or the decoded image. In one example, the greater the texture complexity of the first data, the greater the scaling factor of the first data.

[0054] In one possible design, the encoding method further includes:

[0055] The probability estimation result of the second preset area is preprocessed to obtain a processed probability estimation result.

[0056] In one possible design, the probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes:

[0057] The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean and the first variance of the Gaussian distribution, or the variance of the Gaussian distribution is processed according to the scaling factor of the second preset area to obtain a second variance, wherein the processed probability estimation result includes the mean and the second variance of the Gaussian distribution, and the scaling factor of the first preset area and the scaling factor of the second prefabricated area are the same or different.

[0058] In one possible design, the first context information includes part or all of the pixel values ​​in the first image.

[0059] By preprocessing the probability estimation results, reconstructed images with different properties can be obtained according to user needs, thereby improving the quality of the reconstructed images. For example, by setting the variance of the probability estimation results to 0 as the processed variance, a reconstructed image with optimal signal quality (optimal objective quality) can be obtained, which means increasing the image's peak signal-to-noise ratio (PSNR) or reducing the mean-square error (MSE). By setting the scaling factors of multiple data points to the same, an image with optimal subjective quality can be obtained, which means reducing the image's PSNR or increasing the image's MSE. By setting the scaling factors of data belonging to the same part of the image to the same value and setting the scaling factors of data belonging to different parts to different values, an image with properties between optimal subjective quality and optimal objective quality can be obtained.

[0060] In one possible design, the encoding method further includes:

[0061] The first image is transformed to obtain a first transformed image; wherein, if the transform is a wavelet transform, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are wavelet coefficients or quantized wavelet coefficients, or; if the transform is a DCT, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are DCT coefficients or quantized DCT coefficients; or, if the transform is a feature transform, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are feature coefficients or quantized feature coefficients.

[0062] In one possible design, performing probability estimation based on the first context information to obtain a first probability estimation result includes:

[0063] Inputting the first context information into a first probability estimation network for processing to obtain parameters of a first probability distribution model; the probability estimation result is the parameters of the first probability distribution model;

[0064] or,

[0065] The first context information is input into the second probability estimation network for processing to obtain a target probability distribution, and the probability estimation result includes parameters of the target probability distribution; wherein the first probability estimation network and the second probability estimation network are implemented by neural networks.

[0066] Based on the second aspect, the present application relates to a method for encoding a video image. The method is performed by an encoding device, and the method includes:

[0067] A plurality of coefficients are obtained according to an image to be encoded, the plurality of coefficients including a first coefficient; a first probability estimation result is obtained according to context information of the first coefficient; and the first coefficient and the first probability estimation result are written into a compressed code stream.

[0068] The first coefficient may be a pixel in the image to be encoded or a coefficient in a transformed image obtained by transforming the image to be encoded.

[0069] Probability estimation is performed at the encoding end to obtain a probability estimation result, and the probability estimation result is transmitted to the decoding end, so that the decoding end can obtain a high-quality image by sampling based on the probability estimation result.

[0070] In one possible design, the plurality of coefficients further includes a second coefficient, and the encoding method further includes:

[0071] Obtaining a second probability estimation result according to context information of the second coefficient; writing the first coefficient and the first probability estimation result into a compressed code stream, including: writing the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result into the compressed code stream.

[0072] The encoder calculates the probability estimation results of each coefficient in the image to be encoded one by one, and transmits the probability estimation results of each coefficient to the decoder, so that the decoder can accurately sample based on the probability estimation results of each coefficient, thereby obtaining a higher quality reconstructed image.

[0073] In one possible design, the multiple coefficients further include a second coefficient, the first coefficient and the second coefficient belong to the same preset region, the preset region is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining a first probability estimation result based on context information of the first coefficient includes:

[0074] Performing a probability estimation based on the context information of the first coefficient to obtain a third probability estimation result; performing a probability estimation based on the context information of the second coefficient to obtain a second probability estimation result; and determining a first probability estimation result from the third probability estimation result and the second probability estimation result.

[0075] Writing the first coefficient and the first probability estimation result into the compressed code stream includes: writing the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream.

[0076] The preset area is an image block in the image to be encoded, or a subband obtained by performing wavelet transform on the image to be encoded, or a frequency band obtained by performing DCT on the image to be encoded, or a transform block obtained by performing DCT on the image to be encoded, or a channel in a three-dimensional feature map obtained by performing feature extraction on the image to be encoded.

[0077] Wherein, performing DCT transformation on the image to be coded in units of one or more image blocks can obtain one or more transformation blocks.

[0078] For data in a preset area, the encoder uses a probability estimation result as the probability estimation result of all coefficients in the preset area, so that only one probability estimation result needs to be transmitted during transmission, thereby reducing the number of transmission code streams and the resources required to transmit the code streams.

[0079] In one possible design, the multiple coefficients further include a second coefficient, the first coefficient and the second coefficient belong to the same preset region, the preset region is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining the first probability distribution based on context information of the first coefficient includes:

[0080] Probability estimation is performed based on context information of a preset area to obtain a first probability estimation result; the context information of the preset area includes context information of the first coefficient; the first coefficient and the first probability estimation result are written into a compressed code stream, including: writing the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream.

[0081] For data in a preset area, the encoder uses a probability estimation result as the probability estimation result of all coefficients in the preset area, so that only one probability estimation result needs to be transmitted during transmission, thereby reducing the number of transmission code streams and the resources required to transmit the code streams.

[0082] In one possible design, the encoding method further includes:

[0083] The value of the first identifier of the preset area is set to the first value to indicate that the first probability estimation result is used when sampling to obtain the estimated coefficient in the preset area; the first probability estimation result is saved in the probability estimation result set, and the index of the first probability estimation result in the probability estimation result set is recorded; the first coefficient, the second coefficient and the first probability estimation result are written into the compressed code stream, including: writing the first coefficient, the second coefficient, the probability estimation result set, the index, the size information of the preset area and the first identifier into the compressed code stream.

[0084] For multiple probability estimation results for multiple preset regions, the encoder saves the probability estimation results for the multiple preset regions into a probability estimation result set and records the position (i.e., index) of the probability estimation result for each preset region in the probability estimation result set. This allows the decoder to accurately determine the probability estimation result for each preset region from the probability estimation result set obtained based on bitstream decoding based on the index, thereby ensuring decoding accuracy. By introducing size information to indicate the number of times the probability estimation results for the preset region need to be sampled when sampling to obtain the estimated coefficients in the preset region, all the estimated coefficients in the preset region are obtained.

[0085] In one possible design, the encoding method further includes:

[0086] The value of the first identifier of the preset area is set to the first value to indicate that the first probability estimation result is used to write the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream when sampling the estimated coefficient in the preset area, including: writing the first coefficient, the second coefficient, the first probability estimation result, the size information of the preset area and the first identifier into the compressed code stream.

[0087] By setting the value of the first identifier of the preset area to the first value, it is indicated that the decoding end uses the probability estimation result of the preset area when sampling to obtain the estimation coefficients in the preset area after decoding the probability estimation result of the preset area; by introducing size information to indicate the number of times that sampling needs to be performed based on the probability estimation result of the preset area when sampling to obtain the estimation coefficients in the preset area, thereby obtaining all the estimation coefficients in the preset area.

[0088] In one possible design, the first coefficient and the second coefficient belong to the same preset region, and the encoding method further includes:

[0089] The value of the first identifier of the preset area is set to the second value to indicate that the respective probability estimation results are used when sampling to obtain the estimated coefficients in the preset area; the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result are written into the compressed code stream, including: writing the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result and the first identifier of the preset area into the compressed code stream.

[0090] In one possible design, the encoding method further includes:

[0091] The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result.

[0092] In a possible design, the probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution. The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result, including:

[0093] The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

[0094] In a possible design, the probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution. The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result, including:

[0095] Preprocessing the variance of the Gaussian distribution according to a scaling factor of the first coefficient to obtain a processed variance, wherein the processed probability estimation result includes a mean of the Gaussian distribution and the processed variance;

[0096] The scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; or,

[0097] Preprocessing a probability estimation result of the first coefficient based on content information of a preset region to which the first coefficient belongs to obtain a processed probability estimation result includes: determining a scaling factor of the first coefficient based on the content information of the preset region to which the first coefficient belongs, and preprocessing a variance of a Gaussian distribution based on the scaling factor to obtain a processed variance. The content information of the preset region includes a texture resolution level or texture complexity of the preset region.

[0098] As an example, the complexity of the texture can be calculated, and the resolution level is considered to be high for the preset area with complex texture, and the resolution level is considered to be low for the preset area with smooth texture. For the first coefficient and the second coefficient belonging to the preset area with high resolution level, the shrinkage factor of the first coefficient and the shrinkage factor of the second coefficient are different. For the first coefficient and the second coefficient in the preset area with low resolution level, the shrinkage factor of the first coefficient and the shrinkage factor of the second coefficient are the same. As another example, for the first coefficient and the second coefficient in the preset area with high texture complexity, the shrinkage factor of the first coefficient and the shrinkage factor of the second coefficient are different. For the first coefficient and the second coefficient in the preset area with low texture complexity, the shrinkage factor of the first coefficient and the shrinkage factor of the second coefficient are the same.

[0099] The preset area mentioned above may be an image block, a sub-band, a frequency band, a transform block or a channel mentioned below.

[0100] If the first coefficient and the second coefficient belong to the same image block in the image to be encoded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different image blocks, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the subband to which the first coefficient belongs; or,

[0101] If the first coefficient and the second coefficient belong to one of a plurality of subbands obtained by performing a wavelet transform on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different subbands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to a texture complexity of the subband to which the first coefficient belongs;

[0102] or,

[0103] If the first coefficient and the second coefficient belong to one of a plurality of frequency bands obtained by performing DCT on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different frequency bands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; if the scaling factor of the first coefficient is determined according to the texture complexity of the frequency band to which the first coefficient belongs;

[0104] or,

[0105] If the first coefficient and the second coefficient belong to the same channel of the three-dimensional feature map obtained by feature extraction of the image to be encoded, the scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or if the first coefficient and the second coefficient belong to different channels, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; if the scaling factor of the first coefficient is determined according to the texture complexity of the channel to which the first coefficient belongs.

[0106] It should be noted that the texture complexity of the image block to which the first coefficient belongs can be determined based on the content of the image block in the image to be encoded; the texture complexity of the subband to which the first coefficient belongs can be determined based on the content of the corresponding portion of the subband in the image to be encoded; the texture complexity of the frequency band to which the first coefficient belongs can be determined based on the content of the corresponding portion of the frequency band in the image to be encoded; and the texture complexity of the channel to which the first coefficient belongs can be determined based on the content of the corresponding portion of the channel in the image to be encoded. The greater the texture complexity of the first coefficient, the greater the scaling factor of the first coefficient.

[0107] In one possible design, the encoding method further includes:

[0108] The probability estimation result of the preset area is preprocessed to obtain a processed probability estimation result.

[0109] In a possible design, the probability estimation result of the preset area includes the mean and variance of the Gaussian distribution. The probability estimation result of the preset area is preprocessed to obtain a processed probability estimation result, including:

[0110] The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the first variance, or the variance of the Gaussian distribution is processed according to a scaling factor of a preset area to obtain a second variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the second variance.

[0111] By preprocessing the probability estimation results, reconstructed images with different properties can be obtained according to user needs, thereby improving the quality of the reconstructed images. For example, setting the variance of the probability estimation results to 0 as the processed variance can produce a reconstructed image with the best signal quality (optimal objective quality), which means increasing the PSNR or MSE of the image. By setting the scaling factors of multiple coefficients to the same, an image with the best subjective quality can be obtained, which means reducing the PSNR or increasing the MSE of the image. By setting the scaling factors of coefficients belonging to the same part of the image to the same value and setting the scaling factors of coefficients belonging to different parts to different values, an image with properties between the best subjective quality and the best objective quality can be obtained.

[0112] In one possible design, if the multiple coefficients are multiple pixel values ​​in the image to be encoded, the first context information includes part or all of the pixel values ​​in the first image; or,

[0113] A plurality of coefficients are obtained according to the image to be encoded, including:

[0114] If a plurality of coefficients are obtained by performing wavelet transform on the image to be coded, the plurality of coefficients are a plurality of wavelet coefficients, and the first context information includes part or all of the plurality of wavelet coefficients; or, if a plurality of coefficients are obtained by performing wavelet transform and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized wavelet coefficients, and the first context information includes part or all of the plurality of quantized wavelet coefficients; or, if a plurality of coefficients are obtained by performing DCT on the image to be coded, the plurality of coefficients are a plurality of DCT coefficients, and the first context information includes part or all of the plurality of DCT coefficients; or, if a plurality of coefficients are obtained by performing DCT and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized DCT coefficients, and the first context information includes part or all of the plurality of quantized DCT coefficients; or, if a plurality of coefficients are obtained by performing feature extraction on the image to be coded, the plurality of coefficients are a plurality of feature coefficients, and the first context information includes part or all of the plurality of feature coefficients; or, if a plurality of coefficients are obtained by performing feature extraction and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized feature coefficients, and the first context information includes part or all of the plurality of quantized feature coefficients.

[0115] In one possible design, obtaining a first probability estimation result according to context information of the first coefficient includes:

[0116] Obtaining a second probability distribution model, inputting the first context information into a third probability estimation network for processing to obtain parameters of the second probability distribution model; obtaining a first probability estimation result based on the second probability distribution model and the parameters of the second probability distribution model;

[0117] or,

[0118] The first context information is input into a fourth probability estimation model for processing to obtain a probability estimation result; wherein the third probability estimation network and the fourth probability estimation network are implemented by neural networks.

[0119] Based on the third aspect, the present application relates to a method for decoding a video image. The method is performed by a decoding device and includes:

[0120] A first probability estimation result is obtained by decoding the compressed code stream; a first estimation coefficient is obtained by sampling according to the first probability estimation result; and a first reconstructed image is obtained according to the first estimation coefficient.

[0121] In one possible design, the decoding method further includes:

[0122] Obtaining a second probability estimation result from decoding the compressed code stream; sampling according to the second probability estimation result to obtain a second estimation coefficient; and obtaining a first reconstructed image according to the first estimation coefficient, including: obtaining the first reconstructed image according to the first estimation coefficient and the second estimation coefficient.

[0123] In one possible design, obtaining a first probability estimation result from decoding a compressed code stream includes:

[0124] Decoding a first identifier from the compressed code stream; if the value of the first identifier is a first value, decoding the compressed code stream to obtain a first probability estimation result, including:

[0125] A probability estimation result set and an index of a preset area are decoded from a compressed code stream; the preset area includes a first estimation coefficient, the preset area is an area in the first reconstructed image, and the probability estimation result of the preset area is determined from the probability estimation result set according to the index, and the first probability estimation result is the probability estimation result of the preset area; wherein the value of the first identifier is a first value used to indicate that the probability estimation result of the preset area is used when sampling to obtain all estimation coefficients within the preset area.

[0126] In one possible design, the decoding method further includes:

[0127] A first identifier is decoded from a compressed code stream; if the value of the first identifier is a first value, a first probability estimation result is obtained from the compressed code stream decoding, including: decoding a probability estimation result of a preset area and size information of the preset area from the compressed code stream; the preset area includes a first estimation coefficient, and the preset area is an area in the first reconstructed image; the probability estimation result of the preset area is a first probability estimation result; wherein the value of the first identifier is the first value used to indicate that the probability estimation result of the preset area is used when sampling to obtain all coefficients to be estimated in the preset area.

[0128] In one possible design, the first estimation coefficient and the second estimation coefficient belong to the same preset region, where the preset region is a region in the first reconstructed image. The decoding method further includes:

[0129] A first identifier is decoded from the compressed code stream; if the value of the first identifier is the second value, the value of the first identifier is the second value used to indicate that the respective probability estimation results are used when sampling all the to-be-estimated systems in the preset area.

[0130] In one possible design, the first probability estimation result includes a mean and a variance of a Gaussian distribution, and sampling is performed based on the first probability estimation result to obtain a first estimation coefficient, including:

[0131] Obtain a first random number; determine a first reference value based on the first random number, where the first reference value obeys a Gaussian distribution; and determine a first estimation coefficient based on the first reference value and a mean and a variance of a first probability estimation result.

[0132] In one possible design, the decoding method further includes:

[0133] Preprocessing the variance of the first probability estimation result to obtain a processed variance;

[0134] Determining a first estimation coefficient according to the first reference value and the mean and variance of the first probability estimation result includes:

[0135] A first estimation coefficient is determined according to the first reference value, the mean of the first probability estimation result, and the processed variance.

[0136] In a possible design, preprocessing the variance of the first probability estimation result to obtain a processed variance includes:

[0137] The variance of the first probability distribution is set to 0 as the processed variance.

[0138] In one possible design, the first estimated coefficient is a quantized wavelet coefficient, or a wavelet coefficient, or a quantized DCT coefficient, or a DCT coefficient, or a characteristic coefficient, or a quantized characteristic coefficient, and the variance of the first probability distribution is preprocessed to obtain a processed variance, including:

[0139] Preprocessing the variance of the first probability distribution according to a scaling factor of the first estimated coefficient to obtain a processed variance,

[0140] The scaling factor of the first estimated coefficient is the same as the scaling factor of the second estimated coefficient; or, the scaling factor of the first estimated coefficient is different from the scaling factor of the second estimated coefficient; or,

[0141] Preprocessing a probability estimation result of the first estimation coefficient based on content information of a preset region to which the first estimation coefficient belongs to obtain a processed probability estimation result, including: determining a scaling factor of the first estimation coefficient based on the content information of the preset region to which the first estimation coefficient belongs, and preprocessing a variance of a Gaussian distribution based on the scaling factor to obtain a processed variance. The content information of the preset region includes a texture resolution level or texture complexity of the preset region.

[0142] As an example, the complexity of the texture can be calculated, and the resolution level is considered to be high for the preset area with complex texture, and the resolution level is considered to be low for the preset area with smooth texture. For the first estimation coefficient and the second estimation coefficient that belong to the preset area with high resolution level, the shrinkage factor of the first estimation coefficient and the shrinkage factor of the second estimation coefficient are different. For the first estimation coefficient and the second estimation coefficient that belong to the preset area with low resolution level, the shrinkage factor of the first estimation coefficient and the shrinkage factor of the second estimation coefficient are the same. As another example, for the first estimation coefficient and the second estimation coefficient that belong to the preset area with high texture complexity, the shrinkage factor of the first estimation coefficient and the shrinkage factor of the second estimation coefficient are different. For the first estimation coefficient and the second estimation coefficient that belong to the preset area with low texture complexity, the shrinkage factor of the first estimation coefficient and the shrinkage factor of the second estimation coefficient are the same.

[0143] The preset area mentioned above may be an image block, a sub-band, a frequency band, a transform block or a channel mentioned below.

[0144] When the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients or wavelet coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same subband, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different subbands, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the image block to which the first estimation coefficient belongs;

[0145] or,

[0146] When the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients or DCT coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same frequency band or transform block, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different frequency bands or transform blocks, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the frequency band or transform block to which the first estimation coefficient belongs;

[0147] or,

[0148] When the first estimation coefficient and the second estimation coefficient are characteristic coefficients or quantized characteristic coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same channel, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different channels, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the channel to which the first estimation coefficient belongs.

[0149] In one possible design, the first estimation coefficient and the second estimation coefficient are pixel values, and the variance of the first probability estimation result is preprocessed to obtain a processed variance, including:

[0150] Preprocessing the variance of the first probability estimation result according to the scaling factor of the first coefficient to obtain a processed variance,

[0151] The scaling factor of the first estimation coefficient is the same as the scaling factor of the second estimation coefficient, or the scaling factor of the first estimation coefficient is different from the scaling factor of the second estimation coefficient; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the image block to which the first estimation coefficient belongs.

[0152] It should be noted here that the texture complexity of the image block to which the first estimated coefficient belongs can be determined based on the content of the image block in the first reconstructed image or the second reconstructed image; the texture complexity of the subband to which the first estimated coefficient belongs can be determined based on the content of the corresponding part of the subband in the first reconstructed image or the second reconstructed image; the texture complexity of the frequency band to which the first estimated coefficient belongs can be determined based on the content of the corresponding part of the frequency band in the first reconstructed image or the second reconstructed image; the texture complexity of the channel to which the first estimated coefficient belongs can be determined based on the content of the corresponding part of the channel in the first reconstructed image or the second reconstructed image. Among them, the greater the texture complexity of the first estimated coefficient, the greater the scaling factor of the first estimated coefficient.

[0153] In one possible design, obtaining a first reconstructed image according to the first estimated coefficient and the second estimated coefficient includes:

[0154] If the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients, the first estimation coefficient and the second estimation coefficient are dequantized and inverse wavelet transformed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are wavelet coefficients, the first estimation coefficient and the second estimation coefficient are inverse wavelet transformed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients, the first estimation coefficient and the second estimation coefficient are dequantized and inverse DCT performed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are DCT coefficients, inverse DCT is performed on the first estimation coefficient and the second estimation coefficient to obtain the first reconstructed image.

[0155] By preprocessing the probability estimation results, reconstructed images with different properties can be obtained according to user needs, thereby improving the quality of the reconstructed images. For example, by setting the variance of the probability estimation results to 0 as the processed variance, a reconstructed image with optimal signal quality (optimal objective quality) can be obtained, which means increasing the PSNR or MSE of the image. By setting the scaling factors of multiple data to the same, an image with optimal subjective quality can be obtained, which means reducing the PSNR or increasing the MSE of the image. By setting the scaling factors of data belonging to the same part of the image to the same value and setting the scaling factors of data belonging to different parts to different values, an image with properties between the optimal subjective quality and the optimal objective quality can be obtained.

[0156] In one possible design, the decoding method further includes:

[0157] A plurality of reconstruction coefficients are obtained by decoding the compressed code stream; and a second reconstructed image is obtained according to the plurality of reconstruction coefficients.

[0158] In one possible design, obtaining a second reconstructed image according to a plurality of coefficients includes:

[0159] If multiple reconstruction coefficients are quantized wavelet coefficients, the multiple reconstruction coefficients are dequantized and inverse wavelet transformed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are wavelet coefficients, the multiple reconstruction coefficients are inverse wavelet transformed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are quantized DCT coefficients, the multiple reconstruction coefficients are dequantized and inverse DCTed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are DCT coefficients, the multiple reconstruction coefficients are inverse DCTed to obtain a second reconstructed image.

[0160] Since the sampling process is random, the sampling step can be repeated in the present application to obtain multiple reconstructed images. The multiple reconstructed images can be reconstructed images with the best subjective quality or reconstructed images with the best objective quality. The reconstructed image can be used as a reference for intra-frame or inter-frame prediction within the codec loop; it can also be used outside the codec loop to optimize the image quality as a post-processing method. For example: after obtaining multiple reconstructed images through the sampling step and the inverse transformation step, the reconstructed image with the best subjective quality is placed in the image buffer (decoded picture buffer, DPB) or the reference frame set, and used as a reference image for intra-frame or inter-frame prediction within the codec loop; the reconstructed image with the best objective quality is used for post-processing to adjust the subjective quality of the reconstructed image after coding and decoding, so as to improve the quality of the compressed and reconstructed image / video.

[0161] It should be pointed out here that the beneficial effects of the decoding end can refer to the beneficial effects of the encoding end, and will not be described here.

[0162] Based on a fourth aspect, the present application relates to a video image encoding device. The beneficial effects can be found in the description of the first or second aspects and are not further elaborated here. The encoding device has the function of implementing the behaviors described in the method examples of the first or second aspects. The functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0163] Based on a fifth aspect, the present application relates to a decoding device based on video images. The beneficial effects can be found in the description of the third aspect and are not further elaborated here. The encoding device has the function of implementing the behavior of the method example of the third aspect. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0164] The method described in the first or second aspect of the present application can be performed by the apparatus described in the fourth aspect of the present application. Other features and implementations of the method described in the first or second aspect of the present application directly depend on the functionality and implementation of the apparatus described in the fourth aspect of the present application.

[0165] The method described in the third aspect of the present application can be performed by the apparatus described in the fifth aspect of the present application. Other features and implementations of the method described in the third aspect of the present application directly depend on the functionality and implementation of the apparatus described in the fifth aspect of the present application.

[0166] Based on a sixth aspect, the present application relates to an apparatus for encoding a video stream, comprising a processor and a memory, wherein the memory stores instructions, and the instructions enable the processor to execute the method according to the first aspect or the second aspect.

[0167] Based on a seventh aspect, the present application relates to an apparatus for decoding a video stream, comprising a processor and a memory, wherein the memory stores instructions, and the instructions enable the processor to execute the method according to the third aspect.

[0168] According to an eighth aspect, a computer-readable storage medium is provided, having stored thereon instructions that, when executed, cause one or more processors to encode video data. The instructions cause the one or more processors to perform the method of the first, second, or third aspect, or any possible embodiment of the first, second, or third aspect.

[0169] Based on the ninth aspect, the present application relates to a computer program product comprising program code, which, when running, executes the method in the first, second or third aspect or any possible embodiment of the first, second or third aspect.

[0170] One or more embodiments will be described in detail in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0171] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0172] Figure 1 is a block diagram of an example of a video decoding system for implementing an embodiment of the present application;

[0173] Figure 2 is a block diagram of another example of a video decoding system for implementing an embodiment of the present application;

[0174] Figure 3 is a schematic block diagram of a video decoding device for implementing an embodiment of the present application;

[0175] Figure 4 is a schematic block diagram of a video decoding device for implementing an embodiment of the present application;

[0176] Figure 5 A schematic diagram of the structure of a video encoding and decoding device provided in an embodiment of the present application;

[0177] Figure 6a This is a schematic diagram of the result after a wavelet transform;

[0178] Figure 6b A schematic diagram of first context information and second context information of first data;

[0179] Figure 6c A schematic diagram of first context information and second context information of a first preset area;

[0180] Figure 6d A schematic diagram of the structure of a probability estimation network provided in an embodiment of the present application;

[0181] Figure 6e A schematic diagram of the structure of a residual network provided in an embodiment of the present application;

[0182] Figure 7 A schematic diagram of the structure of a video codec provided in an embodiment of the present application;

[0183] Figure 8 A schematic diagram of an encoding process provided in an embodiment of the present application;

[0184] Figure 9 Another coding process diagram provided in an embodiment of the present application;

[0185] Figure 10 A schematic diagram of a decoding process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0186] The embodiments of the present application provide an AI-based video image compression technology, in particular, a neural network-based video compression technology, and specifically provide a decoding method based on probability distribution and sampling to improve the traditional hybrid video encoding and decoding system.

[0187] Video coding generally refers to the processing of a sequence of images to form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video coding (or commonly referred to as coding) includes two parts: video encoding and video decoding. Video coding is performed on the source side and generally includes processing (for example, compressing) the original video image to reduce the amount of data required to represent the video image (thereby making it more efficient to store and / or transmit). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video image. The "encoding" of the video image (or commonly referred to as image) involved in the embodiment should be understood as the "encoding" or "decoding" of the video image or video sequence. The encoding part and the decoding part are also collectively referred to as codec (encoding and decoding, CODEC).

[0188] In the case of lossless video coding, the original video image can be reconstructed, that is, the reconstructed video image has the same quality as the original video image (assuming there is no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed through quantization, etc. to reduce the amount of data required to represent the video image, but the decoder side cannot fully reconstruct the video image, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0189] Since the embodiments of the present application involve the application of neural networks, in order to facilitate understanding, some nouns or terms used in the embodiments of the present application are explained below, and these nouns or terms are also considered part of the content of the invention.

[0190] (1) Neural Network

[0191] A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be:

[0192]

[0193] Where s = 1, 2, ... n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal of the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0194] (2) Deep Neural Networks

[0195] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. Based on the location of the different layers, the neural network within a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. Each layer is fully connected, meaning that any neuron in layer i is connected to any neuron in layer i+1.

[0196] Although DNN looks complicated, the work of each layer is actually not complicated. In simple terms, it can be expressed as the following linear relationship: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and a() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The number of these parameters is also relatively large. The definitions of these parameters in DNN are as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0197] In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as

[0198] It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0199] (3) Convolutional Neural Networks

[0200] A convolutional neural network (CNN) is a deep neural network with a convolutional architecture. It consists of a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons that performs convolution processing on the input signal. In a convolutional layer of a CNN, a neuron can only connect to a subset of neurons in adjacent layers. A convolutional layer typically contains several feature planes, each of which can be composed of a rectangular arrangement of neurons. Neurons in the same feature plane share weights, which are referred to as convolution kernels. Shared weights can be understood as extracting image information in a position-independent manner. Convolution kernels can be initialized as matrices of random size, and during CNN training, they can learn to acquire reasonable weights. Furthermore, shared weights have the direct benefit of reducing the number of connections between layers of the CNN, thereby reducing the risk of overfitting.

[0201] (4) Recurrent neural networks (RNNs) are used to process sequence data. In traditional neural network models, the layers are fully connected from the input layer to the hidden layer and then to the output layer, while the nodes within each layer are disconnected. Although this ordinary neural network solves many difficult problems, it is still powerless to solve many problems. For example, if you want to predict the next word in a sentence, you generally need to use the previous word because the previous and next words in a sentence are not independent. RNN is called a recurrent neural network because the current output of a sequence is also related to the previous output. The specific manifestation is that the network will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer disconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment. In theory, RNN can process sequence data of any length. The training of RNN is the same as the training of traditional CNN or DNN. RNN aims to make machines have the ability to remember like humans. Therefore, the output of RNN needs to rely on the current input information and historical memory information.

[0202] (5) Loss function

[0203] During the training of a deep neural network, because we want the output of the deep neural network to be as close as possible to the desired predicted value, we can compare the current network's predicted value with the desired target value and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, before the first update, there is usually an initialization process, which is to pre-configure the parameters for each layer in the deep neural network.) For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the deep neural network can predict the desired target value or a value very close to the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the target value. For example, the loss function output value (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss as much as possible.

[0204] (6) Backpropagation algorithm

[0205] Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, the forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0206] In the following embodiment of the decoding system 10, the encoder 20 and the decoder 30 are based on Figures 1 to 3 Provide a description.

[0207] Figure 1 FIG1 is a schematic block diagram of an exemplary decoding system 10, such as a video decoding system 10 (or simply decoding system 10) that can utilize the techniques of the present application. Video encoder 20 (or simply encoder 20) and video decoder 30 (or simply decoder 30) in video decoding system 10 represent devices that can be used to perform various techniques according to various examples described in this application.

[0208] like Figure 1 As shown, the decoding system 10 includes a source device 12 for providing encoded image data 21 such as an encoded image to a destination device 14 for decoding the encoded image data 21.

[0209] The source device 12 includes an encoder 20 , and optionally, may include an image source 16 , a preprocessor (or preprocessing unit) 18 such as an image preprocessor, and a communication interface (or communication unit) 22 .

[0210] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0211] In order to distinguish the processing performed by the pre-processor (or pre-processing unit) 18 , the image (or image data) 17 may also be referred to as a raw image (or raw image data) 17 .

[0212] The preprocessor 18 is configured to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image (or preprocessed image data) 19. For example, the preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color grading, or denoising. It will be appreciated that the preprocessor 18 may be an optional component.

[0213] The video encoder (or encoder) 20 is used to receive the pre-processed image data 19 and provide the encoded image data 21 (hereinafter referred to as Figure 2 etc. for further description).

[0214] The communication interface 22 in the source device 12 can be used to receive the encoded image data 21 and send the encoded image data 21 (or any other processed version) to another device such as the destination device 14 or any other device through the communication channel 13 for storage or direct reconstruction.

[0215] The destination device 14 includes a decoder 30 and, in addition or alternatively, may include a communication interface (or communication unit) 28 , a post-processor (or post-processing unit) 32 , and a display device 34 .

[0216] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0217] The communication interface 22 and the communication interface 28 can be used to send or receive encoded image data (or encoded data) 21 through a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any combination thereof.

[0218] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0219] The communication interface 28 corresponds to the communication interface 22 , and can be used, for example, to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain the encoded image data 21 .

[0220] Both the communication interface 22 and the communication interface 28 can be configured as follows Figure 1 The unidirectional communication interface or the bidirectional communication interface indicated by the arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14 can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0221] The video decoder (or decoder) 30 is used to receive the encoded image data 21 and provide decoded image data (or decoded image data) 31 (hereinafter referred to as Figure 3 etc. for further description).

[0222] The post-processor 32 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data) such as the decoded image to obtain post-processed image data 33 such as the post-processed image. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color grading, cropping, or resampling, or any other processing for generating the decoded image data 31 for display on a display device 34 or the like.

[0223] The display device 34 is configured to receive the post-processed image data 33 and display the image to a user or viewer. The display device 34 may be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen.

[0224] The decoding system 10 further includes a training engine 25. The specific training process implemented by the training engine 25 will be described in detail later and will not be repeated here.

[0225] although Figure 1 The source device 12 and the destination device 14 are shown as independent devices, but the device embodiments may also include both the source device 12 and the destination device 14 or the functions of both the source device 12 and the destination device 14, that is, include both the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions. In these embodiments, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0226] According to the description, Figure 1 It will be apparent to the skilled person that the presence and (exact) division of the different units or functionalities in the source device 12 and / or destination device 14 shown may vary depending on the actual device and application.

[0227] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30), or both, may be configured to generate a video signal by: Figure 2 The processing circuitry shown may be implemented, for example, by one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video encoding processors, or any combination thereof. The encoder 20 may be implemented by processing circuitry 46 to include reference to Figure 2The various modules discussed in connection with encoder 20 and / or any other encoder systems or subsystems described herein. Decoder 30 may be implemented by processing circuitry 46 to include reference to Figure 3 The processing circuitry 46 may be used to perform the various operations discussed below. Figure 4 As shown, if part of the technology is implemented in software, the device can store the software instructions in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology of the present invention. One of the video encoder 20 and the video decoder 30 can be integrated into a single device as part of a combined codec (encoder / decoder, CODEC), such as Figure 2 shown.

[0228] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook computer or laptop, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (e.g., a content service server or content distribution server), a broadcast receiving device, a broadcast transmitting device, etc., and may not use or use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped with components for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.

[0229] In some cases, Figure 1 The video decoding system 10 shown is merely exemplary, and the techniques provided herein may be applicable to video encoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In other examples, data is retrieved from a local memory, sent over a network, and so on. A video encoding device may encode data and store the data in a memory, and / or a video decoding device may retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve and decode data from a memory.

[0230] Figure 2 According to an exemplary embodiment, Figure 2 The video encoder 20 and / or Figure 3FIG2 is an illustrative diagram of an example of a video decoding system 40 including a video decoder 30. The video decoding system 40 may include an imaging device 41, a video encoder 20, a video decoder 30 (and / or a video codec implemented by processing circuitry 46), an antenna 42, one or more processors 43, one or more memory stores 44, and / or a display device 45.

[0231] like Figure 2 As shown, imaging device 41, antenna 42, processing circuit 46, video encoder 20, video decoder 30, processor 43, memory storage 44, and / or display device 45 can communicate with each other. In different embodiments, video decoding system 40 can include only video encoder 20 or only video decoder 30.

[0232] In some instances, antenna 42 can be used to transmit or receive an encoded bitstream of video data. Additionally, in some instances, display device 45 can be used to present the video data. Processing circuitry 46 can include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, and the like. Video decoding system 40 can also include an optional processor 43, which can similarly include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, and the like. Furthermore, memory storage 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory). In a non-limiting example, memory storage 44 can be implemented as cache memory. In other instances, processing circuitry 46 can include memory (e.g., cache memory, etc.) for implementing an image buffer, etc.

[0233] In some examples, video encoder 20 implemented by logic circuitry may include an image buffer (e.g., implemented by processing circuitry 46 or memory storage 44) and a graphics processing unit (e.g., implemented by processing circuitry 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 20 implemented by processing circuitry 46 to implement reference Figure 2 and / or any other encoder system or subsystem described herein. Logic circuits may be used to perform the various operations discussed herein.

[0234] In some examples, video decoder 30 may be implemented in a similar manner by processing circuitry 46 to implement reference Figure 3 The various modules discussed herein may be used with reference to the video decoder 30 and / or any other decoder systems or subsystems described herein. In some examples, the video decoder 30 implemented by logic circuitry may include an image buffer (implemented by processing circuitry 46 or memory storage 44) and a graphics processing unit (e.g., implemented by processing circuitry 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video decoder 30 implemented by processing circuitry 46 to implement the video decoder 30 as described above. Figure 3 and / or the various modules discussed with respect to any other decoder system or subsystem described herein.

[0235] In some examples, antenna 42 may be used to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoded video frames, indicators, index values, mode selection data, etc., as discussed herein, such as data related to encoding partitions (e.g., transform coefficients or quantized transform coefficients, optional indicators (as discussed), and / or data defining encoding partitions). Video decoding system 40 may also include video decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0236] It should be understood that for the examples described herein with reference to video encoder 20, video decoder 30 can be configured to perform the reverse process. With respect to signaling syntax elements, video decoder 30 can be configured to receive and parse such syntax elements and decode the associated video data accordingly. In some examples, video encoder 20 can entropy encode the syntax elements into an encoded video bitstream. In such examples, video decoder 30 can parse such syntax elements and decode the associated video data accordingly.

[0237] For ease of description, embodiments of the present invention are described with reference to the Versatile Video Coding (VVC) reference software or the High-Efficiency Video Coding (HEVC) developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will appreciate that embodiments of the present invention are not limited to HEVC or VVC.

[0238] Figure 3 Schematic diagram of a video decoding device 300 provided in an embodiment of the present invention. The video decoding device 300 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 300 may be a decoder, such as Figure 1 The video decoder 30 in , may also be an encoder, such as Figure 1 The video encoder 20 in.

[0239] The video decoding device 300 includes: an input port 310 (or input port 310) and a receiver unit (Rx) 320 for receiving data; a processor, logic unit, or central processing unit (CPU) 330 for processing data; for example, the processor 330 may be a neural network processor 330; a transmitter unit (Tx) 340 and an output port 350 (or output port 350) for transmitting data; and a memory 360 for storing data. The video decoding device 400 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the input port 310, the receiver unit 320, the transmitter unit 340, and the output port 350 for outputting or receiving optical or electrical signals.

[0240] The processor 330 is implemented in hardware and software. The processor 330 can be implemented as one or more processor chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 330 communicates with the input port 310, the receiving unit 320, the transmitting unit 340, the output port 350, and the memory 360. The processor 330 includes a decoding module 370 (e.g., a decoding module 370 based on a neural network NN). The decoding module 370 implements the embodiments disclosed above. For example, the decoding module 370 performs, processes, prepares, or provides various encoding operations. Therefore, the decoding module 370 provides substantial improvements to the functionality of the video decoding device 300 and affects the switching of the video decoding device 300 to different states. Alternatively, the decoding module 370 is implemented by instructions stored in the memory 360 and executed by the processor 330.

[0241] Memory 360 includes one or more disks, tape drives, and solid-state drives and can be used as overflow data storage for storing programs when such programs are selected for execution, and for storing instructions and data read during program execution. Memory 360 can be volatile and / or non-volatile and can be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0242] Figure 4 A simplified block diagram of an apparatus 400 is provided for an exemplary embodiment. The apparatus 400 may be used as Figure 1 Either or both of the source device 12 and the destination device 14 in .

[0243] Processor 402 in apparatus 400 may be a central processing unit (CPU). Alternatively, processor 402 may be any other type of device or devices, now available or later developed, capable of manipulating or processing information. While the disclosed implementations may be implemented using a single processor, such as processor 402, as shown, using more than one processor provides greater speed and efficiency.

[0244] In one implementation, the memory 404 in the apparatus 400 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 404. The memory 404 may include code and data 406 that are accessed by the processor 402 via a bus 412. The memory 404 may also include an operating system 408 and application programs 410, which include at least one program that allows the processor 402 to perform the methods described herein. For example, the application programs 410 may include applications 1 through N, as well as a video decoding application that performs the methods described herein.

[0245] The apparatus 400 may also include one or more output devices, such as a display 418. In one example, the display 418 may be a touch-sensitive display that combines a display with touch-sensitive elements that can be used to sense touch input. The display 418 may be coupled to the processor 402 via the bus 412.

[0246] Although bus 412 in device 400 is described herein as a single bus, bus 412 may include multiple buses. Furthermore, secondary storage may be directly coupled to other components of device 400 or accessed via a network and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, device 400 may have a variety of configurations.

[0247] Codecs and encoding and decoding methods

[0248] Figure 5 This is a schematic block diagram of an example of a video codec for implementing the technology of this application. Figure 5 In the example, the video encoder 20 includes an encoding unit 501, a forward transform unit 502 and a probability estimation unit 503; the video decoder 30 includes a decoding unit 504, a sampling unit 505 and an inverse transform unit 506. Figure 5 The video codec shown may also be referred to as an end-to-end video codec or a video codec based on an end-to-end video codec.

[0249] Coding unit 501

[0250] The encoding unit 501 performs image encoding on the image to be encoded to obtain a compressed code stream.

[0251] Optionally, the above-mentioned image coding can be a joint photographic experts group (JPEG) coding method, a JPEG2000 coding method, an H.264 intra-frame coding method, an H.265 intra-frame coding method, an H.266 intra-frame coding method or other image coding methods.

[0252] Forward transform unit 502

[0253] The forward transformation unit 502 is configured to transform the first image to obtain a first transformed image.

[0254] The first image is an image to be encoded or a decoded image.

[0255] Optionally, the forward transformation unit 502 is further configured to transform the second image to obtain a second transformed image.

[0256] The second image is an image to be encoded or a decoded image, and the first image is different from the second image.

[0257] In one example, wavelet transform is performed N times on the first image, with 3N+1 sub-bands, each sub-band including one or more wavelet coefficients, where N is an integer greater than 0.

[0258] Among them, the wavelet transform method can be a traditional wavelet transform or a wavelet transform based on a deep network or other similar transform methods, which are not limited here. The wavelet transform method based on a deep network is different from the traditional wavelet transform in that the transform and prediction are implemented using a method based on a deep network. The specific implementation method of the deep network is not limited here. This application takes a wavelet transform as an example, that is, N=1, such as Figure 6a As shown, the first image is transformed into four sub-bands LL1, HL1, LH1 and HH1 after one wavelet transform.

[0259] For the first image, the image composed of sub-bands obtained by performing wavelet transform on the first image is the above-mentioned first transformed image. Similarly, for the second image, the image composed of sub-bands obtained by performing wavelet transform on the second image is the above-mentioned second transformed image.

[0260] Optionally, after a plurality of wavelet coefficients are obtained through wavelet transform, each wavelet coefficient is quantized to obtain a plurality of quantized wavelet coefficients. Specifically, when quantizing each wavelet coefficient, each subband may be processed according to a preset order 1, and then the wavelet coefficients in the current subband may be quantized according to a preset order 2 to obtain quantized wavelet coefficients. The preset order 1 may be an existing Z-shaped scanning order, for example, LL1→HL1→LH1→HH1. The preset order 2 may be an existing Z-shaped scanning order, a horizontal scanning order, or a vertical scanning order.

[0261] It should be understood that the above-mentioned preset order 1 and preset order 2 are only examples and are not limitations on the application. Of course, other orders are also possible.

[0262] Optionally, before quantizing each wavelet coefficient, the wavelet coefficient may be preprocessed to obtain processed wavelet coefficients, which are then quantized. For example, the obtained wavelet coefficients may be subjected to feature extraction through a neural network, and the feature extraction results may be quantized. Processing the wavelet coefficients before quantization allows the decoder to decode and obtain a high-quality first reconstructed image.

[0263] For the first image, the image composed of quantized wavelet coefficients obtained by quantizing the wavelet coefficients obtained by performing wavelet transform on the first image is the above-mentioned first transformed image. Similarly, for the second image, the image composed of quantized wavelet coefficients obtained by quantizing the wavelet coefficients obtained by performing wavelet transform on the second image is the above-mentioned second transformed image.

[0264] In another example, DCT is performed on the first image to obtain a DCT image, which includes multiple frequency bands, each frequency band including one or more DCT coefficients; wherein, after the first image is transformed, its low-frequency components are concentrated in the upper left corner, and the high-frequency components are distributed in the lower right corner, wherein the coefficient value in the first row and first column represents the direct current (DC) coefficient, that is, the average value of the first image, and the other coefficients are alternating current (AC) coefficients, and the DC coefficient and the AC coefficient are collectively referred to as DCT coefficients.

[0265] Optionally, the first image is divided into blocks to obtain a plurality of image blocks, and then DCT is performed on the image blocks as units to obtain transform blocks. For example, 1) the first image is divided into image blocks of a preset size, and the size of the image blocks of the preset size can be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128 and 256x256, etc. Or 2) the first image is divided into one or more image blocks, and the size of the image blocks is not limited. The first image can be divided using the quadtree, binary tree or ternary tree division method in the existing coding standards (H266, H265, H264, AVS2 or AVS3) to obtain one or more image blocks.

[0266] It should be pointed out that the frequency band can be understood as a coefficient block (a coefficient block obtained by performing DCT transform on an image block, because DCT transform is in blocks) or as coefficients at the same position in each coefficient block, forming a frequency band.

[0267] It should be understood that the image formed by the DCT coefficients obtained based on the first image is the above-mentioned first transformed image. The second image can also be processed in the above manner to obtain the DCT coefficients of the second image, and the DCT coefficients of the second image can constitute the above-mentioned second transformed image.

[0268] Optionally, the obtained DCT coefficients are quantized, such as uniformly quantized, to obtain quantized DCT coefficients. For the first image, an image constructed based on the quantized DCT coefficients obtained for the first image is the first transformed image. Similarly, for the second image, an image constructed based on the quantized DCT coefficients obtained for the second image is the second transformed image.

[0269] In another example, feature extraction is performed on the first image to obtain a three-dimensional feature map, which is the first transformed image. Optionally, feature coefficients in the three-dimensional feature map are quantized to obtain quantized feature coefficients, and the three-dimensional feature map composed of the quantized feature coefficients is the first transformed image.

[0270] It should be understood that the above processing can be performed on the second image, and the obtained three-dimensional feature map is the above-mentioned second transformed image; or the feature coefficients in the three-dimensional feature map are quantized to obtain a three-dimensional feature map composed of quantized feature coefficients, which is the above-mentioned second transformed image.

[0271] It should be noted that the forward transform unit 502 is optional. Figure 5 That is, when the forward transform unit 502 does not exist, the pixel domain image input to the probability estimation unit 503.

[0272] Probability estimation unit 503

[0273] The probability estimation unit 503 performs probability estimation based on the first context information of the first data to obtain a probability estimation result of the first data.

[0274] In one example, the first data is a pixel of a first image, and the first context information of the pixel includes all or part of the pixels in the first image. Furthermore, the first context information of the pixel includes pixels adjacent to the pixel in the first image, or includes part or all of the pixels in an image block adjacent to the pixel, or includes part or all of the pixels in the image block where the pixel is located.

[0275] It should be noted here that the above-mentioned “surrounding pixels” refer to pixels whose distance from the first data is less than a preset threshold, and the unit of the preset threshold is “pixel”.

[0276] In one example, the first data is a coefficient in a first transformed image. If the first data is a wavelet coefficient or a quantized wavelet coefficient, the first context information of the first data includes some or all of the coefficients in the first transformed image, where the coefficients are wavelet coefficients or quantized wavelet coefficients. Furthermore, the first context information of the first data includes the wavelet coefficients or quantized wavelet coefficients surrounding the first data in the first transformed image, or the first context information includes some or all of the coefficients in a subband adjacent to the first data, where the coefficients are wavelet coefficients or quantized wavelet coefficients; or the first context information includes some or all of the coefficients in the subband where the first data is located, where the coefficients are wavelet coefficients or quantized wavelet coefficients.

[0277] or,

[0278] If the first data is a DCT coefficient or a quantized DCT coefficient, the first context information of the first data includes some or all coefficients in the first transformed image, and the coefficients are DCT coefficients or quantized DCT coefficients. The first context information of the first data includes the DCT coefficients or quantized DCT coefficients surrounding the first data in the first transformed image, or the first context information includes some or all coefficients in a subband adjacent to the first data, and the coefficients are DCT coefficients or quantized DCT coefficients; or the first context information includes some or all coefficients in the subband where the first data is located, and the coefficients are DCT coefficients or quantized DCT coefficients;

[0279] Alternatively, if the first data is a feature coefficient or a quantized feature coefficient, and the first transformed image is a three-dimensional feature map obtained by performing feature extraction on the first image, then the first context information of the first data includes some or all coefficients in the first transformed image, and the coefficients are feature coefficients or quantized feature coefficients; further, the first context information of the first data includes feature coefficients or quantized feature coefficients surrounding the first data in the first transformed image, or the first context information includes some or all coefficients within a channel where the first data is located, and the coefficients are feature coefficients or quantized feature coefficients.

[0280] The above-mentioned "surrounding wavelet coefficients or quantized wavelet coefficients" refer to the wavelet coefficients or quantized wavelet coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "wavelet coefficient or quantized wavelet coefficient"; the above-mentioned "surrounding DCT coefficients or quantized DCT coefficients" refer to the DCT coefficients or quantized DCT coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "DCT coefficient or quantized DCT coefficient"; the above-mentioned "surrounding feature coefficients or quantized feature coefficients" refer to the feature coefficients or quantized feature coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "feature coefficient or quantized feature coefficient".

[0281] In an example, the probability estimation unit 503 performs probability estimation based on the first context information of the first data to obtain a probability estimation result of the first data, including:

[0282] The probability estimation unit 503 performs probability estimation based on first context information and second context information of the first data to obtain a probability estimation result of the first data; wherein the first context information and the second context information are obtained based on the first image and the second image respectively.

[0283] For example, Figure 6b As shown, assuming that the first data is a pixel at position P in the image to be encoded, the first context information of the first data includes pixels around the pixel at position P in the image to be encoded ( Figure 6bThe second context information includes pixels surrounding the pixel at position P in the decoded image, or includes some or all pixels in the image block adjacent to the pixel at position P, or includes some or all pixels within the image block where the pixel at position P is located; the second context information includes pixels surrounding the pixel at position P in the decoded image, or includes some or all pixels in the image block adjacent to the pixel at position P, or includes some or all pixels within the image block where the pixel at position P is located.

[0284] Assume that the first data is a wavelet coefficient or a quantized wavelet coefficient at a position P in a first transformed image, the first context information of the first data includes coefficients around the coefficient at position P in the first transformed image, which coefficients are wavelet coefficients or quantized wavelet coefficients, or the first context information includes part or all of the coefficients in a subband adjacent to the coefficient at position P, which coefficients are wavelet coefficients or quantized wavelet coefficients, or the first context information includes part or all of the coefficients within the subband where the coefficient at position P is located, which coefficients are wavelet coefficients or quantized wavelet coefficients; the second context information includes coefficients around the coefficient at position P in the second transformed image, which coefficients are wavelet coefficients or quantized wavelet coefficients, or the second context information includes part or all of the coefficients in a subband adjacent to the coefficient at position P, which coefficients are wavelet coefficients or quantized wavelet coefficients, or the second context information includes part or all of the coefficients within the subband where the coefficient at position P is located, which coefficients are wavelet coefficients or quantized wavelet coefficients.

[0285] Assume that the first data is the DCT coefficient or quantized DCT coefficient at position P in the first transformed image, the first context information of the first data includes the coefficients around the coefficient at position P in the first transformed image, which coefficients are DCT coefficients or quantized DCT coefficients, or the first context information includes part or all of the coefficients in the frequency band adjacent to the coefficient at position P, which coefficients are DCT coefficients or quantized DCT coefficients, or the first context information includes part or all of the coefficients within the frequency band where the coefficient at position P is located, which coefficients are DCT coefficients or quantized DCT coefficients; the second context information includes the coefficients around the coefficient at position P in the second transformed image, which coefficients are DCT coefficients or quantized DCT coefficients, or the second context information includes part or all of the coefficients in the frequency band adjacent to the coefficient at position P, which coefficients are DCT coefficients or quantized DCT coefficients, or the second context information includes part or all of the coefficients within the frequency band where the coefficient at position P is located, which coefficients are DCT coefficients or quantized DCT coefficients.

[0286] Assume that the first data is the characteristic coefficient or quantized characteristic coefficient at position P in the first transformed image, that is, the first transformed image and the second transformed image are three-dimensional feature maps obtained by feature extraction of the first image and the second image respectively; the first context information of the first data includes the characteristic coefficient or quantized characteristic coefficient around the coefficient at position P in the first transformed image, or the first context information includes part or all of the coefficients in the channel adjacent to the characteristic coefficient at position P, and the coefficient is a characteristic coefficient or quantized characteristic coefficient, or the first context information includes part or all of the coefficients in the channel where the coefficient at position P is located, and the coefficient is a characteristic coefficient or quantized characteristic coefficient; the second context information includes the characteristic coefficient or quantized characteristic coefficient around the coefficient at position P in the second transformed image, or the second context information includes part or all of the coefficients in the channel adjacent to the coefficient at position P, and the coefficient is a characteristic coefficient or quantized characteristic coefficient, or the second context information includes part or all of the coefficients in the channel where the coefficient at position P is located, and the coefficient is a characteristic coefficient or quantized characteristic coefficient.

[0287] In an example, the probability estimation unit 503 further performs probability estimation based on the first context information of the second data to obtain a probability estimation result of the second data.

[0288] It should be pointed out that the second data and the first data belong to data at different positions of the same image (such as the first image or the first transformed image obtained by transforming the first image). The specific process of performing probability estimation on the first context information of the second data to obtain the probability estimation result of the second data can be referred to the above-mentioned description of performing probability estimation on the first context information of the first data to obtain the probability estimation result of the first data, which will not be described here.

[0289] In a feasible embodiment, the first data and the second data belong to the same preset area. The preset area can be an image block in the first image, or a subband obtained by performing a wavelet transform on the first image, or a frequency band obtained by performing a DCT on the first image, or a channel of a three-dimensional feature map obtained by performing feature extraction on the first image. During probability estimation, only one probability estimation result can be obtained, and this probability estimation result can be referred to as a probability estimation result of the preset area. For data within a preset area, only one probability estimation result is obtained, and only one probability estimation result (i.e., the probability estimation result of the preset area) needs to be transmitted during transmission, which can save bitrate.

[0290] The following describes how to obtain the probability estimation result of the first preset area.

[0291] Method 1: For each data point in the first preset area, the probability estimation results for all data points in the first preset area are processed according to the above-described method for obtaining the probability estimation results for the first data point. For example, if there are five data points in the first preset area, five probability estimation results can be obtained. Then, a target probability estimation result is selected from the probability estimation results for all data points in the first preset area as the probability estimation result for the first preset area. For example, the probability estimation result for the data point located in the middle, upper left corner, upper right corner, lower left corner, or lower right corner of the first preset area is the probability estimation result for the first preset area.

[0292] Method 2: performing probability estimation based on the first context information of the first preset area to obtain a probability estimation result of the first preset area; or performing probability estimation based on the first context information and the second context information of the first preset area to obtain a probability estimation result of the first preset area.

[0293] In one example, if the first preset area is an image block of the first image, the first context information of the first preset area includes some or all pixels in the first image. Further, the first context information of the first preset area includes some or all pixels in the image block surrounding the first preset area in the first image.

[0294] If the preset region is a subband of a first transformed image (obtained by performing a wavelet transform on the first image), the first context information of the first preset region includes some or all coefficients in the first transformed image. Further, the first context information of the first preset region includes some or all coefficients in a subband surrounding the first preset region in the first image, where the coefficients are wavelet coefficients or quantized wavelet coefficients.

[0295] If the first preset region is a frequency band of a first transformed image (obtained by performing DCT on the first image), the first context information of the first preset region includes some or all coefficients in the first transformed image. Further, the first context information of the first preset region includes some or all coefficients in a frequency band surrounding the first preset region in the first image, where the coefficients are DCT coefficients or quantized DCT coefficients.

[0296] If the first preset area is a channel of a first transformed image (a three-dimensional feature map is obtained by performing feature extraction on the first image), the first context information of the first preset area includes part or all of the coefficients in the first transformed image, and the coefficients are feature coefficients or quantized feature coefficients. Further, the first context information of the first preset area includes part or all of the coefficients in the channel to which the first preset area belongs in the first image, and the coefficients are feature coefficients or quantized feature coefficients.

[0297] When the probability estimation unit 503 performs probability estimation based on the first context information and the second context information of the first preset area to obtain a probability estimation result of the first preset area, the first context information and the second context information are based on the first image and the second image respectively, wherein the second image is an image to be encoded or a decoded image, and the first image is different from the second image.

[0298] For example, Figure 6c As shown, assuming that the first preset area is area B in the first image, the first context information of the first preset area includes the surrounding area of ​​area B in the first image ( Figure 6c The second context information includes the surrounding area of ​​area B in the second image ( Figure 6c Region B in the first image is an image block in the first image.

[0299] Assume that the first preset area is subband B in the first transformed image, the first context information of the first preset area includes all or part of the coefficients in the subband surrounding subband B in the first transformed image, and the second context information includes all or part of the coefficients in the subband surrounding subband B in the second transformed image, where the coefficients are wavelet coefficients or quantized wavelet coefficients.

[0300] Assume that the first preset area is frequency band B in the first transformed image, the first context information of the first preset area includes all or part of the coefficients in the frequency band around subband B in the first transformed image, and the second context information includes all or part of the coefficients in the frequency band around subband B in the second transformed image, where the coefficients are DCT coefficients or quantized DCT coefficients.

[0301] Assume that the first preset area is channel B in the first transformed image, the first context information of the first preset area includes all or part of the coefficients in the subband around channel B in the first transformed image, and the second context information includes all or part of the coefficients in the channel around channel B in the second transformed image, and the coefficients are wavelet coefficients or quantized wavelet coefficients.

[0302] In one example, for the probability estimation result of the first data, the probability estimation unit 503 obtains a probability distribution model of the first data; processes the first context information and / or the second context information of the first data through a first probability estimation network to obtain parameters of the probability distribution model; obtains the probability distribution of the first data based on the probability distribution model of the first data and the parameters of the probability distribution model; the probability estimation result of the first data includes the probability distribution of the first data, or the parameters of the probability distribution model of the first data;

[0303] or,

[0304] The first context information and / or the second context information of the first data are processed by a second probability estimation network to obtain a probability distribution of the first data; the probability estimation result of the above-mentioned first data includes the probability distribution of the first data, or includes parameters of a probability distribution model corresponding to the probability distribution, wherein the first probability estimation network and the second probability estimation network are implemented based on neural networks.

[0305] According to the above method, the probability estimation result of the second data can be obtained.

[0306] In one example, the probability estimation result for the first preset area can be obtained as follows:

[0307] The probability estimation unit 503 obtains a probability distribution model of the first preset area; processes the first context information and / or the second context information of the first preset area through a third probability estimation network to obtain parameters of the probability distribution model; obtains the probability distribution of the first preset area based on the probability distribution model of the first preset area and the parameters of the probability distribution model; wherein the probability estimation result of the first preset area includes the probability distribution of the first preset area, or the parameters of the probability distribution model of the first preset area.

[0308] or,

[0309] The first context information and / or the second context information of the first preset area are processed by a fourth probability estimation network to obtain a probability distribution of the first preset area; the probability estimation result of the above-mentioned first preset area includes the probability distribution of the first preset area, or includes parameters of a probability distribution model corresponding to the probability distribution; wherein, the third probability estimation network and the fourth probability estimation network are implemented based on neural networks.

[0310] Optionally, the probability distribution model can be a Gaussian single model (GSM), an asymmetric Gaussian model, a Gaussian mixture model (GMM), or a Laplace distribution model. The probability estimation network can be implemented based on a deep learning network, such as a recurrent neural network (RNN) and a pixel convolutional neural network (PixelCNN), etc., without limitation herein.

[0311] As an example, when the probability distribution model is a Gaussian model (a single Gaussian model, an asymmetric Gaussian model, or a mixed Gaussian model), the parameters of the probability distribution model are the parameters of the Gaussian model, including the mean μ and the variance σ.

[0312] As an example, when the probability distribution model is a Laplace distribution model, the parameters of the probability distribution model are the parameters of the Laplace distribution model, including a location parameter μ and a scale parameter b;.

[0313] As an example, a typical PixelCNN-based probability estimation network (including the above-mentioned first probability estimation network, second probability estimation network, third probability estimation network and fourth probability estimation network) is as follows: Figure 6d As shown in the figure. "h×w" means that the current convolution layer uses a convolution kernel of size "h×w", and "ResB" means the residual module, the structure is as follows Figure 6e As shown in the figure, “* / relu” means using the relu activation function after the current layer.

[0314] In one example, after obtaining the rough estimate result of the first data, the probability estimation unit 503 preprocesses the rough estimate result of the first data to obtain a processed probability estimation result. Specifically, if the probability estimation result of the first data includes the mean and variance of a Gaussian distribution, the variance of the Gaussian distribution is processed to obtain a processed variance, and the mean and the processed variance of the Gaussian distribution are used as the processed probability estimation result of the first data; or,

[0315] The mean of the Gaussian distribution is processed to obtain the processed mean, and the variance of the Gaussian distribution and the processed mean are used as the processed probability estimation results of the first data.

[0316] In one example, the variance of the Gaussian distribution is processed to obtain a processed variance, including:

[0317] The variance of the Gaussian distribution is set to 0 as the variance after processing.

[0318] In one example, the variance of the Gaussian distribution is processed to obtain a processed variance, including:

[0319] Processing the variance of the Gaussian distribution according to the scaling factor of the first data to obtain a processed variance;

[0320] wherein the scaling factor of the first data is the same as the scaling factor of the second data; or

[0321] The scaling factor of the first data is different from the scaling factor of the second data; or,

[0322] If the first data and the second data belong to the same image block in the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different image blocks, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the image block to which the first data belongs;

[0323] or,

[0324] If the first data and the second data belong to one of the plurality of sub-bands obtained by performing a wavelet transform on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different sub-bands, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the sub-band to which the first data belongs;

[0325] or,

[0326] If the first data and the second data belong to one of a plurality of frequency bands obtained by performing DCT on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different frequency bands, the scaling factor of the first data and the scaling factor of the second data are different; or if the scaling factor of the first data is determined according to the texture complexity of the frequency band to which the first data belongs;

[0327] or,

[0328] If the first data and the second data belong to the same channel of the three-dimensional feature map obtained by feature extraction of the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different channels, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the channel to which the first data belongs.

[0329] In one example, when the probability estimation result of the first data includes the location parameter and scale parameter of the Laplace distribution, the scale parameter of the Laplace distribution is processed according to the scaling factor of the first data, and the processed probability estimation result of the first data includes the processed scale parameter and the location parameter of the Laplace distribution.

[0330] In one example, when the probability estimation result of the first data includes the location parameter and scale parameter of the Laplace distribution, the location parameter of the Laplace distribution is processed according to the scaling factor of the first data, and the processed probability estimation result of the first data includes the processed location parameter and the scale parameter of the Laplace distribution.

[0331] In one example, after obtaining the rough estimation result of the first preset area, the probability estimation unit 503 preprocesses the rough estimation result of the first preset area to obtain a processed probability estimation result. Specifically, if the probability estimation result of the first preset area includes the mean and variance of the Gaussian distribution, the variance of the Gaussian distribution is processed to obtain the processed variance, and the mean and the processed variance of the Gaussian distribution are used as the processed probability estimation result of the first preset area; or,

[0332] The mean of the Gaussian distribution is processed to obtain a processed mean, and the variance of the Gaussian distribution and the processed mean are used as the processed probability estimation result of the preset area. In one example, processing the variance of the Gaussian distribution to obtain the processed variance includes: setting the variance of the Gaussian distribution to 0 as the processed variance. In one example, processing the variance of the Gaussian distribution to obtain the processed variance includes:

[0333] Processing the variance of the Gaussian distribution according to the scaling factor of the first preset region to obtain a processed variance;

[0334] The scaling factor of the first preset area is the same as the scaling factors of the other preset areas; or

[0335] The scaling factor of the first preset area is different from the scaling factors of the other preset areas.

[0336] In one example, when the probability estimation result of the first preset area includes the position parameter and scale parameter of the Laplace distribution, the scale parameter of the Laplace distribution is processed according to the scaling factor of the first preset area, and the processed probability estimation result of the first preset area includes the processed scale parameter and the position parameter of the Laplace distribution.

[0337] In one example, when the probability estimation result of the first preset area includes the position parameter and scale parameter of the Laplace distribution, the position parameter of the Laplace distribution is processed according to the scaling factor of the first preset area, and the processed probability estimation result of the first preset area includes the processed position parameter and the scale parameter of the Laplace distribution.

[0338] In one example, after obtaining the probability estimation result of the first data and the probability estimation result of the second data, the encoding unit 501 directly writes the probability estimation result of the first data and the probability estimation result of the second data into the compressed code stream. In one example, in video compression, the probability estimation result of the first data and the probability estimation result of the second data can be saved in a sequence header, a picture header, a slice header, or supplemental enhancement information (SEI) and transmitted to the decoder 30.

[0339] In one example, after obtaining the probability estimation result of the first preset area, the first identifier enable_flag of the first preset area is set to a first value (such as 1 or true) to indicate that the same probability distribution is used when sampling to obtain the estimation coefficient in the first preset area at the decoding end, that is, the probability estimation result of the first preset area, and the probability estimation result of the first preset area is saved in a probability estimation result set, and the index of the probability estimation result of the first preset area in the probability estimation result set and the size information of the first preset area are recorded. The encoding unit 501 writes the probability estimation result set, the enable_flag, index and size information of the first preset area into the compressed code stream.

[0340] It should be noted that for multiple different preset areas, multiple probability estimation results can be obtained, and the multiple probability estimation results constitute a probability estimation result set. The position of the probability estimation result of the preset area in the probability estimation result set is the index of the preset area.

[0341] In one example, the probability estimation result set may be transmitted to the decoder 30 via an adaptation parameter set (APS).

[0342] In one example, after obtaining the probability estimation result of the first preset area, the enable_flag of the first preset area is set to a first value (such as 1 or true) to indicate that the same probability distribution, that is, the probability estimation result of the first preset area, is used when sampling to obtain the estimation coefficient in the first preset area at the decoding end; the encoding unit 501 writes the probability estimation result of the first preset area, enable_flag and the size information of the first preset area into the compressed code stream.

[0343] In one example, if all data within the first preset region uses their respective probability estimation results during sampling, the enable_flag of the first preset region is set to a second value (e.g., 0 or false), and the encoding unit 501 writes the respective probability estimation results of all data within the first preset region and the enable_flag of the first preset region into the compressed bitstream. Optionally, the encoding unit 501 also writes size information of the first preset region into the compressed bitstream.

[0344] In one example, the encoding unit does not write the size information of the preset area into the bitstream. Before encoding and decoding, the encoding end and the decoding end may negotiate the size of the preset area and store the size of the preset area in advance at the encoding end and the decoding end, respectively.

[0345] Decoding unit 504

[0346] The decoding unit 504 decodes the compressed code stream to obtain a first probability estimation result.

[0347] In an example, the decoding unit 504 further decodes the compressed code stream to obtain a second probability estimation result.

[0348] Optionally, the first probability estimation result includes parameters of a first probability distribution or a first probability distribution model, and the second probability estimation result includes parameters of a second probability distribution or a second probability distribution model.

[0349] In one example, the decoding unit 504 further decodes a first identifier from the compressed code stream. If the first identifier is a first value, it indicates that the same probability estimation result (i.e., the probability estimation result of the first preset region) is used when sampling to obtain all estimation coefficients in the first preset region, and the first preset region is a region in the enhanced image. The decoding unit 504 further decodes a probability estimation result set and an index of the first preset region from the compressed code stream. The probability estimation result set includes probability estimation results of multiple preset regions. The decoding unit 504 obtains the probability estimation result of the first preset region from the probability estimation result set according to the index of the first preset region.

[0350] If the first identifier is the second value, it indicates that the probability estimation results of the respective estimated coefficients are used when sampling to obtain all the estimated coefficients in the first preset area; the decoding unit 504 decodes the size information H1*W1 of the first preset area from the bit stream, indicating that the decoding unit 504 decodes H1*W1 probability estimation results from the compressed bit stream, and the sampling unit 505 can sample all the estimated coefficients in the first preset area through the H1*W1 probability estimation results, where H1 and W1 are both integers greater than 1.

[0351] In one example, the decoding unit 504 also decodes a first identifier from the compressed code stream, indicating that the same probability estimation result (i.e., the probability estimation result of the first preset area) is used when sampling to obtain all estimation coefficients in the first preset area. The first preset area is an area in the enhanced image. The decoding unit 504 also decodes the probability estimation result of the first preset area and H1*W1 from the code stream. The sampling unit 505 performs H1*W1 sampling on the probability estimation result of the first preset area to obtain H1*W1 estimation coefficients, i.e., the first preset area includes H1*W1 estimation coefficients.

[0352] Sampling unit 505

[0353] The sampling unit 505 samples according to the first probability estimation result to obtain the first estimation coefficient, and samples according to the first probability estimation result to obtain the second estimation coefficient. Since the two sampling processes are consistent, the following specifically describes the process of sampling according to the first probability estimation result to obtain the first estimation coefficient.

[0354] In one example, the first probability estimation result includes the mean and variance of the Gaussian distribution, and the sampling unit 505 performs sampling according to the first probability estimation result to obtain the first estimation coefficient, including:

[0355] Obtain a first random number; determine a first reference value based on the first random number, where the first reference value obeys a Gaussian distribution; and determine a first estimation coefficient based on the first reference value and a mean and a variance of a first probability estimation result.

[0356] Specifically, a linear congruential method is used to generate a uniformly distributed random number u on [0,1]; let Then z1 obeys the standard Gaussian distribution. Among them, erf() is the Gaussian error function, which is the cumulative distribution function of the standard normal distribution and is defined as follows:

[0357]

[0358] Let z2 = δ·z1 + μ, then z2 obeys a Gaussian distribution with a mean of μ and a variance of δ, and z2 is the first estimated coefficient mentioned above, where δ and μ are the mean and variance of the first probability estimation result mentioned above, respectively.

[0359] Optionally, before sampling, the variance of the first probability estimation result is processed. The specific processing process includes: setting the variance of the first probability estimation result to 0 as the processed variance; and then sampling according to the above-mentioned sampling method based on the processed variance and the mean of the first probability estimation result to obtain the first estimation coefficient.

[0360] Optionally, before sampling, the variance of the first probability estimation result is processed according to the scaling factor of the first estimation coefficient, and then sampling is performed according to the processed variance and the mean of the probability estimation result in accordance with the above sampling method to obtain the first estimation coefficient.

[0361] Optionally, before sampling, the mean of the first probability estimation result is processed according to the scaling factor of the first estimation coefficient, and then sampling is performed according to the processed mean and the variance of the first probability estimation result in accordance with the above sampling method to obtain the first estimation coefficient.

[0362] It should be understood that the second estimation coefficient can be obtained by sampling according to the second probability estimation result in the above manner.

[0363] Optionally, the scaling factor of the first estimated coefficient and the scaling factor of the second estimated coefficient are the same; or,

[0364] The scaling factor of the first estimated coefficient is different from the scaling factor of the second estimated coefficient; or,

[0365] When the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients or wavelet coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same subband, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different subbands, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the subband to which the first estimation coefficient belongs;

[0366] or,

[0367] When the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients or DCT coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same frequency band, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different frequency bands, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the frequency band to which the first estimation coefficient belongs;

[0368] or,

[0369] When the first estimation coefficient and the second estimation coefficient are characteristic coefficients or characteristic coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same channel, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different channels, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the channel to which the first estimation coefficient belongs.

[0370] In one example, the first estimation coefficient and the second estimation coefficient are pixel values, and preprocessing the variance of the first probability estimation result to obtain a processed variance includes:

[0371] Preprocessing the variance of the first probability distribution according to a scaling factor of the first coefficient to obtain a processed variance,

[0372] The scaling factor of the first estimation coefficient is the same as the scaling factor of the second estimation coefficient, or the scaling factor of the first estimation coefficient is different from the scaling factor of the second estimation coefficient; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the channel to which the first estimation coefficient belongs.

[0373] By preprocessing the first probability distribution, reconstructed images with different properties can be obtained according to user needs. For example, setting the variance of the first probability distribution to 0 as the processed variance can produce a reconstructed image with optimal signal quality (optimal objective quality), that is, increasing the PSNR of the image or reducing the MSE. By setting the scaling factors of multiple coefficients to the same, an image with optimal subjective quality can be obtained, that is, reducing the PSNR of the image or increasing the MSE of the image. By setting the scaling factors of coefficients belonging to the same part of the image to the same, and setting the scaling factors of coefficients belonging to different parts to different, an image with properties between the optimal subjective quality and the optimal objective quality can be obtained.

[0374] When the first probability estimation result includes a location parameter and a scale parameter of a Laplace distribution, performing probability estimation based on the first probability estimation result to obtain a first estimation coefficient includes:

[0375] Generate two uniformly distributed random numbers μ1 and μ2, let z3 = b·log(μ1), z4 = b·log(μ2), and the first estimated coefficient is z5 = z3-z4+μ, where μ and b are the location parameter and scale parameter of the Laplace distribution, respectively.

[0376] Optionally, before sampling, the scale parameter of the Laplace distribution is processed according to the scaling factor of the first coefficient, and then sampling is performed according to the processed scale parameter and the location parameter of the Laplace distribution in the above sampling manner to obtain the first estimated coefficient.

[0377] Optionally, before sampling, the location parameter of the Laplace distribution is processed according to the scaling factor of the first coefficient, and then sampling is performed according to the processed location parameter and the scale parameter of the Laplace distribution in accordance with the above sampling method to obtain the first estimated coefficient.

[0378] According to the above method, a plurality of estimated coefficients can be obtained, and the plurality of estimated coefficients include a first estimated coefficient and a second estimated coefficient.

[0379] Inverse transform unit 506

[0380] The inverse transform unit 506 obtains an enhanced image according to the multiple estimation coefficients.

[0381] Specifically, if the multiple estimated coefficients are multiple quantized wavelet coefficients, the inverse transform unit 506 performs inverse quantization and inverse wavelet transform on the multiple estimated coefficients to obtain an enhanced image, or,

[0382] If the multiple estimated coefficients are multiple wavelet coefficients, the inverse transform unit 506 performs inverse wavelet transform on the multiple estimated coefficients to obtain an enhanced image, or,

[0383] If the multiple estimated coefficients are multiple quantized DCT coefficients, the inverse transform unit 506 performs inverse quantization and inverse DCT on the multiple estimated coefficients to obtain a reconstructed image, or,

[0384] If the multiple estimated coefficients are multiple DCT coefficients, the inverse transform unit 506 performs inverse DCT on the multiple estimated coefficients to obtain an enhanced image.

[0385] If the multiple estimated coefficients are multiple pixel values, that is, multiple reconstructed pixel values, an enhanced image is obtained based on the multiple estimated coefficients.

[0386] In one example, after obtaining a feature map composed of multiple feature elements, the feature map can be passed through a neural network to output the enhanced image. The neural network can adopt any structure, such as a fully connected network, a convolutional neural network, a recurrent neural network, etc. The neural network can adopt a multi-layer deep neural network structure to obtain a higher quality first reconstructed image or second reconstructed image.

[0387] In one example, after obtaining a feature map composed of multiple feature elements, the feature map can be input into a machine vision task module to perform a corresponding machine task, such as completing machine vision tasks such as object classification, recognition, and segmentation.

[0388] It should be noted that the encoding solution of this embodiment is to obtain a compressed code stream after encoding the image to be encoded, and then refer to the encoding information (such as the compressed code stream or the coefficient information obtained by the transformation during the encoding process) to perform probability estimation. The decoding solution of this embodiment is performed on the premise that the compressed code stream is decoded to obtain the decoded image. It can also be said that the decoding solution of this embodiment is a post-processing process.

[0389] As can be seen, probability estimation is performed at the encoder to obtain a probability estimation result, which is then transmitted to the decoder. The decoder then performs sampling based on the probability estimation result to obtain estimated coefficients, and then samples the estimated coefficients to obtain an enhanced image. Because the sampling process is random and uncertain, the decoder can decode the same compressed bitstream multiple times using the above method to obtain multiple high-quality images with different properties. For example, the image with the best subjective quality or the image with the best objective quality.

[0390] In one example, during the entropy encoding process, the encoding unit 501 first performs a probability estimation on the first data to obtain a probability estimation result of the first data, which is referred to as probability estimation result A. The first data is then entropy encoded based on probability estimation result A. During the entropy decoding process, the decoding unit 504 first performs a probability estimation on the first data to obtain a probability estimation result of the first data, which is also referred to as probability estimation result A. Entropy decoding is then performed based on probability estimation result A. The probability estimation result referred to in the above embodiment is referred to as probability estimation result B.

[0391] Optionally, at the encoding end, entropy encoding is performed on the first data according to the probability estimation result A, and the decoding end performs probability estimation on the first data in the manner in which the encoding end performs probability estimation on the first data to obtain a probability estimation result (which can also be regarded as probability estimation result A), and entropy decoding is performed according to the probability estimation result A. Sampling can also be performed according to the probability estimation result A, and the sampling method is consistent with the above embodiment.

[0392] Optionally, the first data is entropy encoded at the encoding end according to the probability estimation result A, and the probability estimation result A is transmitted to the decoding end. The decoding end performs entropy decoding according to the probability estimation result A, and can also perform sampling according to the probability estimation result A. The sampling method is consistent with the above embodiment.

[0393] Optionally, the first data is entropy encoded at the encoding end according to the probability estimation result B, and the encoding end sends the probability estimation result B to the decoding end. The decoding end performs entropy decoding according to the probability estimation result B, and can also perform sampling according to the probability estimation result B. The sampling method is consistent with the above embodiment.

[0394] Optionally, at the encoding end, the first data is entropy encoded according to the probability estimation result B; at the decoding end, the first data is probability estimated to obtain the probability estimation result B, and then entropy decoding is performed according to the probability estimation result B. Sampling can also be performed according to the probability estimation result B, and the sampling method is consistent with the above embodiment.

[0395] Figure 7 FIG. 1 is a schematic block diagram of another example of a video codec for implementing the technology of the present application. Figure 7 In the example, the video encoder 20 includes a coefficient acquisition unit 701, a probability estimation unit 702 and an entropy encoding unit 703; the video decoder 30 includes an entropy decoding unit 704, a sampling unit 705, a first reconstruction unit 706 and a second reconstruction unit 707. Figure 5 The video codec shown may also be referred to as an end-to-end video codec or a video codec based on an end-to-end video codec.

[0396] Coefficient acquisition unit 701

[0397] The coefficient acquisition unit 701 obtains a plurality of coefficients from the image to be encoded, where the plurality of coefficients includes a first coefficient.

[0398] Optionally, the multiple coefficients may be multiple pixels.

[0399] In one example, the coefficient acquisition unit 701 divides the image to be encoded into image blocks of a preset size. The size of the image blocks of the preset size can be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, and 256x256, etc. Or 2) the coefficient acquisition unit 701 divides the image to be encoded into one or more image blocks, and the size of the image blocks is not limited. The image to be encoded can be divided using a quadtree, binary tree, or ternary tree partitioning method in existing coding standards (H266, H265, H264, AVS2, or AVS3) to obtain one or more image blocks. Each image block includes one or more pixels.

[0400] In one example, the image to be encoded is subjected to N wavelet transforms, with 3N+1 sub-bands, each sub-band including one or more wavelet coefficients, where N is an integer greater than 0.

[0401] Among them, the wavelet transform method can be a traditional wavelet transform or a wavelet transform based on a deep network or other similar transform methods, which are not limited here. The wavelet transform method based on a deep network is different from the traditional wavelet transform in that the transform and prediction are implemented using a method based on a deep network. The specific implementation method of the deep network is not limited here. This application takes a wavelet transform as an example, that is, N=1, such as Figure 6aAs shown, the image to be encoded is transformed into four sub-bands LL1, HL1, LH1 and HH1 after one wavelet transform.

[0402] For the image to be encoded, the image composed of subbands obtained by performing wavelet transform on the image to be encoded is the above-mentioned first transformed image. Similarly, for the decoded image, the image composed of subbands obtained by performing wavelet transform on the decoded image is the above-mentioned second transformed image.

[0403] Optionally, after a plurality of wavelet coefficients are obtained through wavelet transform, each wavelet coefficient is quantized to obtain a plurality of quantized wavelet coefficients. Specifically, when quantizing each wavelet coefficient, each subband may be processed according to a preset order 1, and then the wavelet coefficients in the current subband may be quantized according to a preset order 2 to obtain quantized wavelet coefficients. The preset order 1 may be an existing Z-shaped scanning order, for example, LL1→HL1→LH1→HH1. The preset order 2 may be an existing Z-shaped scanning order, a horizontal scanning order, or a vertical scanning order.

[0404] It should be understood that the above-mentioned preset order 1 and preset order 2 are only examples and are not limitations on the application. Of course, other orders are also possible.

[0405] Optionally, before quantizing each wavelet coefficient, the wavelet coefficient may be preprocessed to obtain processed wavelet coefficients, which are then quantized. For example, the obtained wavelet coefficients may be subjected to feature extraction through a neural network, and the feature extraction results may be quantized. Processing the wavelet coefficients before quantization allows the decoder to decode and obtain a high-quality first reconstructed image.

[0406] The aforementioned multiple coefficients may be multiple wavelet coefficients or quantized wavelet coefficients.

[0407] In another example, the coefficient acquisition unit 701 performs DCT on the image to be encoded to obtain a DCT image, which includes multiple frequency bands, and each frequency band includes one or more DCT coefficients; wherein, after the image to be encoded is transformed, its low-frequency components are concentrated in the upper left corner, and the high-frequency components are distributed in the lower right corner, wherein the coefficient value in the first row and first column represents the direct current (DC) coefficient, that is, the average value of the image to be encoded, and the other coefficients are alternating current (AC) coefficients, and the DC coefficient and the AC coefficient are collectively referred to as DCT coefficients.

[0408] Optionally, the coefficient acquisition unit 701 divides the image to be encoded into blocks to obtain multiple image blocks, and then performs DCT on the image blocks to obtain transform blocks. For example, 1) the image to be encoded is divided into image blocks of a preset size, and the size of the preset image blocks can be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128 and 256x256, etc. Or 2) the image to be encoded is divided into one or more image blocks, and the size of the image blocks is not limited. The image to be encoded can be divided using the quadtree, binary tree or ternary tree division method in the existing coding standards (H266, H265, H264, AVS2 or AVS3) to obtain one or more image blocks.

[0409] It should be understood that the image constructed based on the DCT coefficients of the image to be encoded is the above-mentioned first transformed image.

[0410] Optionally, the obtained DCT coefficients are quantized, such as uniformly quantized, to obtain quantized DCT coefficients. For the image to be encoded, the image constructed based on the quantized DCT coefficients obtained for the image to be encoded is the first transformed image.

[0411] The aforementioned multiple coefficients may be multiple DCT coefficients or quantized DCT coefficients.

[0412] In another example, feature extraction is performed on the image to be encoded to obtain a three-dimensional feature map, which is the first transformed image. Optionally, feature elements in the three-dimensional feature map are quantized to obtain quantized feature elements, and the three-dimensional feature map composed of the quantized feature elements is the first transformed image. The multiple coefficients can be multiple feature coefficients or multiple quantized feature coefficients.

[0413] Probability estimation unit 702

[0414] The probability estimation unit 702 obtains a first probability estimation result according to the context information of the first coefficient.

[0415] In one example, the first coefficient is a pixel of an image to be encoded, and the first context information of the pixel includes all or part of the pixels in the image to be encoded. Furthermore, the first context information of the pixel includes pixels adjacent to the pixel in the image to be encoded, or includes part or all of the pixels in an image block adjacent to the pixel, or includes part or all of the pixels in the image block where the pixel is located.

[0416] It should be noted here that the above-mentioned “surrounding pixels” refer to pixels whose distance from the first data is less than a preset threshold, and the unit of the preset threshold is “pixel”.

[0417] In one example, the first coefficient is a coefficient in the first transformed image, and if the first data is a wavelet coefficient or a quantized wavelet coefficient, the first context information of the first coefficient includes part or all of the coefficients in the first transformed image, and the coefficients are wavelet coefficients or quantized wavelet coefficients. Furthermore, the first context information of the first coefficient includes the wavelet coefficients or quantized wavelet coefficients surrounding the first coefficient in the first transformed image, or the first context information includes part or all of the coefficients in a subband adjacent to the first coefficient, and the coefficients are wavelet coefficients or quantized wavelet coefficients; or the first context information includes part or all of the coefficients in the subband where the first coefficient is located, and the coefficients are wavelet coefficients or quantized wavelet coefficients;

[0418] or,

[0419] If the first coefficient is a DCT coefficient or a quantized DCT coefficient, the first context information of the first coefficient includes part or all of the coefficients in the first transformed image, and the coefficients are DCT coefficients or quantized DCT coefficients. The first context information of the first coefficient includes DCT coefficients or quantized DCT coefficients surrounding the first data in the first transformed image, or the first context information includes part or all of the coefficients in a subband adjacent to the first coefficient, and the coefficients are DCT coefficients or quantized DCT coefficients; or the first context information includes part or all of the coefficients in the subband where the first coefficient is located, and the coefficients are DCT coefficients or quantized DCT coefficients;

[0420] Alternatively, if the first data is a characteristic coefficient or a quantized characteristic coefficient, the first context information of the first coefficient includes part or all of the coefficients in the first transformed image, and the coefficient is a characteristic coefficient or a quantized characteristic coefficient; further, the first context information of the first coefficient includes the characteristic coefficients or quantized characteristic coefficients surrounding the first coefficient in the first transformed image, or the first context information includes part or all of the coefficients in the channel where the first coefficient is located, and the coefficient is a characteristic coefficient or a quantized characteristic coefficient.

[0421] The above-mentioned "surrounding wavelet coefficients or quantized wavelet coefficients" refer to the wavelet coefficients or quantized wavelet coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "wavelet coefficient or quantized wavelet coefficient"; the above-mentioned "surrounding DCT coefficients or quantized DCT coefficients" refer to the DCT coefficients or quantized DCT coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "DCT coefficient or quantized DCT coefficient"; the above-mentioned "surrounding feature coefficients or quantized feature coefficients" refer to the feature coefficients or quantized feature coefficients whose distance from the first data is less than the preset threshold, and the unit of the preset threshold is "feature coefficient or quantized feature coefficient".

[0422] In an example, the multiple coefficients further include a second coefficient, and the probability estimation unit 702 is further configured to perform probability estimation according to context information of the second coefficient to obtain a second probability estimation result.

[0423] The second coefficient and the first coefficient are located at different positions in the same image (such as the image to be encoded or the first transformed image obtained by transforming the image to be encoded). The specific process of performing probability estimation based on the context information of the second coefficient to obtain the second probability estimation result can be found in the above-mentioned description of performing probability estimation based on the context information of the first coefficient to obtain the first probability estimation result, which will not be described here.

[0424] In one example, the first coefficient and the second coefficient belong to the same preset region. The preset region can be an image block in the image to be encoded, a subband obtained by performing a wavelet transform on the image to be encoded, a frequency band or image block obtained by performing a DCT on the image to be encoded, or a channel of a three-dimensional feature map obtained by performing feature extraction on the image to be encoded. During probability estimation, only one probability estimation result can be obtained, which can be referred to as a probability estimation result of the preset region. For data within a preset region, only one probability estimation result is obtained, and during transmission, only one probability estimation result (i.e., the probability estimation result of the preset region) needs to be transmitted, which can save bitrate.

[0425] The following describes how to obtain the probability estimation result of the first preset area.

[0426] Method 1: For each coefficient in the first preset area, process it according to the above method for obtaining the probability estimation result of the first coefficient to obtain the probability estimation results of all coefficients in the first preset area. For example, if there are five coefficients in the first preset area, five probability estimation results can be obtained. Then, a target probability estimation result is selected from the probability estimation results of all coefficients in the first preset area as the probability estimation result of the first preset area. For example, the probability estimation result of the coefficient located in the middle, upper left corner, upper right corner, lower left corner, or lower right corner of the first preset area is the probability estimation result of the first preset area.

[0427] Method 2: performing probability estimation according to the context information of the first preset area to obtain a probability estimation result of the first preset area.

[0428] Optionally, if the first preset area is an image block of the first image, the context information of the first preset area includes part or all of the pixels in the first image. Further, the context information of the first preset area includes part or all of the pixels in the image block surrounding the first preset area in the first image.

[0429] If the first preset region is a subband of a first transformed image (obtained by performing a wavelet transform on the first image), the context information of the first preset region includes some or all coefficients in the first transformed image. Further, the context information of the first preset region includes some or all coefficients in a subband surrounding the first preset region in the first image, where the coefficients are wavelet coefficients or quantized wavelet coefficients.

[0430] If the first preset region is a frequency band of a first transformed image (obtained by performing DCT on the first image), the context information of the first preset region includes some or all coefficients in the first transformed image. Further, the context information of the first preset region includes some or all coefficients in a frequency band surrounding the first preset region in the first image, where the coefficients are DCT coefficients or quantized DCT coefficients.

[0431] If the first preset region is a transformation block of the first transformation image (obtained by performing DCT on the first image), one or more transformation blocks may be obtained by performing DCT on the first image in units of one or more image blocks.

[0432] If the first preset area is a channel of a first transformed image (a three-dimensional feature map is obtained by performing feature extraction on the first image), the context information of the first preset area includes part or all of the coefficients in the first transformed image, and the coefficients are feature coefficients or quantized feature coefficients. Further, the context information of the first preset area includes part or all of the coefficients in the channel to which the first preset area belongs in the first image, and the coefficients are feature coefficients or quantized feature coefficients.

[0433] In one example, for the first probability estimation result, the probability estimation unit 702 obtains a probability distribution model of the first coefficient; processes the context information of the first coefficient through a fifth probability estimation network to obtain parameters of the probability distribution model; obtains a first probability distribution based on the probability distribution model of the first coefficient and the parameters of the probability distribution model; the first probability estimation result includes the first probability distribution or the parameters of the first probability distribution model;

[0434] or,

[0435] The context information of the first coefficient is processed by a sixth probability estimation network to obtain a first probability distribution; the above-mentioned first probability estimation result includes the first probability distribution, or includes parameters of a probability distribution model corresponding to the probability distribution, wherein the fifth probability estimation network and the sixth probability estimation network are implemented based on neural networks.

[0436] According to the above method, the probability estimation result of the second coefficient can be obtained.

[0437] In one example, the probability estimation result for the first preset area can be obtained as follows:

[0438] The probability estimation unit 503 obtains a probability distribution model of the first preset area; processes the context information of the first preset area through the seventh probability estimation network to obtain parameters of the probability distribution model; obtains the probability distribution of the first preset area based on the probability distribution model of the first preset area and the parameters of the probability distribution model; wherein the probability estimation result of the first preset area includes the probability distribution of the first preset area, or the parameters of the probability distribution model of the first preset area.

[0439] or,

[0440] The context information of the first preset area is processed by the eighth probability estimation network to obtain the probability distribution of the first preset area; the probability estimation result of the above-mentioned first preset area includes the probability distribution of the first preset area, or includes the parameters of the probability distribution model corresponding to the probability distribution; wherein, the seventh probability estimation network and the eighth probability estimation network are implemented based on neural networks.

[0441] Optionally, the probability distribution model may be: GSM, asymmetric Gaussian model, GMM or Laplace distribution model. The probability estimation network may be implemented based on a deep learning network, such as RNN and PixelCNN, etc., which is not limited here.

[0442] As an example, when the probability distribution model is a Gaussian model (a single Gaussian model, an asymmetric Gaussian model, or a mixed Gaussian model), the parameters of the probability distribution model are the parameters of the Gaussian model, including the mean μ and the variance σ.

[0443] As an example, when the probability distribution model is a Laplace distribution model, the parameters of the probability distribution model are the parameters of the Laplace distribution model, including a location parameter μ and a scale parameter b;.

[0444] As an example, a typical PixelCNN-based probability estimation network (including the fifth probability estimation network, the sixth probability estimation network, the seventh probability estimation network, and the eighth probability estimation network) is as follows: Figure 6d As shown in the figure. "h×w" means that the current convolution layer uses a convolution kernel of size "h×w", and "ResB" means the residual module, the structure is as follows Figure 6e As shown in the figure, “* / relu” means using the relu activation function after the current layer.

[0445] In one example, after obtaining the first rough estimation result, the probability estimation unit 702 preprocesses the first rough estimation result to obtain a processed probability estimation result. Specifically, if the first probability estimation result includes the mean and variance of a Gaussian distribution, the variance of the Gaussian distribution is processed to obtain a processed variance, and the mean and the processed variance of the Gaussian distribution are used as the processed probability estimation result; or,

[0446] The mean of the Gaussian distribution is processed to obtain the processed mean, and the variance of the Gaussian distribution and the processed mean are used as the processed probability estimation results.

[0447] In one example, the variance of the Gaussian distribution is processed to obtain a processed variance, including:

[0448] The variance of the Gaussian distribution is set to 0 as the variance after processing.

[0449] In one example, the variance of the Gaussian distribution is processed to obtain a processed variance, including:

[0450] Processing the variance of the Gaussian distribution according to a scaling factor of the first coefficient to obtain a processed variance;

[0451] wherein the scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or

[0452] The scaling factor of the first coefficient is different from the scaling factor of the second coefficient; or,

[0453] If the first coefficient and the second coefficient belong to the same image block in the image to be encoded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different image blocks, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the subband to which the first coefficient belongs; or,

[0454] If the first coefficient and the second coefficient belong to one of a plurality of subbands obtained by performing a wavelet transform on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different subbands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to a texture complexity of the subband to which the first coefficient belongs;

[0455] or,

[0456] If the first coefficient and the second coefficient belong to one frequency band or a transform block among a plurality of frequency bands obtained by performing DCT on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different frequency band transform blocks, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; if the scaling factor of the first coefficient is determined according to the texture complexity of the frequency band transform block to which the first coefficient belongs;

[0457] or,

[0458] If the first coefficient and the second coefficient belong to the same channel of the three-dimensional feature map obtained by feature extraction of the image to be encoded, the scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or if the first coefficient and the second coefficient belong to different channels, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; if the scaling factor of the first coefficient is determined according to the texture complexity of the channel to which the first coefficient belongs.

[0459] In one example, the probability estimation result is implemented based on a Gaussian distribution model, and the probability estimation result includes the Gaussian distribution or the mean and / or variance of the Gaussian distribution.

[0460] In one example, when the probability estimation result of the first coefficient includes the location parameter and scale parameter of the Laplace distribution, the scale parameter of the Laplace distribution is processed according to the scaling factor of the first coefficient, and the processed probability estimation result of the first coefficient includes the processed scale parameter and the location parameter of the Laplace distribution.

[0461] In one example, when the probability estimation result of the first coefficient includes the location parameter and scale parameter of the Laplace distribution, the location parameter of the Laplace distribution is processed according to the scaling factor of the first coefficient, and the processed probability estimation result of the first coefficient includes the processed location parameter and the scale parameter of the Laplace distribution.

[0462] In one example, after obtaining the rough estimation result of the first preset area, the probability estimation unit 702 preprocesses the rough estimation result of the first preset area to obtain a processed probability estimation result. Specifically, if the probability estimation result of the first preset area includes the mean and variance of a Gaussian distribution, the variance of the Gaussian distribution is processed to obtain a processed variance, and the mean and the processed variance of the Gaussian distribution are used as the processed probability estimation result of the first preset area; or,

[0463] The mean of the Gaussian distribution is processed to obtain a processed mean, and the variance of the Gaussian distribution and the processed mean are used as the processed probability estimation result of the preset area. In one example, processing the variance of the Gaussian distribution to obtain the processed variance includes: setting the variance of the Gaussian distribution to 0 as the processed variance. In one example, processing the variance of the Gaussian distribution to obtain the processed variance includes:

[0464] Processing the variance of the Gaussian distribution according to the scaling factor of the first preset region to obtain a processed variance;

[0465] The scaling factor of the first preset area is the same as the scaling factors of the other preset areas; or

[0466] The scaling factor of the first preset area is different from the scaling factors of the other preset areas.

[0467] In one example, when the probability estimation result of the first preset area includes the position parameter and scale parameter of the Laplace distribution, the scale parameter of the Laplace distribution is processed according to the scaling factor of the first preset area, and the processed probability estimation result of the first preset area includes the processed scale parameter and the position parameter of the Laplace distribution.

[0468] In one example, when the probability estimation result of the first preset area includes the position parameter and scale parameter of the Laplace distribution, the position parameter of the Laplace distribution is processed according to the scaling factor of the first preset area, and the processed probability estimation result of the first preset area includes the processed position parameter and the scale parameter of the Laplace distribution.

[0469] In one example, if the probability estimation result is obtained based on the Laplace distribution, the probability estimation result includes the Laplace distribution, or a scale parameter and / or location parameter of the Laplace distribution.

[0470] Entropy coding unit 704

[0471] The entropy coding unit 704 writes the first coefficient, the second coefficient, the first probability estimation result, and the second probability estimation result into the compressed code stream.

[0472] In one example, in video compression, the first probability estimation result and the second probability estimation result may be saved in a sequence header, a picture header, a slice, or SEI and transmitted to the decoder 30 .

[0473] In one example, after obtaining the probability estimation result of the first preset area, the first identifier enable_flag of the first preset area is set to a first value (such as 1 or true) to indicate that the same probability distribution is used when sampling the estimated coefficients in the first preset area at the decoding end, that is, the probability estimation result of the first preset area, and the probability estimation result of the first preset area is saved in a probability estimation result set, and the index of the probability estimation result of the first preset area in the probability estimation result set and the size information of the first preset area are recorded. The entropy coding unit 703 writes all coefficients in the first preset area, the probability estimation result set, the enable_flag, the index and the size information of the first preset area into the compressed code stream.

[0474] It should be noted that for multiple different preset areas, multiple probability estimation results can be obtained, and the multiple probability estimation results constitute a probability estimation result set. The position of the probability estimation result of the preset area in the probability estimation result set is the index of the preset area.

[0475] In one example, the set of probability estimates may be transmitted to decoder 30 via an APS.

[0476] In one example, after obtaining the probability estimation result of the first preset area, the enable_flag of the first preset area is set to a first value (such as 1 or true) to indicate that the same probability distribution, that is, the probability estimation result of the first preset area, is used when sampling the estimated coefficients in the first preset area at the decoding end; the entropy coding unit 703 writes all coefficients in the first preset area, the probability estimation result of the first preset area, the enable_flag and the size information of the first preset area into the compressed code stream.

[0477] In one example, if all coefficients in the first preset region use their respective probability estimation results during sampling, the enable_flag of the first preset region is set to a second value (e.g., 0 or false), and the entropy coding unit 703 writes all coefficients in the first preset region, the probability estimation results of all coefficients in the first preset region, and the enable_flag of the first preset region into the compressed bitstream. Optionally, the entropy coding unit 703 also writes size information of the first preset region into the compressed bitstream.

[0478] It should be noted that the entropy coding unit 703 writes the data into a compressed bitstream, specifically performing entropy coding on the data to obtain a compressed bitstream. Optionally, Huffman coding, CABAC coding, or entropy coding methods in H.264 / H265 / H.266 may be used.

[0479] Entropy decoding unit 704

[0480] The decoding unit 504 decodes the compressed code stream to obtain a first probability estimation result.

[0481] In an example, the entropy decoding unit 704 further decodes the compressed code stream to obtain a second probability estimation result.

[0482] Optionally, the first probability estimation result includes parameters of a first probability distribution or a first probability distribution model, and the second probability estimation result includes parameters of a second probability distribution or a second probability distribution model.

[0483] In one example, the entropy decoding unit 704 further decodes a first identifier from the compressed code stream. If the first identifier is a first value, it indicates that the same probability estimation result (i.e., the probability estimation result of the first preset region) is used when sampling all estimation coefficients in the first preset region, and the first preset region is a region in the enhanced image; the entropy decoding unit 704 further decodes a probability estimation result set and an index of the first preset region from the compressed code stream. The probability estimation result set includes probability estimation results of multiple preset regions. The entropy decoding unit 704 obtains the probability estimation result of the first preset region from the probability estimation result set according to the index of the first preset region;

[0484] If the first identifier is the second value, it indicates that the probability estimation results of the respective estimation coefficients are adopted when sampling all the estimation coefficients in the first preset area; the entropy decoding unit 704 decodes the size information H1*W1 of the first preset area from the bit stream, indicating that the entropy decoding unit 704 decodes H1*W1 probability estimation results from the compressed bit stream, and the sampling unit 705 can sample all the estimation coefficients in the first preset area through the H1*W1 probability estimation results, where H1 and W1 are both integers greater than 1.

[0485] In one example, the entropy decoding unit 704 also decodes a first identifier from the compressed code stream, indicating that the same probability estimation result (i.e., the probability estimation result of the first preset area) is used when sampling to obtain all estimation coefficients in the first preset area. The first preset area is an area in the enhanced image. The entropy decoding unit 704 also decodes the probability estimation result of the first preset area and H1*W1 from the code stream. The sampling unit 705 performs H1*W1 sampling on the probability estimation result of the first preset area to obtain H1*W1 estimation coefficients, i.e., the first preset area includes H1*W1 estimation coefficients.

[0486] The entropy decoding unit 704 is further configured to decode the compressed code stream to obtain a plurality of reconstruction coefficients.

[0487] It should be noted that the decoding method adopted by the entropy decoding unit 704 to decode the compressed code stream corresponds to the entropy encoding method adopted by the entropy encoding unit 703 .

[0488] Sampling unit 705

[0489] The specific process can be found in the related description of the sampling unit 505 above, which will not be described again here.

[0490] First reconstruction unit 706

[0491] The first reconstruction unit 706 obtains a first reconstructed image according to the multiple estimation coefficients.

[0492] Specifically, if the multiple estimation coefficients are multiple pixel values, a first reconstructed image can be obtained based on the multiple pixel values.

[0493] If the multiple estimated coefficients are multiple quantized wavelet coefficients, the first reconstruction unit 706 performs inverse quantization and inverse wavelet transform on the multiple estimated coefficients to obtain a first reconstructed image, or,

[0494] If the multiple estimated coefficients are multiple wavelet coefficients, the first reconstruction unit 706 performs inverse wavelet transform on the multiple estimated coefficients to obtain a first reconstructed image, or,

[0495] If the multiple estimated coefficients are multiple quantized DCT coefficients, the first reconstruction unit 706 performs inverse quantization and inverse DCT on the multiple estimated coefficients to obtain a reconstructed image, or,

[0496] If the multiple estimated coefficients are multiple DCT coefficients, the first reconstruction unit 706 performs inverse DCT on the multiple estimated coefficients to obtain a first reconstructed image.

[0497] If the multiple estimated coefficients are multiple characteristic coefficients, the first reconstruction unit 706 processes the characteristic map composed of the multiple characteristic coefficients to obtain a first reconstructed image; or,

[0498] If the multiple estimated coefficients are multiple quantized feature coefficients, the first reconstruction unit 706 dequantizes the multiple estimated coefficients to obtain multiple feature coefficients; and processes the feature map composed of the multiple feature coefficients to obtain a first reconstructed image.

[0499] In an example, multiple estimation coefficients may be input into a second reconstruction unit for processing to obtain a reconstructed image, and the reconstructed image may be used as a reference image for subsequent image prediction.

[0500] Second reconstruction unit 707

[0501] The second reconstruction unit 707 obtains a second reconstructed image according to the multiple reconstruction coefficients.

[0502] Specifically, if the multiple reconstruction coefficients are multiple pixel values, a second reconstructed image can be obtained based on the multiple pixel values.

[0503] If the multiple reconstruction coefficients are multiple quantized wavelet coefficients, the second reconstruction unit 707 performs inverse quantization and inverse wavelet transform on the multiple quantized coefficients to obtain a first reconstructed image, or,

[0504] If the multiple reconstruction coefficients are multiple wavelet coefficients, the second reconstruction unit 707 performs inverse wavelet transform on the multiple reconstruction coefficients to obtain a first reconstructed image, or,

[0505] If the multiple reconstruction coefficients are multiple quantized DCT coefficients, the second reconstruction unit 707 performs inverse quantization and inverse DCT on the multiple reconstruction coefficients to obtain a reconstructed image, or,

[0506] If the multiple reconstruction coefficients are multiple DCT coefficients, the second reconstruction unit 707 performs inverse DCT on the multiple reconstruction coefficients to obtain a first reconstructed image.

[0507] If the multiple reconstruction coefficients are multiple characteristic coefficients, the second reconstruction unit 707 processes the characteristic map composed of the multiple characteristic coefficients to obtain a first reconstructed image; or,

[0508] If the multiple reconstruction coefficients are multiple quantized feature coefficients, the second reconstruction unit 707 dequantizes the multiple reconstruction coefficients to obtain multiple feature coefficients; and processes a feature map composed of the multiple feature coefficients to obtain a first reconstructed image.

[0509] Optionally, the second reconstruction unit 707 may be implemented in the same manner as or differently from the first reconstruction unit 706, which is not limited herein.

[0510] In one example, after obtaining a feature map composed of multiple feature elements, the feature map can be passed through a neural network to output the first reconstructed image or the second reconstructed image. The neural network can adopt any structure, such as a fully connected network, a convolutional neural network, a recurrent neural network, etc. The neural network can adopt a multi-layer deep neural network structure to obtain a first reconstructed image or a second reconstructed image of better quality.

[0511] In one example, after obtaining a feature map composed of multiple feature elements, the feature map can be input into a machine vision task module to perform a corresponding machine task, such as completing machine vision tasks such as object classification, recognition, and segmentation.

[0512] In one example, the multiple estimation coefficients obtained by the sampling unit 705 can be input into the second reconstruction unit 707 simultaneously with the multiple reconstruction coefficients; specifically, when the multiple estimation coefficients and the multiple reconstruction coefficients are all feature coefficients, the second reconstruction unit 707 processes the multiple estimation coefficients in the manner of the first reconstruction unit 706 to obtain a first feature map, and the second reconstruction unit 707 obtains a second feature map based on the multiple reconstruction coefficients, and then obtains a second reconstructed image based on the first feature map and the second feature map through the above-mentioned neural network processing.

[0513] It can be seen that due to the randomness of the sampling process, the sampling step can be repeated in the present application to obtain multiple first reconstructed images. The multiple first reconstructed images can be reconstructed images with the best subjective quality, or they can be reconstructed images with the best objective quality. The first reconstructed image can be used as a reference for intra-frame or inter-frame prediction within the codec loop; it can also be used outside the codec loop to optimize the image quality as a post-processing method. For example: after obtaining multiple first reconstructed images through the sampling step and the reconstruction step, the reconstructed image with the best subjective quality is placed in the image buffer (decoded picture buffer, DPB) or the reference frame set, and used as a reference image for intra-frame or inter-frame prediction within the codec loop; the reconstructed image with the best objective quality is used for post-processing to adjust the subjective quality of the reconstructed image after coding and decoding, thereby improving the quality of the compressed reconstructed image / video. Optionally, a second reconstructed image is obtained based on the reconstruction coefficient, which can be used as a reference frame for predicting the next frame in video compression.

[0514] In one example, during the entropy encoding process, the entropy encoding unit 703 first performs a probability estimation on the first coefficient to obtain a probability estimation result of the first coefficient, which is referred to as a probability estimation result C. The first coefficient is then entropy encoded based on the probability estimation result C. During the entropy decoding process, the entropy decoding unit 704 first performs a probability estimation on the first coefficient to obtain a probability estimation result of the first coefficient, which is also referred to as a probability estimation result C. Entropy decoding is then performed based on the probability estimation result C. The probability estimation result referred to in the above embodiment is referred to as a probability estimation result D.

[0515] Optionally, at the encoding end, entropy encoding is performed on the first coefficient according to the probability estimation result C, and the decoding end performs probability estimation on the first coefficient in the manner in which the encoding end performs probability estimation on the first data, to obtain a probability estimation result (which can also be regarded as a probability estimation result C, and entropy decoding is performed according to the probability estimation result C. Sampling can also be performed according to the probability estimation result C, and the sampling method is consistent with the above embodiment.

[0516] Optionally, the first coefficient is entropy encoded at the encoding end according to the probability estimation result C, and the probability estimation result C is transmitted to the decoding end. The decoding end performs entropy decoding according to the probability estimation result C, and can also perform sampling according to the probability estimation result C. The sampling method is consistent with the above embodiment.

[0517] Optionally, the first coefficient is entropy encoded at the encoding end according to the probability estimation result D, and the encoding end sends the probability estimation result D to the decoding end. The decoding end performs entropy decoding according to the probability estimation result D, and can also perform sampling according to the probability estimation result D. The sampling method is consistent with the above embodiment.

[0518] Optionally, at the encoding end, entropy encoding is performed on the first coefficient according to the probability estimation result D; at the decoding end, probability estimation is performed on the first coefficient to obtain the probability estimation result D, and then entropy decoding is performed according to the probability estimation result D. Sampling can also be performed according to the probability estimation result D, and the sampling method is consistent with the above embodiment.

[0519] Figure 8 1 is a flow chart illustrating a process 800 of an encoding method according to an embodiment of the present application. The process 800 may be performed by the video encoder 20. The process 800 is described as a series of steps or operations. It should be understood that the process 1000 may be performed in various orders and / or occur simultaneously, not limited to the following. Figure 8 The execution order shown.

[0520] like Figure 8 As shown, the encoding method includes:

[0521] S801: Acquire a first image, where the first image is an image to be encoded or a decoded image.

[0522] S802: Perform probability estimation based on first context information to obtain a first probability estimation result; the first context information is obtained from the first image.

[0523] The first context information may be pixels in the first image or coefficients in a first transformed image obtained by transforming the first image.

[0524] In one possible design, the method of this embodiment further includes:

[0525] Acquiring a second image, where the second image is an image to be encoded or a decoded image, and the second image is different from the first image; and performing probability estimation based on the first context information to obtain a first probability estimation result, including:

[0526] The first probability estimation result is obtained by performing probability estimation based on the first context information and the second context information; the second context information is obtained from the second image.

[0527] In one possible design, performing probability estimation based on the first context information to obtain a first probability estimation result includes:

[0528] Performing probability estimation based on the context information of the first data to obtain a probability estimation result of the first data;

[0529] A probability estimation result of the second data is obtained by performing probability estimation based on the context information of the second data; wherein the first data and the second data are obtained based on the first image; and the first context information includes the context information of the first data and the context information of the second data.

[0530] In one possible design, the first probability estimation result includes a probability estimation result of a first preset area, the first preset area includes first data and second data, the first preset area is located in the first image, or in an image obtained by transforming the first image, and performing probability estimation based on the first context information to obtain the first probability estimation result includes:

[0531] A probability estimation is performed based on the context information of the first data to obtain a probability estimation result of the first data; a probability estimation is performed based on the context information of the second data to obtain a probability estimation result of the second data, wherein the first context information includes the context information of the first data and the context information of the second data; a probability estimation result of a first preset area is obtained based on the probability estimation result of the first data and the probability estimation result of the second data, wherein the first probability estimation result includes the probability estimation result of the first preset area.

[0532] In one possible design, the first probability estimation result includes a probability estimation result of a second preset area, the second preset area is located in the first image, or in an image obtained by transforming the first image, the first context information includes the context information of the second preset area, and the first probability estimation result is obtained by performing probability estimation based on the first context information, including: performing probability estimation based on the context information of the second preset area to obtain a probability estimation result of the second preset area, and the first probability estimation result includes the probability estimation result of the second preset area.

[0533] S803: Write the first probability estimation result into the compressed code stream.

[0534] In one possible design, the encoding method also includes: setting the value of the first identifier of the first preset area to a first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; saving the probability estimation result of the first preset area to a probability estimation result set, and recording the index of the probability estimation result of the first preset area in the probability estimation result set; writing the probability estimation result into a compressed code stream, including: writing the probability estimation result set, index, size information of the first preset area and the first identifier into the compressed code stream.

[0535] In one possible design, the encoding method further includes:

[0536] The value of the first identifier of the first preset area is set to the first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; the probability estimation result of the first preset area is preprocessed according to the scaling factor of the first preset area to obtain a processed probability estimation result, the processed probability estimation result is saved in a probability estimation result set, and the index of the processed probability estimation result in the probability estimation result set is recorded; the probability estimation result is written into the compressed code stream, including: writing the probability estimation result set, the index, the size information of the first preset area and the first identifier into the compressed code stream.

[0537] In one possible design, the encoding method further includes:

[0538] The value of the first identifier of the first preset area is set to a first value to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; the first probability estimation result is written into the compressed code stream, including: writing the probability estimation result of the first preset area, the size information of the first preset area and the first identifier into the code stream.

[0539] In one possible design, the encoding method further includes:

[0540] The probability estimation result of the first data is preprocessed to obtain a processed probability estimation result.

[0541] In one possible design, the probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes:

[0542] The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

[0543] In one possible design, the probability estimation result of the first data includes the mean and variance of the Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes: preprocessing the variance of the Gaussian distribution according to a scaling factor of the first data to obtain a processed variance, wherein the processed probability estimation result includes the mean and the processed variance of the Gaussian distribution; then

[0544] The scaling factor of the first data and the scaling factor of the second data are the same; or, the scaling factor of the first data and the scaling factor of the second data are different; or,

[0545] If the first data and the second data belong to the same image block in the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different image blocks, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the image block to which the first data belongs;

[0546] or,

[0547] If the first data and the second data belong to one of the plurality of sub-bands obtained by performing a wavelet transform on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different sub-bands, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the sub-band to which the first data belongs;

[0548] or,

[0549] If the first data and the second data belong to one of the plurality of frequency bands or the transform block obtained by performing DCT on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different frequency bands or the transform block, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the frequency band or the transform block to which the first data belongs;

[0550] or,

[0551] If the first data and the second data belong to the same channel of the three-dimensional feature map obtained by feature extraction of the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different channels, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the channel to which the first data belongs.

[0552] In one possible design, the encoding method further includes:

[0553] The probability estimation result of the second preset area is preprocessed to obtain a processed probability estimation result.

[0554] In one possible design, the probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and preprocessing the probability estimation result of the first data to obtain a processed probability estimation result includes:

[0555] The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean and the first variance of the Gaussian distribution, or the variance of the Gaussian distribution is processed according to the scaling factor of the second preset area to obtain a second variance, wherein the processed probability estimation result includes the mean and the second variance of the Gaussian distribution, and the scaling factor of the first preset area and the scaling factor of the second prefabricated area are the same or different.

[0556] In one possible design, the first context information includes part or all of the pixel values ​​in the first image.

[0557] In one possible design, the encoding method further includes:

[0558] The first image is transformed to obtain a first transformed image; wherein, if the transform is a wavelet transform, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are wavelet coefficients or quantized wavelet coefficients, or; if the transform is a DCT, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are DCT coefficients or quantized DCT coefficients; or, if the transform is a feature transform, the first context information includes part or all of the coefficients in the first transformed image, and the coefficients are feature coefficients or quantized feature coefficients.

[0559] In one possible design, performing probability estimation based on the first context information to obtain a first probability estimation result includes:

[0560] Inputting the first context information into a first probability estimation network for processing to obtain parameters of a first probability distribution model; the probability estimation result is the parameters of the first probability distribution model;

[0561] or,

[0562] The first context information is input into the second probability estimation network for processing to obtain a target probability distribution, and the probability estimation result includes parameters of the target probability distribution; wherein the first probability estimation network and the second probability estimation network are implemented by neural networks.

[0563] It should be noted here that Figure 8 The specific implementation process of the embodiment shown can be found in Figure 5The relevant descriptions of the encoding unit 501, the forward transformation unit 502 and the probability estimation unit 503 are not repeated here.

[0564] Figure 9 900 is a flow chart illustrating a process 900 of an encoding method according to an embodiment of the present application. The process 900 may be performed by the video encoder 20. The process 900 is described as a series of steps or operations. It should be understood that the process 900 may be performed in various orders and / or may occur simultaneously, not limited to the following. Figure 9 The execution order shown.

[0565] like Figure 9 As shown, the encoding method includes:

[0566] S901. Obtain multiple coefficients according to an image to be encoded, where the multiple coefficients include a first coefficient.

[0567] S902: Obtain a first probability estimation result according to context information of the first coefficient.

[0568] S903: Write the first coefficient and the first probability estimation result into the compressed code stream.

[0569] In one possible design, the plurality of coefficients further includes a second coefficient, and the encoding method further includes:

[0570] Obtaining a second probability estimation result according to context information of the second coefficient; writing the first coefficient and the first probability estimation result into a compressed code stream, including: writing the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result into the compressed code stream.

[0571] In one possible design, the multiple coefficients further include a second coefficient, the first coefficient and the second coefficient belong to the same preset region, the preset region is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining a first probability estimation result based on context information of the first coefficient includes:

[0572] Performing a probability estimation based on the context information of the first coefficient to obtain a third probability estimation result; performing a probability estimation based on the context information of the second coefficient to obtain a second probability estimation result; and determining a first probability estimation result from the third probability estimation result and the second probability estimation result.

[0573] Writing the first coefficient and the first probability estimation result into the compressed code stream includes: writing the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream.

[0574] In one possible design, the multiple coefficients further include a second coefficient, the first coefficient and the second coefficient belong to the same preset region, the preset region is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining the first probability distribution based on context information of the first coefficient includes:

[0575] Probability estimation is performed based on context information of a preset area to obtain a first probability estimation result; the context information of the preset area includes context information of the first coefficient; the first coefficient and the first probability estimation result are written into a compressed code stream, including: writing the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream.

[0576] In one possible design, the encoding method further includes:

[0577] The value of the first identifier of the preset area is set to the first value to indicate that the first probability estimation result is used when sampling to obtain the estimated coefficient in the preset area; the first probability estimation result is saved in the probability estimation result set, and the index of the first probability estimation result in the probability estimation result set is recorded; the first coefficient, the second coefficient and the first probability estimation result are written into the compressed code stream, including: writing the first coefficient, the second coefficient, the probability estimation result set, the index, the size information of the preset area and the first identifier into the compressed code stream.

[0578] In one possible design, the encoding method further includes:

[0579] The value of the first identifier of the preset area is set to the first value to indicate that the first probability estimation result is used to write the first coefficient, the second coefficient and the first probability estimation result into the compressed code stream when sampling the estimated coefficient in the preset area, including: writing the first coefficient, the second coefficient, the first probability estimation result, the size information of the preset area and the first identifier into the compressed code stream.

[0580] In one possible design, the first coefficient and the second coefficient belong to the same preset region, and the encoding method further includes:

[0581] The value of the first identifier of the preset area is set to the second value to indicate that the respective probability estimation results are used when sampling to obtain the estimated coefficients in the preset area; the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result are written into the compressed code stream, including: writing the first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result and the first identifier of the preset area into the compressed code stream.

[0582] In one possible design, the encoding method further includes:

[0583] The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result.

[0584] In a possible design, the probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution. The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result, including:

[0585] The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

[0586] In a possible design, the probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution. The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result, including:

[0587] Preprocessing the variance of the Gaussian distribution according to a scaling factor of the first coefficient to obtain a processed variance, wherein the processed probability estimation result includes a mean of the Gaussian distribution and the processed variance;

[0588] The scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; or,

[0589] If the first coefficient and the second coefficient belong to the same image block in the image to be encoded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different image blocks, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the subband to which the first coefficient belongs; or,

[0590] If the first coefficient and the second coefficient belong to one of a plurality of subbands obtained by performing a wavelet transform on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different subbands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to a texture complexity of the subband to which the first coefficient belongs;

[0591] or,

[0592] If the first coefficient and the second coefficient belong to one of a plurality of frequency bands obtained by performing DCT on the image to be coded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different frequency bands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; if the scaling factor of the first coefficient is determined according to the texture complexity of the frequency band to which the first coefficient belongs;

[0593] or,

[0594] If the first coefficient and the second coefficient belong to the same channel of the three-dimensional feature map obtained by feature extraction of the image to be encoded, the scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or if the first coefficient and the second coefficient belong to different channels, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; if the scaling factor of the first coefficient is determined according to the texture complexity of the channel to which the first coefficient belongs.

[0595] In one possible design, the encoding method further includes:

[0596] The probability estimation result of the preset area is preprocessed to obtain a processed probability estimation result.

[0597] In a possible design, the probability estimation result of the preset area includes the mean and variance of the Gaussian distribution. The probability estimation result of the preset area is preprocessed to obtain a processed probability estimation result, including:

[0598] The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the first variance, or the variance of the Gaussian distribution is processed according to a scaling factor of a preset area to obtain a second variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the second variance.

[0599] In one possible design, if the multiple coefficients are multiple pixel values ​​in the image to be encoded, the first context information includes part or all of the pixel values ​​in the image to be encoded; or,

[0600] A plurality of coefficients are obtained according to the image to be encoded, including:

[0601] If a plurality of coefficients are obtained by performing wavelet transform on the image to be coded, the plurality of coefficients are a plurality of wavelet coefficients, and the first context information includes part or all of the plurality of wavelet coefficients; or, if a plurality of coefficients are obtained by performing wavelet transform and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized wavelet coefficients, and the first context information includes part or all of the plurality of quantized wavelet coefficients; or, if a plurality of coefficients are obtained by performing DCT on the image to be coded, the plurality of coefficients are a plurality of DCT coefficients, and the first context information includes part or all of the plurality of DCT coefficients; or, if a plurality of coefficients are obtained by performing DCT and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized DCT coefficients, and the first context information includes part or all of the plurality of quantized DCT coefficients; or, if a plurality of coefficients are obtained by performing feature extraction on the image to be coded, the plurality of coefficients are a plurality of feature coefficients, and the first context information includes part or all of the plurality of feature coefficients; or, if a plurality of coefficients are obtained by performing feature extraction and quantization on the image to be coded, the plurality of coefficients are a plurality of quantized feature coefficients, and the first context information includes part or all of the plurality of quantized feature coefficients.

[0602] In one possible design, obtaining a first probability estimation result according to context information of the first coefficient includes:

[0603] Obtaining a second probability distribution model, inputting the first context information into a third probability estimation network for processing to obtain parameters of the second probability distribution model; obtaining a first probability estimation result based on the second probability distribution model and the parameters of the second probability distribution model;

[0604] or,

[0605] The first context information is input into a fourth probability estimation model for processing to obtain a probability estimation result; wherein the third probability estimation network and the fourth probability estimation network are implemented by neural networks.

[0606] It should be noted here that Figure 9 The specific implementation process of the embodiment shown can be found in Figure 7 The relevant descriptions of the coefficient acquisition unit 701, probability estimation unit 702 and entropy coding unit 703 are not repeated here.

[0607] Figure 10 1 is a flow chart illustrating a process 1000 of a decoding method according to an embodiment of the present application. The process 1000 may be performed by the video decoder 30. The process 1000 is described as a series of steps or operations. It should be understood that the process 1000 may be performed in various orders and / or may occur simultaneously, not limited to the following. Figure 10 The execution order shown.

[0608] like Figure 10 As shown, the decoding method includes:

[0609] S1001. Obtain a first probability estimation result from decoding a compressed code stream.

[0610] S1002. Sampling is performed according to the first probability estimation result to obtain a first estimation coefficient.

[0611] S1003: Obtain a first reconstructed image according to the first estimation coefficient.

[0612] In one possible design, the decoding method further includes:

[0613] Obtaining a second probability estimation result from decoding the compressed code stream; sampling according to the second probability estimation result to obtain a second estimation coefficient; and obtaining a first reconstructed image according to the first estimation coefficient, including: obtaining the first reconstructed image according to the first estimation coefficient and the second estimation coefficient.

[0614] In one possible design, obtaining a first probability estimation result from decoding a compressed code stream includes:

[0615] Decoding a first identifier from the compressed code stream; if the value of the first identifier is a first value, decoding the compressed code stream to obtain a first probability estimation result, including:

[0616] A probability estimation result set and an index of a preset area are decoded from a compressed code stream; the preset area includes a first estimation coefficient, the preset area is an area in the first reconstructed image, and the probability estimation result of the preset area is determined from the probability estimation result set according to the index, and the first probability estimation result is the probability estimation result of the preset area; wherein the value of the first identifier is a first value used to indicate that the probability estimation result of the preset area is used when sampling to obtain all estimation coefficients within the preset area.

[0617] In one possible design, the decoding method further includes:

[0618] A first identifier is decoded from a compressed code stream; if the value of the first identifier is a first value, a first probability estimation result is obtained from the compressed code stream decoding, including: decoding a probability estimation result of a preset area and size information of the preset area from the compressed code stream; the preset area includes a first estimation coefficient, and the preset area is an area in the first reconstructed image; the probability estimation result of the preset area is a first probability estimation result; wherein the value of the first identifier is the first value used to indicate that the probability estimation result of the preset area is used when sampling to obtain all coefficients to be estimated in the preset area.

[0619] In one possible design, the first estimation coefficient and the second estimation coefficient belong to the same preset region, where the preset region is a region in the first reconstructed image. The decoding method further includes:

[0620] A first identifier is decoded from the compressed code stream; if the value of the first identifier is the second value, the value of the first identifier is the second value used to indicate that the respective probability estimation results are used when sampling all the to-be-estimated systems in the preset area.

[0621] In one possible design, the first probability estimation result includes a mean and a variance of a Gaussian distribution, and sampling is performed based on the first probability estimation result to obtain a first estimation coefficient, including:

[0622] Obtain a first random number; determine a first reference value based on the first random number, where the first reference value obeys a Gaussian distribution; and determine a first estimation coefficient based on the first reference value and a mean and a variance of a first probability estimation result.

[0623] In one possible design, the decoding method further includes:

[0624] Preprocessing the variance of the first probability estimation result to obtain a processed variance;

[0625] Determining a first estimation coefficient according to the first reference value and the mean and variance of the first probability estimation result includes:

[0626] A first estimation coefficient is determined according to the first reference value, the mean of the first probability estimation result, and the processed variance.

[0627] In a possible design, preprocessing the variance of the first probability estimation result to obtain a processed variance includes:

[0628] The variance of the first probability distribution is set to 0 as the processed variance.

[0629] In one possible design, the first estimated coefficient is a quantized wavelet coefficient, or a wavelet coefficient, or a quantized DCT coefficient, or a DCT coefficient, or a characteristic coefficient, or a quantized characteristic coefficient, and the variance of the first probability distribution is preprocessed to obtain a processed variance, including:

[0630] Preprocessing the variance of the first probability distribution according to a scaling factor of the first estimated coefficient to obtain a processed variance,

[0631] The scaling factor of the first estimated coefficient is the same as the scaling factor of the second estimated coefficient; or the scaling factor of the first estimated coefficient is different from the scaling factor of the second estimated coefficient; or

[0632] When the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients or wavelet coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same subband, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different subbands, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the image block to which the first estimation coefficient belongs;

[0633] or,

[0634] When the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients or DCT coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same frequency band, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different frequency bands, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the frequency band to which the first estimation coefficient belongs;

[0635] or,

[0636] When the first estimation coefficient and the second estimation coefficient are characteristic coefficients or quantized characteristic coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same channel, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimation coefficient and the second estimation coefficient belong to different channels, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the channel to which the first estimation coefficient belongs.

[0637] In one possible design, the first estimation coefficient and the second estimation coefficient are pixel values, and the variance of the first probability estimation result is preprocessed to obtain a processed variance, including:

[0638] Preprocessing the variance of the first probability estimation result according to the scaling factor of the first coefficient to obtain a processed variance,

[0639] The scaling factor of the first estimation coefficient is the same as the scaling factor of the second estimation coefficient, or the scaling factor of the first estimation coefficient is different from the scaling factor of the second estimation coefficient; or the scaling factor of the first estimation coefficient is determined based on the texture complexity of the image block to which the first estimation coefficient belongs.

[0640] In one possible design, obtaining a first reconstructed image according to the first estimated coefficient and the second estimated coefficient includes:

[0641] If the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients, the first estimation coefficient and the second estimation coefficient are dequantized and inverse wavelet transformed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are wavelet coefficients, the first estimation coefficient and the second estimation coefficient are inverse wavelet transformed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients, the first estimation coefficient and the second estimation coefficient are dequantized and inverse DCT performed to obtain the first reconstructed image, or, if the first estimation coefficient and the second estimation coefficient are DCT coefficients, inverse DCT is performed on the first estimation coefficient and the second estimation coefficient to obtain the first reconstructed image.

[0642] In one possible design, the decoding method further includes:

[0643] A plurality of reconstruction coefficients are obtained by decoding the compressed code stream; and a second reconstructed image is obtained according to the plurality of reconstruction coefficients.

[0644] In one possible design, obtaining a second reconstructed image according to a plurality of coefficients includes:

[0645] If multiple reconstruction coefficients are quantized wavelet coefficients, the multiple reconstruction coefficients are dequantized and inverse wavelet transformed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are wavelet coefficients, the multiple reconstruction coefficients are inverse wavelet transformed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are quantized DCT coefficients, the multiple reconstruction coefficients are dequantized and inverse DCTed to obtain a second reconstructed image, or, if multiple reconstruction coefficients are DCT coefficients, the multiple reconstruction coefficients are inverse DCTed to obtain a second reconstructed image.

[0646] It should be noted here that Figure 10 The specific implementation process of the embodiment shown can be found in Figure 5 The decoding unit 504, the sampling unit 505 and the inverse transform unit 506 in the embodiment shown, and Figure 7 The descriptions related to the entropy decoding unit 704, the sampling unit 705, the first reconstruction unit 706 and the second reconstruction unit 707 in the illustrated embodiment are not repeated here.

[0647] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another (e.g., based on a communication protocol). In this manner, computer-readable media can generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product can include computer-readable media.

[0648] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are actually directed to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0649] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0650] The techniques of this application can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., a chipset). Various components, modules, or units are described herein to emphasize functional aspects of devices for performing the disclosed techniques, but they do not necessarily require implementation by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by interoperating hardware units (including one or more processors as described above).

[0651] The above description is merely an exemplary embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image processing method implemented by an encoding device, characterized in that: include: Acquire a first image, where the first image is an image to be encoded or a decoded image, performing probability estimation based on first context information to obtain a first probability estimation result, wherein the first context information is obtained from the first image; Writing the first probability estimation result into a compressed code stream; The first probability estimation result includes a probability estimation result of a first preset area, the first preset area includes first data and second data, the first preset area is located in the first image, or in an image obtained by transforming the first image, and the performing probability estimation based on the first context information to obtain the first probability estimation result includes: Performing probability estimation based on the context information of the first data to obtain a probability estimation result of the first data; performing probability estimation based on the context information of the second data to obtain a probability estimation result of the second data, wherein the first context information includes the context information of the first data and the context information of the second data; The probability estimation result of the first preset area is obtained according to the probability estimation result of the first data and the probability estimation result of the second data, where the first probability estimation result includes the probability estimation result of the first preset area.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a second image, where the second image is an image to be encoded or a decoded image, and the second image is different from the first image; The performing probability estimation according to the first context information to obtain the first probability estimation result includes: The first probability estimation result is obtained by performing probability estimation based on the first context information and the second context information, wherein the second context information is obtained from the second image.

3. The method according to claim 1 or 2, characterized in that in, The first data and the second data are obtained according to the first image.

4. The method according to claim 1 or 2, characterized in that The first probability estimation result includes a probability estimation result of a second preset area, where the second preset area is located in the first image or in an image obtained by transforming the first image, the first context information includes context information of the second preset area, and performing probability estimation based on the first context information to obtain the first probability estimation result includes: Probability estimation is performed according to the context information of the second preset area to obtain a probability estimation result of the second preset area, where the first probability estimation result includes the probability estimation result of the second preset area.

5. The method according to claim 1, wherein The method further comprises: Setting the value of the first identifier of the first preset area to a first value, so as to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; Saving the probability estimation result of the first preset area into a probability estimation result set, and recording the index of the probability estimation result of the first preset area in the probability estimation result set; Writing the first probability estimation result into the compressed code stream includes: The probability estimation result set, the index, the size information of the first preset area and the first identifier are written into the compressed code stream.

6. The method according to claim 1, characterized in that The method further comprises: Setting the value of the first identifier of the first preset area to a first value, so as to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; preprocessing the probability estimation result of the first preset area according to the scaling factor of the first preset area to obtain a processed probability estimation result, saving the processed probability estimation result to a probability estimation result set, and recording the index of the processed probability estimation result in the probability estimation result set; Writing the first probability estimation result into the compressed code stream includes: The probability estimation result set, the index, the size information of the first preset area and the first identifier are written into the compressed code stream.

7. The method according to claim 1, characterized in that The method further comprises: Setting the value of the first identifier of the first preset area to a first value, so as to indicate that the probability estimation result of the first preset area is used when sampling to obtain the estimation coefficient in the first preset area; Writing the first probability estimation result into the compressed code stream includes: The probability estimation result of the first preset area, the size information of the first preset area, and the first identifier are written into the code stream.

8. The method according to claim 1, characterized in that The method further comprises: The probability estimation result of the first data is preprocessed to obtain a processed probability estimation result.

9. The method according to claim 8, characterized in that The probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and the preprocessing of the probability estimation result of the first data to obtain a processed probability estimation result includes: The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

10. The method according to claim 8, characterized in that The probability estimation result of the first data includes a mean and a variance of a Gaussian distribution. The preprocessing of the probability estimation result of the first data to obtain a processed probability estimation result includes: preprocessing the variance of the Gaussian distribution according to a scaling factor of the first data to obtain a processed variance, wherein the processed probability estimation result includes the mean and the processed variance of the Gaussian distribution. The method further includes: preprocessing the variance of the second probability distribution according to the scaling factor of the second data, wherein: The scaling factor of the first data is the same as the scaling factor of the second data; or, The scaling factor of the first data is different from the scaling factor of the second data; or, If the first data and the second data belong to the same image block in the first image, the scaling factor of the first data is the same as the scaling factor of the second data; or if the first data and the second data belong to different image blocks, the scaling factor of the first data is different from the scaling factor of the second data; or the scaling factor of the first data is determined according to the texture complexity of the image block to which the first data belongs; or If the first data and the second data belong to one of the subbands obtained by performing wavelet transform on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different subbands, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the subband to which the first data belongs; or, If the first data and the second data belong to one of the frequency bands obtained by performing DCT on the first image, the scaling factor of the first data and the scaling factor of the second data are the same; or if the first data and the second data belong to different frequency bands, the scaling factor of the first data and the scaling factor of the second data are different; or the scaling factor of the first data is determined according to the texture complexity of the frequency band to which the first data belongs; or, If the first data and the second data belong to the same channel of the three-dimensional feature map obtained by feature extraction of the first image, the scaling factor of the first data is the same as the scaling factor of the second data; or if the first data and the second data belong to different channels, the scaling factor of the first data is different from the scaling factor of the second data; or the scaling factor of the first data is determined according to the texture complexity of the channel to which the first data belongs.

11. The method according to claim 4, characterized in that The method further comprises: The probability estimation result of the second preset area is preprocessed to obtain a processed probability estimation result.

12. The method according to claim 11, characterized in that The probability estimation result of the first data includes a mean and a variance of a Gaussian distribution, and the preprocessing of the probability estimation result of the first data to obtain a processed probability estimation result includes: The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the first variance, or, The variance of the Gaussian distribution is processed according to the scaling factor of the second preset area to obtain a second variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the second variance, and the scaling factor of the first preset area and the scaling factor of the second preset area are the same or different.

13. The method according to claim 12, characterized in that The first context information includes part or all of the pixel values ​​in the first image.

14. The method according to claim 12, characterized in that The method further comprises: transforming the first image to obtain a first transformed image; Wherein, if the transformation is a wavelet transformation, the first context information includes part or all of the coefficients in the first transformed image, the coefficients being wavelet coefficients or quantized wavelet coefficients, or; If the transform is discrete cosine transform (DCT), the first context information includes part or all of the coefficients in the first transformed image, where the coefficients are DCT coefficients or quantized DCT coefficients; or, If the transformation is a feature transformation, the first context information includes part or all of the coefficients in the first transformed image, where the coefficients are feature coefficients or quantized feature coefficients.

15. The method according to claim 14, wherein performing probability estimation based on the first context information to obtain a first probability estimation result comprises: Inputting the first context information into a first probability estimation network for processing to obtain parameters of a first probability distribution model; the first probability estimation result includes the parameters of the first probability distribution model; or, Inputting the first context information into a second probability estimation network for processing to obtain a target probability distribution, wherein the first probability estimation result includes parameters of the target probability distribution; The first probability estimation network and the second probability estimation network are implemented by neural networks.

16. A coding method implemented by a coding device, characterized in that: include: Obtaining a plurality of coefficients according to the image to be encoded, the plurality of coefficients including a first coefficient; Obtaining a first probability estimation result according to context information of the first coefficient; Writing the first coefficient and the first probability estimation result into a compressed code stream; The multiple coefficients also include a second coefficient, the first coefficient and the second coefficient belong to the same preset area, the preset area is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining a first probability estimation result based on the context information of the first coefficient includes: Performing a probability estimation based on the context information of the first coefficient to obtain a third probability estimation result; performing a probability estimation based on the context information of the second coefficient to obtain a second probability estimation result; and determining the first probability estimation result from the third probability estimation result and the second probability estimation result; Writing the first coefficient and the first probability estimation result into a compressed code stream includes: The first coefficient, the second coefficient and the first probability estimation result are written into the compressed code stream.

17. The method according to claim 16, characterized in that The plurality of coefficients further includes a second coefficient, the method further comprising: Obtaining a second probability estimation result according to the context information of the second coefficient; Writing the first coefficient and the first probability estimation result into a compressed code stream includes: The first coefficient, the first probability estimation result, the second coefficient and the second probability estimation result are written into the compressed code stream.

18. The method according to claim 16, characterized in that The multiple coefficients further include a second coefficient, the first coefficient and the second coefficient belong to the same preset area, the preset area is located in the image to be encoded, or in an image obtained by transforming the image to be encoded, and obtaining the first probability distribution according to context information of the first coefficient includes: performing probability estimation based on the context information of the preset area to obtain a first probability estimation result; the context information of the preset area includes context information of the first coefficient; Writing the first coefficient and the first probability estimation result into a compressed code stream includes: The first coefficient, the second coefficient and the first probability estimation result are written into a compressed code stream.

19. The method according to claim 16 or 18, characterized in that The method further comprises: Setting the value of the first identifier of the preset area to a first value to indicate that the first probability estimation result is used when sampling to obtain the estimation coefficients in the preset area; Saving the first probability estimation result to a probability estimation result set, and recording the index of the first probability estimation result in the probability estimation result set; Writing the first coefficient, the second coefficient, and the first probability estimation result into a compressed code stream includes: The first coefficient, the second coefficient, the probability estimation result set, the index, the size information of the preset area and the first identifier are written into the compressed code stream.

20. The method according to claim 16 or 18, characterized in that The method further comprises: Setting the value of the first identifier of the preset area to a first value to indicate that the first probability estimation result is used when sampling to obtain the estimation coefficients in the preset area; Writing the first coefficient, the second coefficient, and the first probability estimation result into a compressed code stream includes: The first coefficient, the second coefficient, the first probability estimation result, the size information of the preset area and the first identifier are written into the compressed code stream.

21. The method according to claim 17, wherein The first coefficient and the second coefficient belong to the same preset area, and the method further includes: Setting the value of the first identifier of the preset area to a second value to indicate that the respective probability estimation results are used when sampling to obtain the estimation coefficients in the preset area; Writing the first coefficient, the first probability estimation result, the second coefficient, and the second probability estimation result into the compressed code stream includes: The first coefficient, the first probability estimation result, the second coefficient, the second probability estimation result, and the first identifier of the preset area are written into the compressed code stream.

22. The method according to claim 21, characterized in that The method further comprises: The probability estimation result of the first coefficient is preprocessed to obtain a processed probability estimation result.

23. The method according to claim 22, characterized in that The probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution, and the preprocessing of the probability estimation result of the first coefficient to obtain a processed probability estimation result includes: The variance of the Gaussian distribution is set to 0 as the processed variance, wherein the processed probability estimation result includes the mean of the Gaussian distribution and the processed variance.

24. The method according to claim 22, characterized in that The probability estimation result of the first coefficient includes a mean and a variance of a Gaussian distribution, and the preprocessing of the probability estimation result of the first coefficient to obtain a processed probability estimation result includes: Preprocessing the variance of the Gaussian distribution according to the scaling factor of the first coefficient to obtain a processed variance, wherein the processed probability estimation result includes a mean of the Gaussian distribution and the processed variance; The method further comprises preprocessing the variance of the second probability distribution according to a scaling factor of the second coefficient, wherein: The scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or The scaling factor of the first coefficient is different from the scaling factor of the second coefficient; or, If the first coefficient and the second coefficient belong to the same image block in the to-be-encoded image, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different image blocks, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the image block to which the first coefficient belongs; or, If the first coefficient and the second coefficient belong to a sub-band obtained by performing wavelet transform on the image to be encoded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different sub-bands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the sub-band to which the first coefficient belongs; or, If the first coefficient and the second coefficient belong to one of the frequency bands obtained by performing DCT on the image to be encoded, the scaling factor of the first coefficient and the scaling factor of the second coefficient are the same; or if the first coefficient and the second coefficient belong to different frequency bands, the scaling factor of the first coefficient and the scaling factor of the second coefficient are different; or the scaling factor of the first coefficient is determined according to the texture complexity of the frequency band to which the first coefficient belongs; or, If the first coefficient and the second coefficient belong to the same channel of the three-dimensional feature map obtained by feature extraction of the image to be encoded, the scaling factor of the first coefficient is the same as the scaling factor of the second coefficient; or if the first coefficient and the second coefficient belong to different channels, the scaling factor of the first coefficient is different from the scaling factor of the second coefficient; or the scaling factor of the first coefficient is determined according to the texture complexity of the channel to which the first coefficient belongs.

25. The method according to claim 18, wherein The method further comprises: The probability estimation result of the preset area is preprocessed to obtain a processed probability estimation result.

26. The method according to claim 25, characterized in that The probability estimation result of the preset area includes a mean and a variance of a Gaussian distribution, and the preprocessing of the probability estimation result of the preset area to obtain a processed probability estimation result includes: The variance of the Gaussian distribution is set to 0 as a first variance, wherein the processed probability estimation result includes the mean and the first variance of the Gaussian distribution, or, The variance of the Gaussian distribution is processed according to the scaling factor of the preset area to obtain a second variance, wherein the processed probability estimation result includes the mean and the second variance of the Gaussian distribution.

27. The method according to claim 26, characterized in that If the multiple coefficients are multiple pixel values ​​in the image to be encoded, the context information of the first coefficient includes part or all of the pixel values ​​in the image to be encoded; or If the plurality of coefficients are obtained by performing wavelet transform on the image to be encoded, the plurality of coefficients are a plurality of wavelet coefficients, and the context information of the first coefficient includes part or all of the plurality of wavelet coefficients; or If the image to be encoded is subjected to wavelet transformation and quantization to obtain the multiple coefficients, the multiple coefficients are multiple quantized wavelet coefficients, and the context information of the first coefficient includes part or all of the multiple quantized wavelet coefficients; or If DCT is performed on the image to be encoded to obtain the multiple coefficients, the multiple coefficients are multiple DCT coefficients, and the context information of the first coefficient includes part or all of the multiple DCT coefficients; or If DCT and quantization are performed on the image to be encoded to obtain the multiple coefficients, the multiple coefficients are multiple quantized DCT coefficients, and the context information of the first coefficient includes part or all of the multiple quantized DCT coefficients; or If the plurality of coefficients are obtained by performing feature extraction on the image to be encoded, the plurality of coefficients are a plurality of characteristic coefficients, and the context information of the first coefficient includes part or all of the plurality of characteristic coefficients; or If feature extraction and quantization are performed on the image to be encoded to obtain the multiple coefficients, the multiple coefficients are multiple quantized feature coefficients, and the context information of the first coefficient includes part or all of the multiple quantized feature coefficients.

28. The method according to claim 27, wherein obtaining a first probability estimation result based on the context information of the first coefficient comprises: Obtaining a second probability distribution model, inputting the context information of the first coefficient into a third probability estimation network for processing to obtain parameters of the second probability distribution model; and obtaining the first probability estimation result based on the second probability distribution model and the parameters of the second probability distribution model; or, Inputting the context information of the first coefficient into a fourth probability estimation network for processing to obtain the probability estimation result; Wherein, the third probability estimation network and the fourth probability estimation network are implemented by neural networks.

29. An image processing method implemented by a decoding device, characterized in that: include: Obtaining a first probability estimation result from decoding the compressed code stream; Sampling is performed according to the first probability estimation result to obtain a first estimation coefficient; Obtaining a first reconstructed image according to the first estimation coefficient; Obtaining a first probability estimation result from decoding the compressed code stream includes: Decoding a first identifier from the compressed code stream; If the value of the first identifier is a first value, obtaining a first probability estimation result from decoding the compressed code stream includes: Decoding a probability estimation result set and an index of a preset area from the compressed code stream; the preset area includes the first estimation coefficient, and the preset area is an area in the first reconstructed image, Determining a probability estimation result of the preset area from the probability estimation result set according to the index, where the first probability estimation result is the probability estimation result of the preset area; The value of the first identifier is that the first value is used to indicate that the probability estimation result of the preset area is used when sampling to obtain all estimation coefficients in the preset area.

30. The method according to claim 29, wherein The method further comprises: Obtaining a second probability estimation result by decoding the compressed code stream; Sampling is performed according to the second probability estimation result to obtain a second estimation coefficient; Obtaining a first reconstructed image according to the first estimation coefficient includes: The first reconstructed image is obtained according to the first estimation coefficient and the second estimation coefficient.

31. The method according to claim 29, wherein The method further comprises: Decoding a first identifier from the compressed code stream; If the value of the first identifier is a first value, obtaining a first probability estimation result from decoding the compressed code stream includes: Decoding a probability estimation result of a preset region and size information of the preset region from the compressed code stream; the preset region includes the first estimation coefficient, and the preset region is a region in the first reconstructed image; the probability estimation result of the preset region is the first probability estimation result; The value of the first identifier is that the first value is used to indicate that the probability estimation result of the preset area is used when sampling to obtain all coefficients to be estimated in the preset area.

32. The method according to claim 30, wherein The first estimation coefficient and the second estimation coefficient belong to the same preset area, and the preset area is an area in the first reconstructed image. The method further includes: Decoding a first identifier from the compressed code stream; If the value of the first identifier is the second value, the value of the first identifier is the second value used to indicate that respective probability estimation results are used when sampling to obtain all coefficients to be estimated in the preset area.

33. The method according to any one of claims 29 to 32, characterized in that The first probability estimation result includes a mean and a variance of a Gaussian distribution, and sampling according to the first probability estimation result to obtain a first estimation coefficient includes: Get the first random number; determining a first reference value according to the first random number, where the first reference value obeys a Gaussian distribution; The first estimation coefficient is determined according to the first reference value and the mean and variance of the first probability estimation result.

34. The method according to claim 33, wherein The method further comprises: Preprocessing the variance of the first probability estimation result to obtain a processed variance; The determining the first estimation coefficient according to the first reference value and the mean and variance of the first probability estimation result includes: The first estimation coefficient is determined according to the first reference value, the mean of the first probability estimation result, and the processed variance.

35. The method according to claim 34, wherein The preprocessing of the variance of the first probability estimation result to obtain a processed variance includes: The variance of the first probability estimation result is set to 0 as the processed variance.

36. The method according to claim 34, wherein when the first estimation coefficient is a quantized wavelet coefficient, or a wavelet coefficient, or a quantized discrete cosine transform (DCT) coefficient, or a DCT coefficient, or a characteristic coefficient, or a quantized characteristic coefficient, the preprocessing of the variance of the first probability estimation result to obtain a processed variance comprises: preprocessing the variance of the first probability estimation result according to the scaling factor of the first estimation coefficient to obtain a processed variance, The scaling factor of the first estimated coefficient is the same as the scaling factor of the second estimated coefficient; or, The scaling factor of the first estimated coefficient is different from the scaling factor of the second estimated coefficient; or When the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients or wavelet coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same subband, the scaling factor of the first estimation coefficient and the scaling factor of the second estimation coefficient are the same; or if the first estimated coefficient and the second estimated coefficient belong to different subbands, the scaling factor of the first estimated coefficient and the scaling factor of the second estimated coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the image block to which the first estimation coefficient belongs; or, When the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients or DCT coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same frequency band, a scaling factor of the first estimation coefficient and a scaling factor of the second estimation coefficient are the same; or if the first estimated coefficient and the second estimated coefficient belong to different frequency bands, the scaling factor of the first estimated coefficient and the scaling factor of the second estimated coefficient are different; or the scaling factor of the first estimation coefficient is determined according to the texture complexity of the frequency band to which the first estimation coefficient belongs; or, When the first estimation coefficient and the second estimation coefficient are characteristic coefficients or quantized characteristic coefficients, if the first estimation coefficient and the second estimation coefficient belong to the same channel, a scaling factor of the first estimation coefficient and a scaling factor of the second estimation coefficient are the same; or if the first estimated coefficient and the second estimated coefficient belong to different channels, the scaling factor of the first estimated coefficient and the scaling factor of the second estimated coefficient are different; If the scaling factor of the first estimation coefficient is determined according to the texture complexity of the channel to which the first estimation coefficient belongs.

37. The method according to claim 34, wherein when the first estimation coefficient and the second estimation coefficient are pixel values, the preprocessing the variance of the first probability estimation result to obtain a processed variance comprises: preprocessing the variance of the first probability estimation result according to the scaling factor of the first estimation coefficient to obtain a processed variance, The scaling factor of the first estimated coefficient is the same as the scaling factor of the second estimated coefficient, or the scaling factor of the first estimated coefficient is different from the scaling factor of the second estimated coefficient; or, If the first estimation coefficient and the second estimation coefficient belong to the same image block and the resolution of the image block is lower than a preset resolution, a scaling factor of the first estimation coefficient and a scaling factor of the second estimation coefficient are different; Alternatively, if the first estimation coefficient and the second estimation coefficient belong to the same image block, and the resolution of the image block is not lower than the preset resolution, the scaling factor of the first estimation coefficient is the same as the scaling factor of the second estimation coefficient.

38. The method according to claim 30, wherein The obtaining the first reconstructed image according to the first estimation coefficient and the second estimation coefficient includes: If the first estimation coefficient and the second estimation coefficient are quantized wavelet coefficients, the first estimation coefficient and the second estimation coefficient are subjected to inverse quantization and inverse wavelet transformation to obtain the first reconstructed image, or If the first estimation coefficient and the second estimation coefficient are wavelet coefficients, performing inverse wavelet transform on the first estimation coefficient and the second estimation coefficient to obtain the first reconstructed image, or If the first estimation coefficient and the second estimation coefficient are quantized DCT coefficients, the first estimation coefficient and the second estimation coefficient are subjected to inverse quantization and inverse DCT to obtain the first reconstructed image, or If the first estimation coefficient and the second estimation coefficient are DCT coefficients, inverse DCT is performed on the first estimation coefficient and the second estimation coefficient to obtain the first reconstructed image.

39. The method according to claim 38, characterized in that The method further comprises: Decoding the compressed code stream to obtain a plurality of reconstruction coefficients; A second reconstructed image is obtained according to the multiple reconstruction coefficients.

40. The method according to claim 39, wherein Obtaining a second reconstructed image according to the multiple coefficients includes: If the multiple reconstruction coefficients are quantized wavelet coefficients, dequantization and inverse wavelet transform are performed on the multiple reconstruction coefficients to obtain the second reconstructed image, or, If the multiple reconstruction coefficients are wavelet coefficients, performing inverse wavelet transform on the multiple reconstruction coefficients to obtain the second reconstructed image, or, If the multiple reconstruction coefficients are quantized DCT coefficients, performing inverse quantization and inverse DCT on the multiple reconstruction coefficients to obtain the second reconstructed image, or, If the multiple reconstruction coefficients are DCT coefficients, inverse DCT is performed on the multiple reconstruction coefficients to obtain the second reconstructed image.

41. A decoder, characterized in that comprising a processing circuit for performing the method according to any one of claims 29-40.

42. An encoder, characterized in that comprising a processing circuit for executing the method according to any one of claims 1-28.

43. A computer program product, characterized in that The method comprises a program code for executing the method according to any one of claims 1 to 40 when the program code is executed on a computer or a processor.

44. A decoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium is coupled to the processor and stores a program executed by the processor, wherein when the program is executed by the processor, the decoder performs the method according to any one of claims 29 to 40.

45. An encoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium is coupled to the processor and stores a program executed by the processor, wherein when the program is executed by the processor, the encoder performs the method according to any one of claims 1 to 28.

46. ​​A non-transitory computer-readable storage medium, characterized in that The invention comprises a program code, which, when executed by a computer device, is used to perform the method according to any one of claims 1 to 40.

Images