Decoding and coding method, device and equipment

Abstract

The invention provides a decoding and coding method, device and equipment thereof. The method comprises the following steps: acquiring a target probability distribution model corresponding to a first scaling factor based on a first code stream corresponding to a current block; decoding a second code stream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block; and determining a reconstructed image block corresponding to the current block based on the corrected residual feature. According to the invention, the coding performance and the decoding performance are improved.

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to a decoding and coding method, device and equipment thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. Complete video encoding can include prediction, transformation, quantization, entropy coding, filtering and other processes. For the prediction process, the prediction process can include intra-frame prediction and inter-frame prediction. Inter-frame prediction refers to the use of the correlation in the video time domain to use the pixels of the adjacent encoded images to predict the current pixel, so as to effectively remove the redundancy in the video time domain. Intra-frame prediction refers to the use of the correlation in the video space domain to use the pixels of the encoded blocks of the current frame image to predict the current pixel, so as to remove the redundancy in the video space domain.

[0003] With the rapid development of deep learning, deep learning has achieved success in many high-level computer vision problems, such as image classification and target detection. Deep learning has also gradually begun to be applied in the field of encoding and decoding, that is, neural networks can be used to encode and decode images. Although the encoding and decoding methods based on neural networks have shown great performance potential, they still have problems such as poor encoding performance, poor decoding performance and high complexity. Summary of the invention

[0004] In view of this, the present application provides a decoding and encoding method, device and equipment thereof to improve encoding performance and decoding performance.

[0005] The present application provides a decoding method, which is applied to a decoding end, and the method includes:

[0006] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0007] Decoding a second bitstream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block;

[0008] A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

[0009] The present application provides an encoding method, which is applied to an encoding end, and the method includes:

[0010] Correct the residual features corresponding to the current block to obtain the corrected residual features;

[0011] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0012] The modified residual feature is encoded based on the target probability distribution model to obtain a second bitstream corresponding to the current block, and the second bitstream is sent to a decoding end.

[0013] The present application provides a decoding device, the device comprising:

[0014] a memory configured to store video data;

[0015] A decoder configured to implement:

[0016] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0017] Decoding a second bitstream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block;

[0018] A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

[0019] The present application provides an encoding device, the device comprising:

[0020] a memory configured to store video data;

[0021] An encoder configured to implement:

[0022] Correct the residual features corresponding to the current block to obtain the corrected residual features;

[0023] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0024] The modified residual feature is encoded based on the target probability distribution model to obtain a second bitstream corresponding to the current block, and the second bitstream is sent to a decoding end.

[0025] The present application provides a decoding end device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0026] The processor is used to execute machine executable instructions to implement the above decoding method.

[0027] The present application provides an encoding end device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0028] The processor is used to execute machine executable instructions to implement the above encoding method.

[0029] It can be seen from the above technical scheme that in the embodiment of the present application, the target probability distribution model corresponding to the first scaling factor is obtained, and the second code stream corresponding to the current block is decoded based on the target probability distribution model corresponding to the first scaling factor to obtain the residual feature corresponding to the current block. The reconstructed image block corresponding to the current block is determined based on the residual feature. When determining the target probability distribution model, the change of the residual can be fully perceived, and the change of the residual is applied to the target probability distribution model, so that the probability distribution describing the residual is more accurate, thereby achieving the purpose of reducing the bit rate. The parameters of the target probability distribution model used for encoding the residual are adjusted synchronously with the residual, and the scaling factor of the residual on the encoding end acts on the original probability distribution model to obtain the target probability distribution model. When encoding and decoding are performed based on the target probability distribution model, the encoding performance and decoding performance can be improved, and the complexity is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the video coding framework;

[0031] Figure 2 is a flowchart of a decoding method in one embodiment of the present application;

[0032] Figure 3 is a flowchart of an encoding method in one embodiment of the present application;

[0033] Figure 4 is a schematic diagram of a processing process of an encoding end in one implementation manner of the present application;

[0034] Figure 5 is a schematic diagram of the operation of the G-Unit in one embodiment of the present application;

[0035] Figure 6 is a schematic diagram of a processing process of a decoding end in one embodiment of the present application;

[0036] Figure 7 is a schematic diagram of a processing process of an encoding end in one implementation manner of the present application;

[0037] Figure 8 is a flowchart of a decoding method in one embodiment of the present application;

[0038] Fig. 9A and Fig. 9B is a schematic diagram of correction of the initial standard deviation in one embodiment of the present application;

[0039] Fig. 10A is a hardware structure diagram of a decoding end device in one embodiment of the present application;

[0040] Fig. 10B It is a hardware structure diagram of an encoding end device in one implementation of the present application. DETAILED DESCRIPTION

[0041] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, depending on the context. In addition, the word "if" used can be interpreted as "at ... ", or "when ... ", or "in response to determination".

[0042] In the embodiments of the present application, a decoding and encoding method is proposed, which may involve the following concepts:

[0043] Neural Network (NN): Neural network refers to artificial neural network. Neural network is a computing model composed of a large number of nodes (or neurons) connected to each other. In a neural network, neuron processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in a neural network can be divided into three categories: input units, output units, and hidden units. Input units receive signals and data from the outside world; output units realize the output of processing results; hidden units are units between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units, and the representation and processing of information are reflected in the connection relationship of processing units. Neural network is a non-programmed, brain-like information processing method. The essence of neural network is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of neural network, and imitate the information processing function of human brain nervous system to different degrees and levels. In the field of video processing, commonly used neural networks may include but are not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0044] Convolutional Neural Network (CNN): Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. The artificial neurons of convolutional neural network can respond to surrounding units within a certain coverage range and have excellent performance in large-scale image processing. The basic structure of convolutional neural network includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer and extracts the local features. Once the local feature is extracted, its positional relationship with other features is also determined. The second is the feature mapping layer (also called activation layer). Each computing layer of the neural network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since the neurons on a mapping surface share weights, the number of free parameters of the network is reduced.

[0045] For example, one of the advantages of convolutional neural networks over image processing algorithms is that they avoid the complex pre-processing of images (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of convolutional neural networks over ordinary neural networks is that ordinary neural networks use a fully connected approach, that is, all neurons from the input layer to the hidden layer are connected, which will result in a huge number of parameters, making network training time-consuming or even difficult to train, while convolutional neural networks avoid this difficulty through local connections, weight sharing and other methods.

[0046] Deconvolution layer: Deconvolution layer is also called transposed convolution layer. The working process of deconvolution layer and convolution layer is very similar. The main difference is that deconvolution layer will make the output larger than the input through padding (of course, it can also remain the same). If stride is 1, it means that the output size is equal to the input size; if stride is N, it means that the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0047] Generalization Ability: Generalization ability refers to the ability of a machine learning algorithm to adapt to fresh samples. The purpose of learning is to learn the rules hidden behind the data pairs. For data outside the learning set with the same rules, the trained network can also give appropriate outputs. This ability can be called generalization ability.

[0048] Rate-Distortion Optimized: There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode: J(mode) = D + λ*R, where D represents Distortion, which can usually be measured using the SSE indicator. SSE refers to the mean square sum of the difference between the reconstructed image block and the source image. In order to achieve cost considerations, the SAD indicator can also be used. SAD refers to the sum of the absolute values ​​of the difference between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc. When selecting a mode, if the rate-distortion principle is used to make a comparison decision on the encoding mode, the best encoding performance can usually be guaranteed.

[0049] Entropy encoding: Entropy encoding is the encoding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information in the source (a measure of uncertainty). For example, common entropy encoding methods include but are not limited to: Shannon coding, Huffman coding, and arithmetic coding.

[0050] Video Coding Framework: See Figure 1 FIG. 1 is a schematic diagram of a video encoding framework of an encoding end, and the video encoding framework can be used to implement the encoding end processing flow of the embodiment of the present application. The schematic diagram of the video decoding framework can be similar to FIG. Figure 1 Similarly, without repeating the details here, the video decoding framework can be used to implement the decoding end processing flow of the embodiment of the present application.

[0051] For example, see Figure 1 As shown, the video encoding framework may include modules such as prediction, transformation, quantization, entropy encoder, inverse quantization, inverse transformation, reconstruction, and filtering. At the encoding end, the processing flow of the encoding end can be realized through the cooperation between these modules. In addition, the video decoding framework may include modules such as prediction, transformation, quantization, entropy decoder, inverse quantization, inverse transformation, reconstruction, and filtering. At the decoding end, the processing flow of the decoding end can be realized through the cooperation between these modules.

[0052] A large number of coding tools have been proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the coding tools that can achieve the best coding performance are often different. Therefore, in the encoding process, RDO (Rate-Distortion Optimize) is usually used to compare the coding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the marker information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to obtain the best coding performance. The decoding end can obtain relevant mode information by directly parsing the flag information, and the complexity impact is small.

[0053] In the embodiments of the present application, a decoding method and an encoding method are proposed, which can perform decoding and encoding based on a probability distribution model of residual correction, thereby achieving the purpose of reducing the bit rate by obtaining a probability distribution model that is more consistent with the data.

[0054] The decoding method and encoding method in the embodiments of the present application are described in detail below in combination with several specific embodiments.

[0055] Example 1: A decoding method is proposed in the embodiment of the present application, see Figure 2 As shown, it is a flowchart of the decoding method, which can be applied to a decoding end (also called a video decoder), and the method may include:

[0056] Step 201: Obtain a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to a current block.

[0057] Step 202: Decode the second bitstream corresponding to the current block based on the target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block.

[0058] Step 203: Determine the reconstructed image block corresponding to the current block based on the corrected residual features.

[0059] Exemplarily, obtaining a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to a current block may include, but is not limited to: obtaining a target probability distribution parameter corresponding to a first scaling factor based on a first bitstream corresponding to a current block, and determining a target probability distribution model based on the target probability distribution parameter. The target probability distribution parameter includes, but is not limited to, a target mean and / or a target standard deviation. Of course, the target mean and target standard deviation are merely examples and are not limited thereto.

[0060] In a possible implementation, if the target probability distribution parameter is a target standard deviation, obtaining the target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block may include but is not limited to: decoding the first bitstream to obtain an initial coefficient hyperparameter feature corresponding to the current block; performing an inverse transformation of the coefficient hyperparameter feature on the initial coefficient hyperparameter feature to obtain an initial standard deviation; and correcting the initial standard deviation based on the first scaling factor to obtain a target standard deviation.

[0061] Exemplarily, correcting the initial standard deviation based on the first scaling factor to obtain the target standard deviation may include, but is not limited to: generating the target scaling factor based on the first scaling factor; correcting the initial standard deviation based on the target scaling factor to obtain the target standard deviation. The target scaling factor may be the first scaling factor; or the target scaling factor is determined based on the first scaling factor and the second scaling factor; and the second scaling factor is used to represent the mapping relationship between the residual and the standard deviation. The second scaling factor is pre-configured; or the second scaling factor is decoded from the bitstream corresponding to the current block.

[0062] In another possible implementation, if the target probability distribution parameter is a target standard deviation, obtaining the target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block may include but is not limited to: decoding the first bitstream to obtain an initial coefficient hyper-parameter feature corresponding to the current block; correcting the initial coefficient hyper-parameter feature based on the first scaling factor to obtain a target coefficient hyper-parameter feature; and performing an inverse transformation of the coefficient hyper-parameter feature on the target coefficient hyper-parameter feature to obtain a target standard deviation.

[0063] Exemplarily, the target coefficient hyperparameter feature is obtained by correcting the initial coefficient hyperparameter feature based on the first scaling factor, which may include but is not limited to: generating a target scaling factor based on the first scaling factor, and correcting the initial coefficient hyperparameter feature based on the target scaling factor to obtain the target coefficient hyperparameter feature. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and a third scaling factor; wherein the third scaling factor is used to represent the mapping relationship between the residual and the coefficient hyperparameter feature. The third scaling factor may be pre-configured; or the third scaling factor may be decoded from the bitstream corresponding to the current block.

[0064] In a possible implementation, determining the reconstructed image block corresponding to the current block based on the corrected residual feature may include, but is not limited to: correcting the corrected residual feature based on the first scaling factor to obtain the residual feature before correction (i.e., the residual feature before correction at the encoding end), and determining the reconstructed image block corresponding to the current block based on the residual feature. For example, obtaining the mean value corresponding to the current block, and determining the reconstructed image block corresponding to the current block based on the residual feature and the mean value corresponding to the current block.

[0065] In a possible implementation, decoding the first bitstream to obtain the initial coefficient super-parameter feature corresponding to the current block may include, but is not limited to: decoding the first bitstream to obtain the super-parameter quantization feature corresponding to the current block, and dequantizing the super-parameter quantization feature to obtain the initial coefficient super-parameter feature corresponding to the current block. Alternatively, when decoding the first bitstream, the initial coefficient super-parameter feature corresponding to the current block may be directly obtained without the need for a dequantization operation.

[0066] In a possible implementation, decoding the second bitstream corresponding to the current block based on the target probability distribution model corresponding to the first scaling factor to obtain a modified residual feature corresponding to the current block may include but is not limited to: decoding the second bitstream corresponding to the current block based on the target probability distribution model to obtain a residual quantization feature corresponding to the current block, and dequantizing the residual quantization feature to obtain a modified residual feature corresponding to the current block. Alternatively, when decoding the second bitstream corresponding to the current block based on the target probability distribution model, the modified residual feature corresponding to the current block may be directly obtained.

[0067] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0068] It can be seen from the above technical scheme that in the embodiment of the present application, the target probability distribution model corresponding to the first scaling factor is obtained, and the second code stream corresponding to the current block is decoded based on the target probability distribution model corresponding to the first scaling factor to obtain the residual feature corresponding to the current block. The reconstructed image block corresponding to the current block is determined based on the residual feature. When determining the target probability distribution model, the change of the residual can be fully perceived, and the change of the residual is applied to the target probability distribution model, so that the probability distribution describing the residual is more accurate, thereby achieving the purpose of reducing the bit rate. The parameters of the target probability distribution model used for encoding the residual are adjusted synchronously with the residual, and the scaling factor of the residual on the encoding end acts on the original probability distribution model to obtain the target probability distribution model. When encoding and decoding are performed based on the target probability distribution model, the encoding performance and decoding performance can be improved, and the complexity is low.

[0069] Example 2: This application proposes an encoding method, see Figure 3As shown, it is a schematic diagram of the process of the encoding method, which can be applied to an encoding end (also called a video encoder), and the method may include:

[0070] Step 301: Correct the residual features corresponding to the current block to obtain corrected residual features.

[0071] Step 302: Obtain a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block.

[0072] Step 303: Encode the corrected residual feature based on the target probability distribution model corresponding to the first scaling factor to obtain a second bitstream corresponding to the current block, and send the second bitstream to the decoding end.

[0073] Exemplarily, correcting the residual feature corresponding to the current block to obtain the corrected residual feature may include, but is not limited to: correcting the residual feature corresponding to the current block based on the first scaling factor to obtain the corrected residual feature.

[0074] Exemplarily, obtaining a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to a current block may include, but is not limited to: obtaining a target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block, and determining a target probability distribution model corresponding to the first scaling factor based on the target probability distribution parameter.

[0075] Exemplarily, before obtaining the target probability distribution model corresponding to the first scaling factor based on the first bitstream corresponding to the current block, feature transformation can be performed on the current block to obtain image features corresponding to the current block, and coefficient hyperparameter feature transformation can be performed on the image features to obtain initial coefficient hyperparameter features corresponding to the current block, and the initial coefficient hyperparameter features can be encoded to obtain the first bitstream.

[0076] Exemplarily, encoding the modified residual feature based on the target probability distribution model corresponding to the first scaling factor to obtain the second bitstream corresponding to the current block may include, but is not limited to: quantizing the modified residual feature to obtain the residual quantization feature corresponding to the current block, and encoding the residual quantization feature based on the target probability distribution model corresponding to the first scaling factor to obtain the second bitstream corresponding to the current block. Alternatively, the modified residual feature may be directly encoded based on the target probability distribution model corresponding to the first scaling factor to obtain the second bitstream corresponding to the current block without involving a quantization operation.

[0077] Exemplarily, encoding the initial coefficient super-parameter feature to obtain the first bitstream may include but is not limited to: quantizing the initial coefficient super-parameter feature to obtain the super-parameter quantization feature corresponding to the current block, and encoding the super-parameter quantization feature to obtain the first bitstream. Alternatively, the initial coefficient super-parameter feature may be directly encoded to obtain the first bitstream without involving a quantization operation.

[0078] Exemplarily, when obtaining the target probability distribution parameters corresponding to the first scaling factor based on the first bitstream corresponding to the current block, the target probability distribution parameters include but are not limited to the target mean and / or target standard deviation. Of course, the target mean and target standard deviation are just examples and are not limited to this. If the target probability distribution parameter is the target standard deviation, obtaining the target probability distribution parameters corresponding to the first scaling factor based on the first bitstream corresponding to the current block may include but is not limited to: decoding the first bitstream to obtain the initial coefficient hyperparameter features corresponding to the current block; performing a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter features to obtain the initial standard deviation; and correcting the initial standard deviation based on the first scaling factor to obtain the target standard deviation.

[0079] Exemplarily, correcting the initial standard deviation based on the first scaling factor to obtain the target standard deviation may include, but is not limited to: generating the target scaling factor based on the first scaling factor; correcting the initial standard deviation based on the target scaling factor to obtain the target standard deviation. The target scaling factor may be the first scaling factor; or the target scaling factor is determined based on the first scaling factor and the second scaling factor; wherein the second scaling factor is used to represent the mapping relationship between the residual and the standard deviation.

[0080] In another possible implementation, if the target probability distribution parameter is a target standard deviation, obtaining the target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block may include but is not limited to: decoding the first bitstream to obtain an initial coefficient hyper-parameter feature corresponding to the current block; correcting the initial coefficient hyper-parameter feature based on the first scaling factor to obtain a target coefficient hyper-parameter feature; and performing an inverse transformation of the coefficient hyper-parameter feature on the target coefficient hyper-parameter feature to obtain a target standard deviation.

[0081] Exemplarily, the method of modifying the initial coefficient hyperparameter feature based on the first scaling factor to obtain the target coefficient hyperparameter feature may include, but is not limited to: generating a target scaling factor based on the first scaling factor, and modifying the initial coefficient hyperparameter feature based on the target scaling factor to obtain the target coefficient hyperparameter feature. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and a third scaling factor; wherein the third scaling factor is used to represent the mapping relationship between the residual and the coefficient hyperparameter feature.

[0082] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0083] It can be seen from the above technical solutions that in the embodiments of the present application, when determining the target probability distribution model, the change of the residual can be fully perceived, and the change of the residual is applied to the target probability distribution model, so that the probability distribution describing the residual is more accurate, thereby achieving the purpose of reducing the bit rate. The parameters of the target probability distribution model used for encoding the residual are adjusted synchronously with the residual, and the scaling factor of the residual on the encoding end is applied to the original probability distribution model to obtain the target probability distribution model. When encoding and decoding are performed based on the target probability distribution model, the encoding performance and decoding performance can be improved, and the complexity is low.

[0084] Embodiment 3: For Embodiment 1 and Embodiment 2, the processing process of the encoding end can be referred to Figure 4 As shown, of course, Figure 4 This is just an example of the processing process at the encoding end, and there is no limitation on the processing process at the encoding end.

[0085] After obtaining the current block x, the encoder can perform feature transformation on the current block x (i.e., perform analysis transformation on the current block x) to obtain the image feature y corresponding to the current block x. The feature transformation on the current block x means: transforming the current block x to the image feature y in the latent domain, so that all subsequent processes can be operated on the latent domain.

[0086] After obtaining the image feature y, the encoding end performs coefficient hyperparameter feature transformation on the image feature y to obtain the initial coefficient hyperparameter feature z. For example, the image feature y can be input into the super encoding network, and the super encoding network performs coefficient hyperparameter feature transformation on the image feature y to obtain the initial coefficient hyperparameter feature z. The super encoding network can be a trained neural network. There is no restriction on the training process of the super encoding network. It only needs to perform coefficient hyperparameter feature transformation on the image feature y. The image feature y in the latent domain obtains the super prior latent information z after passing through the super encoding network.

[0087] After obtaining the initial coefficient hyperparameter feature z, the encoder can quantize the initial coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the initial coefficient hyperparameter feature z, that is, Figure 4The Q operation in is the quantization process. After obtaining the hyperparameter quantization feature corresponding to the initial coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain Bitstream#1 (i.e., the first bitstream) corresponding to the current block, i.e. Figure 4 The AE operation in represents the encoding process, such as the entropy encoding process. Alternatively, the encoding end can also directly encode the initial coefficient hyperparameter feature z to obtain the Bitstream#1 corresponding to the current block. Among them, the hyperparameter quantization feature or the initial coefficient hyperparameter feature z carried in Bitstream#1 is mainly used to obtain the parameters of the mean and probability distribution model.

[0088] After obtaining the Bitstream#1 corresponding to the current block, the encoder may send the Bitstream#1 corresponding to the current block to the decoder. For the processing process of the decoder for the Bitstream#1 corresponding to the current block, refer to the subsequent embodiments.

[0089] After obtaining the Bitstream#1 corresponding to the current block, the encoder can also decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 4 AD in it represents the decoding process. Then, the hyperparameter quantization feature is dequantized to obtain the initial coefficient hyperparameter feature z_hat. The initial coefficient hyperparameter feature z_hat may be the same as or different from the initial coefficient hyperparameter feature z. Figure 4 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current block, the encoder can also decode the Bitstream#1 to obtain the initial coefficient hyperparameter feature z_hat without involving the inverse quantization process.

[0090] After obtaining the initial coefficient hyperparameter feature z_hat, the encoder can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block (the determination process of the residual feature y_hat is described in the subsequent embodiments) to obtain the mean mu corresponding to the current block, and there is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the initial coefficient hyperparameter feature z_hat and the decoded residual feature y_hat, and the two are jointly input to obtain a more accurate mean mu, and the mean mu is used to obtain the residual by subtracting from the original feature and adding to the decoded residual to obtain the reconstructed y.

[0091] After obtaining the image feature y, the encoder determines the residual feature r based on the image feature y and the mean mu. The residual feature r can be understood as the residual feature r before correction. For example, the difference between the image feature y and the mean mu is taken as the residual feature r.

[0092] After obtaining the residual feature r, the encoder can correct the residual feature r to obtain a corrected residual feature r'. For example, the encoder corrects the residual feature r based on the first scaling factor to obtain a corrected residual feature r'. For example, the G-Unit operation can be a scaling operation. The encoder can perform a scaling operation on the residual feature r based on the first scaling factor (i.e., calculate the product of the first scaling factor and the residual feature r) to obtain a corrected residual feature r'. See Figure 5 As shown, the input of the G-Unit operation is the residual feature r, and the output is the residual feature r' after the scaling operation. The first scaling factor can be scale, that is, the residual feature r is scaled by the first scaling factor to obtain the corrected residual feature r'.

[0093] from Figure 4 It can be seen that the residual feature r undergoes the G-Unit operation before Bitstream#2 encoding and decoding. The relationship between the G-Unit operation and the residual feature r is as follows: Figure 5 As shown, G-Unit includes a trainable matrix, which is denoted as the first scaling factor scale, C represents the number of channels of the latent domain feature, ⊙ represents the channel-level product of the residual feature r and the first scaling factor scale, that is, the residual feature r is multiplied by the correction value in the channel vector corresponding to the first scaling factor scale.

[0094] From the above, it can be seen that the first scaling factor scale is a correction parameter of the residual feature r. The first scaling factor scale is used to correct the residual feature r so that the corrected residual feature r' is more accurate and reliable. The first scaling factor scale can be pre-configured or obtained through training, and there is no restriction on the source of the first scaling factor scale.

[0095] After obtaining the residual feature r', the encoder can quantize the residual feature r' to obtain the residual quantization feature corresponding to the residual feature r', that is, Figure 4 The Q operation in is the quantization process. After obtaining the residual quantization feature corresponding to the residual feature r', the encoder can encode the residual quantization feature to obtain Bitstream#2 (i.e., the second bitstream) corresponding to the current block, i.e. Figure 4 The AE operation in represents an encoding process, such as an entropy encoding process. Alternatively, the encoding end can also directly encode the residual feature r' to obtain the Bitstream#2 corresponding to the current block without involving the quantization process of the residual feature r'.

[0096] After obtaining the Bitstream#2 corresponding to the current block, the encoder may send the Bitstream#2 corresponding to the current block to the decoder. For the processing process of the decoder for the Bitstream#2 corresponding to the current block, refer to the subsequent embodiments.

[0097] After obtaining the Bitstream#2 corresponding to the current block, the encoder can also decode Bitstream#2 to obtain the residual quantization feature, that is, Figure 4 AD in it represents the decoding process. Then, the encoder can dequantize the residual quantization feature to obtain the modified residual feature r'_hat. The residual feature r'_hat can be the same as or different from the residual feature r'. Figure 4 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current block, the encoder can also decode the Bitstream#2 to obtain the modified residual feature r'_hat without involving the inverse quantization process of the residual feature.

[0098] After obtaining the corrected residual feature r'_hat, the encoder can correct the corrected residual feature r'_hat to obtain the residual feature r_hat before correction. The residual feature r_hat and the residual feature r can be the same or different. For example, the encoder corrects the residual feature r'_hat based on the first scaling factor to obtain the residual feature r_hat before correction. For example, the IG-Unit operation can be a scaling operation, and the encoder can perform a scaling operation on the residual feature r'_hat based on the first scaling factor (i.e., calculate the quotient of the residual feature r'_hat and the first scaling factor) to obtain the residual feature r_hat before correction. The IG-Unit process can be understood as the inverse process of the G-Unit process, which is used to correct the residual feature r'_hat to the residual feature r_hat.

[0099] After obtaining the residual feature r_hat, the encoder determines the image feature y_hat based on the residual feature r_hat and the mean mu. The image feature y_hat can be the same as or different from the image feature y. For example, the sum of the residual feature r_hat and the mean mu is used as the image feature y_hat. After obtaining the image feature y_hat, the image feature y_hat can be synthesized and transformed to obtain the reconstructed image block x_hat corresponding to the current block x. At this point, the image reconstruction process is completed and the reconstructed image block x_hat is obtained.

[0100] In a possible implementation, when the encoder encodes the residual quantization feature or the residual feature r' to obtain the Bitstream#2 corresponding to the current block, the encoder needs to first determine the probability distribution model, and then encode the residual quantization feature or the residual feature r' based on the probability distribution model. In addition, when the encoder decodes Bitstream#2, it also needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model.

[0101] In order to obtain the probability distribution model, the probability distribution model is related to the mean and standard deviation. If the mean of the probability distribution model is 0, it is only necessary to determine the standard deviation of the probability distribution model, and then determine the probability distribution model based on the standard deviation. If the mean of the probability distribution model is not 0, it is necessary to determine the mean and standard deviation of the probability distribution model, and then determine the probability distribution model based on the mean and standard deviation. There is no limitation on how to determine the probability distribution model in this embodiment.

[0102] To obtain the standard deviation of the probability distribution model, continue to refer to Figure 4 As shown, after obtaining the initial coefficient hyperparameter feature z_hat, the encoding end performs a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat to obtain the standard deviation σ, and then determines the probability distribution model N(0, σ) based on the standard deviation. Here, the mean of the probability distribution model is taken as 0. If the mean of the probability distribution model is not 0, it is also necessary to determine the mean of the probability distribution model. For example, the initial coefficient hyperparameter feature z_hat can be input into the super-scale decoding network, and the super-scale decoding network performs a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat to obtain the standard deviation. Among them, the super-scale decoding network can be a trained neural network, and there is no restriction on the training process of this super-scale decoding network. It only needs to perform a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat.

[0103] In a possible implementation, the processing process at the encoding end is performed by a deep learning model or a neural network model, thereby realizing an end-to-end image compression and encoding process, and there is no limitation on this process.

[0104] Embodiment 4: For Embodiment 1 and Embodiment 2, the processing process of the decoding end can be referred to Figure 6 As shown, of course, Figure 6 This is just an example of the processing process at the decoding end, and there is no limitation on the processing process at the decoding end.

[0105] After obtaining the Bitstream#1 corresponding to the current block, the decoding end can decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 6AD in it represents the decoding process. Then, the hyperparameter quantization feature is dequantized to obtain the initial coefficient hyperparameter feature z_hat. The initial coefficient hyperparameter feature z_hat may be the same as or different from the initial coefficient hyperparameter feature z. Figure 6 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current block, the decoder can also decode the Bitstream#1 to obtain the initial coefficient hyperparameter feature z_hat without involving the inverse quantization process.

[0106] After obtaining the initial coefficient hyperparameter feature z_hat, the decoding end can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block (the determination process of the residual feature y_hat is described in the subsequent embodiments) to obtain the mean mu corresponding to the current block, and there is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the initial coefficient hyperparameter feature z_hat and the decoded residual feature y_hat, and the two are jointly input to obtain a more accurate mean mu, and the mean mu is used to obtain the residual by subtracting from the original feature and adding to the decoded residual to obtain the reconstructed y.

[0107] After obtaining the Bitstream#2 corresponding to the current block, the decoder can also decode Bitstream#2 to obtain the residual quantization feature, that is, Figure 6 AD in it represents the decoding process. Then, the decoding end can dequantize the residual quantization feature to obtain the modified residual feature r'_hat. The residual feature r'_hat can be the same as or different from the residual feature r'. Figure 6 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current block, the decoder can also decode the Bitstream#2 to obtain the modified residual feature r'_hat without involving the inverse quantization process of the residual feature.

[0108] After obtaining the corrected residual feature r'_hat, the decoding end can correct the corrected residual feature r'_hat to obtain the residual feature r_hat before correction. The residual feature r_hat and the residual feature r can be the same or different. For example, the decoding end corrects the residual feature r'_hat based on the first scaling factor to obtain the residual feature r_hat before correction. For example, the IG-Unit operation can be a scaling operation, and the decoding end can perform a scaling operation on the residual feature r'_hat based on the first scaling factor (i.e., calculate the quotient of the residual feature r'_hat and the first scaling factor) to obtain the residual feature r_hat before correction. The IG-Unit process can be understood as the inverse process of the G-Unit process, which is used to correct the residual feature r'_hat to the residual feature r_hat.

[0109] After obtaining the residual feature r_hat, the decoder determines the image feature y_hat based on the residual feature r_hat and the mean mu. The image feature y_hat can be the same as or different from the image feature y. For example, the sum of the residual feature r_hat and the mean mu is used as the image feature y_hat. After obtaining the image feature y_hat, the image feature y_hat can be synthesized and transformed to obtain the reconstructed image block x_hat corresponding to the current block x. At this point, the image reconstruction process is completed and the reconstructed image block x_hat is obtained.

[0110] When decoding Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model. In order to obtain the probability distribution model, the probability distribution model is related to the mean and standard deviation. If the mean of the probability distribution model is 0, it is only necessary to determine the standard deviation of the probability distribution model, and then determine the probability distribution model based on the standard deviation. If the mean of the probability distribution model is not 0, it is necessary to determine the mean and standard deviation of the probability distribution model, and then determine the probability distribution model based on the mean and standard deviation. As for how to determine the probability distribution model, this embodiment does not limit it.

[0111] To obtain the standard deviation of the probability distribution model, continue to refer to Figure 6 As shown, after obtaining the initial coefficient hyperparameter feature z_hat, the decoding end performs a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat to obtain the standard deviation σ, and then determines the probability distribution model N(0, σ) based on the standard deviation. Here, the mean of the probability distribution model is taken as 0. If the mean of the probability distribution model is not 0, it is also necessary to determine the mean of the probability distribution model. For example, the initial coefficient hyperparameter feature z_hat can be input into the super-scale decoding network, and the super-scale decoding network performs a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat to obtain the standard deviation. Among them, the super-scale decoding network can be a trained neural network, and there is no restriction on the training process of this super-scale decoding network. It only needs to perform a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature z_hat.

[0112] In a possible implementation, the processing process at the decoding end is performed by a deep learning model or a neural network model, thereby realizing an end-to-end image decompression and decoding process, and there is no limitation on this process.

[0113] Example 5: For Example 3, when the encoding end determines the probability distribution model, in order to make the probability distribution model match the G-Unit process, that is, the probability distribution model can perceive the scaling effect of the G-Unit process on the residual features, so that there is no obvious deviation between the probability distribution model and the residual features actually to be encoded, then, in this embodiment, it is also possible to obtain the target probability distribution model corresponding to the first scaling factor (that is, the first scaling factor adopted by the G-Unit process), that is, to correct the probability distribution model based on the first scaling factor to obtain the corrected target probability distribution model, encode the residual quantization feature or the residual feature r' based on the target probability distribution model, and decode Bitstream#2 based on the target probability distribution model.

[0114] For example, see Figure 7 As shown, after obtaining the current block x, the encoder can perform feature transformation on the current block x (i.e., perform analysis transformation on the current block x) to obtain the image feature y corresponding to the current block x. After obtaining the image feature y, the encoder performs coefficient hyperparameter feature transformation on the image feature y to obtain the initial coefficient hyperparameter feature z. After obtaining the initial coefficient hyperparameter feature z, the encoder can quantize the initial coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the initial coefficient hyperparameter feature z, encode the hyperparameter quantization feature to obtain the Bitstream#1 (i.e., the first bitstream) corresponding to the current block, or the encoder can directly encode the initial coefficient hyperparameter feature z to obtain the Bitstream#1 corresponding to the current block.

[0115] After obtaining Bitstream#1 corresponding to the current block, the encoder can also decode Bitstream#1 to obtain the hyperparameter quantization feature, dequantize the hyperparameter quantization feature, and obtain the initial coefficient hyperparameter feature z_hat. Alternatively, the encoder can also decode Bitstream#1 to obtain the initial coefficient hyperparameter feature z_hat without involving the dequantization process.

[0116] After obtaining the initial coefficient hyperparameter feature z_hat, the encoder can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the mean mu corresponding to the current block.

[0117] After obtaining the image feature y, the encoder determines the residual feature r based on the image feature y and the mean mu. The encoder corrects the residual feature r to obtain the corrected residual feature r', that is, Figure 7 The residual scaling process in .

[0118] After obtaining the residual feature r', the encoding end can quantize the residual feature r' to obtain the residual quantization feature corresponding to the residual feature r', encode the residual quantization feature to obtain Bitstream#2 (i.e., the second code stream) corresponding to the current block, or the encoding end can directly encode the residual feature r' to obtain Bitstream#2 corresponding to the current block.

[0119] After obtaining Bitstream#2 corresponding to the current block, the encoder can also decode Bitstream#2 to obtain residual quantization features, dequantize the residual quantization features, and obtain the corrected residual features r'_hat. Alternatively, after obtaining Bitstream#2 corresponding to the current block, the encoder can decode Bitstream#2 to obtain the corrected residual features r'_hat.

[0120] After obtaining the corrected residual feature r'_hat, the encoder can also correct the corrected residual feature r'_hat to obtain the residual feature r_hat before correction, that is, Figure 7 The residual inverse scaling process shown in .

[0121] After obtaining the residual feature r_hat, the encoder determines the image feature y_hat based on the residual feature r_hat and the mean mu, and performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current block x.

[0122] In a possible implementation, when the encoder encodes the residual quantization feature or the residual feature r' to obtain the Bitstream#2 corresponding to the current block, the encoder needs to first determine the probability distribution model, and then encode the residual quantization feature or the residual feature r' based on the probability distribution model. Also, when the encoder decodes Bitstream#2, it also needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model.

[0123] In this embodiment, in order to make the probability distribution model match the G-Unit process, that is, the probability distribution model can perceive the scaling effect of the G-Unit process on the residual features, so that there is no obvious deviation between the probability distribution model and the residual features actually to be encoded, the probability distribution model PDM can be corrected to obtain a corrected probability distribution model PDM_hat, and the corrected probability distribution model PDM_hat can be called a target probability distribution model. Among them, in order to make the probability distribution model match the G-Unit process, the probability distribution model PDM can be corrected by using a first scaling factor (i.e., the scaling factor of the G-Unit process) to obtain a corrected target probability distribution model PDM_hat.

[0124] In this embodiment, see Figure 7 As shown, the input of the probability model correction module is mainly: the first scaling factor scale of the residual feature at the encoding end and the original probability distribution model confirmed by the decoding of Bitstream#1, and the output is the adjusted target probability distribution model PDM_hat. The main purpose of the probability model correction module is to optimize the probability distribution model of the entropy coding of Bitstream#2, and to make the bit rate of the encoded bitstream smaller by adjusting the probability distribution parameters of the probability distribution model. For example, by correcting the probability distribution model PDM output by the super prior network, the probability distribution model is made aware of all scaling changes of the residual feature at the encoding end, and the scaling factor of the residual feature and the probability distribution model decoded by Bitstream#2 are jointly passed into the probability model correction module, and the new target probability distribution model PDM_hat is output. The target probability distribution model PDM_hat is more accurate in describing the distribution of the residual feature than the previous probability distribution model, and can achieve the purpose of reducing the bit rate.

[0125] Exemplarily, since the target probability distribution model PDM_hat is determined based on probability distribution parameters (such as mean and / or standard deviation), the target probability distribution parameters corresponding to the first scaling factor can be obtained, and the target probability distribution model PDM_hat corresponding to the first scaling factor can be determined based on the target probability distribution parameters. For example, the initial probability distribution parameters can be corrected using the first scaling factor to obtain the corrected target probability distribution parameters.

[0126] For example, the probability distribution model is related to the mean and standard deviation. If the mean of the probability distribution model is 0, it is only necessary to determine the standard deviation of the probability distribution model, and then determine the probability distribution model based on the standard deviation. In this case, the target standard deviation corresponding to the first scaling factor can be obtained. For example, the initial standard deviation can be corrected by the first scaling factor to obtain the corrected target standard deviation, and then the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target standard deviation.

[0127] For another example, if the mean of the probability distribution model is not 0, it is necessary to determine the mean and standard deviation of the probability distribution model, and then determine the probability distribution model based on the mean and standard deviation. In this case, the target mean and target standard deviation corresponding to the first scaling factor can be obtained. For example, the initial mean can be corrected by the first scaling factor to obtain the corrected target mean, and the initial standard deviation can be corrected by the first scaling factor to obtain the corrected target standard deviation. Then, the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target mean and the target standard deviation.

[0128] For another example, if the mean of the probability distribution model is not 0, but the standard deviation of the probability distribution model is 0, it is necessary to determine the mean of the probability distribution model, and then determine the probability distribution model based on the mean. In this case, the target mean corresponding to the first scaling factor can be obtained. For example, the initial mean can be corrected by the first scaling factor to obtain the corrected target mean, and then the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target mean.

[0129] Exemplarily, in order to obtain the mean of the probability distribution model, a context-based prediction can be performed based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the mean mu corresponding to the current block.

[0130] Exemplarily, in order to obtain the standard deviation of the probability distribution model, after obtaining the initial coefficient hyperparameter feature z_hat, the initial coefficient hyperparameter feature z_hat is subjected to an inverse transformation of the coefficient hyperparameter feature to obtain the standard deviation σ corresponding to the current block.

[0131] Exemplarily, the encoding end may obtain the target probability distribution parameters corresponding to the first scaling factor, and determine the target probability distribution model corresponding to the first scaling factor based on the target probability distribution parameters. For example, the encoding end may modify the initial probability distribution parameters based on the first scaling factor to obtain the target probability distribution parameters corresponding to the first scaling factor.

[0132] In a possible implementation, the target probability distribution parameters may include, but are not limited to, a target mean and / or a target standard deviation. Of course, the target mean and the target standard deviation are only examples, and no limitation is imposed on the target probability distribution parameters.

[0133] Exemplarily, if the target probability distribution parameter is the target standard deviation, then the encoding end can decode the first bitstream to obtain the initial coefficient hyperparameter feature corresponding to the current block, and perform an inverse transformation of the initial coefficient hyperparameter feature to obtain the initial standard deviation, and correct the initial standard deviation based on the first scaling factor to obtain the target standard deviation.

[0134] For example, the encoding end may generate a target scaling factor based on the first scaling factor, and correct the initial standard deviation based on the target scaling factor to obtain the target standard deviation, for example, taking the product of the target scaling factor and the initial standard deviation as the target standard deviation. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and the second scaling factor, such as determined based on the product of the first scaling factor and the second scaling factor, or, based on the product value of the first scaling factor and the second scaling factor, a certain offset value is added to obtain the target scaling factor.

[0135] Among them, the second scaling factor is used to represent the mapping relationship between the residual and the standard deviation. For example, the residual and the standard deviation are values ​​of different magnitudes. The magnitude of the standard deviation can be adjusted to the magnitude of the residual through the second scaling factor, that is, the second scaling factor is used as an adjustment coefficient to adjust the magnitude of the standard deviation to the magnitude of the residual, thereby adjusting the residual and the standard deviation to the same magnitude for calculation, so that the residual and the standard deviation can better match the calculation and improve the calculation accuracy.

[0136] Exemplarily, if the target probability distribution parameter is the target standard deviation, then the encoding end can decode the first bitstream to obtain the initial coefficient hyperparameter feature corresponding to the current block, and modify the initial coefficient hyperparameter feature based on the first scaling factor to obtain the target coefficient hyperparameter feature, and perform an inverse transformation of the coefficient hyperparameter feature on the target coefficient hyperparameter feature to obtain the target standard deviation.

[0137] For example, the encoding end can generate a target scaling factor based on the first scaling factor, and modify the initial coefficient hyperparameter feature based on the target scaling factor to obtain the target coefficient hyperparameter feature. For example, the encoding end can use the product of the target scaling factor and the initial coefficient hyperparameter feature as the target coefficient hyperparameter feature. The target scaling factor can be the first scaling factor; or, the target scaling factor can be determined based on the first scaling factor and the third scaling factor, for example, the target scaling factor can be determined based on the product of the first scaling factor and the third scaling factor, or, based on the product value of the first scaling factor and the third scaling factor, a certain bias value is added to obtain the target scaling factor. Among them, the third scaling factor is used to represent the mapping relationship between the residual and the coefficient hyperparameter features. For example, the residual and the coefficient hyperparameter features are values ​​of different magnitudes. The magnitude of the coefficient hyperparameter features can be adjusted to the magnitude of the residual through the third scaling factor. That is, the third scaling factor is used as an adjustment coefficient to adjust the magnitude of the coefficient hyperparameter features to the magnitude of the residual, thereby adjusting the residual and the coefficient hyperparameter features to the same magnitude for calculation, so that the residual and the coefficient hyperparameter features can better match the calculation and improve the calculation accuracy.

[0138] Exemplarily, if the target probability distribution parameter is the target mean, then the encoder can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the initial mean corresponding to the current block. The encoder can correct the initial mean based on the first scaling factor to obtain the target mean.

[0139] For example, the encoding end may generate a target scaling factor based on the first scaling factor, and modify the initial mean value based on the target scaling factor to obtain the target mean value, for example, the product of the target scaling factor and the initial mean value is used as the target mean value. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and the second scaling factor, such as determined based on the product of the first scaling factor and the second scaling factor, or, based on the product value of the first scaling factor and the second scaling factor, a certain offset value is added to obtain the target scaling factor.

[0140] Among them, the second scaling factor is used to represent the mapping relationship between the residual and the mean. For example, the residual and the mean are values ​​of different orders of magnitude. The order of magnitude of the mean can be adjusted to the order of magnitude of the residual through the second scaling factor. That is, the second scaling factor is used as an adjustment coefficient to adjust the order of magnitude of the mean to the order of magnitude of the residual, thereby adjusting the residual and the mean to the same order of magnitude for calculation, so that the residual and the mean can better match the calculation and improve the calculation accuracy.

[0141] Example 6: For Example 4, when the decoding end determines the probability distribution model, in order to make the probability distribution model match the G-Unit process, that is, the probability distribution model can perceive the scaling effect of the G-Unit process on the residual features, so that there is no obvious deviation between the probability distribution model and the residual features actually to be encoded, then, the target probability distribution model corresponding to the first scaling factor (that is, the first scaling factor adopted by the G-Unit process) can be obtained, that is, the probability distribution model is corrected based on the first scaling factor to obtain the corrected target probability distribution model, and Bitstream#2 is decoded based on the target probability distribution model.

[0142] For example, after obtaining Bitstream#1 corresponding to the current block, the decoding end can decode Bitstream#1 to obtain the hyperparameter quantization feature, dequantize the hyperparameter quantization feature, and obtain the initial coefficient hyperparameter feature z_hat. Alternatively, the decoding end can also decode Bitstream#1 to obtain the initial coefficient hyperparameter feature z_hat without involving the dequantization process.

[0143] After obtaining the initial coefficient hyperparameter feature z_hat, the decoder can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the mean mu corresponding to the current block.

[0144] After obtaining Bitstream#2 corresponding to the current block, the decoding end can also decode Bitstream#2 to obtain residual quantization features, dequantize the residual quantization features, and obtain the corrected residual features r'_hat. Alternatively, after obtaining Bitstream#2 corresponding to the current block, the decoding end can decode Bitstream#2 to obtain the corrected residual features r'_hat.

[0145] After obtaining the corrected residual feature r'_hat, the decoder can also correct the corrected residual feature r'_hat to obtain the residual feature r_hat before correction, that is, Figure 7 The residual inverse scaling process shown in .

[0146] After obtaining the residual feature r_hat, the decoder determines the image feature y_hat based on the residual feature r_hat and the mean mu, and performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current block x.

[0147] In a possible implementation, when decoding Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model. In order to make the probability distribution model match the G-Unit process, that is, the probability distribution model can perceive the scaling effect of the G-Unit process on the residual features, so that there is no obvious deviation between the probability distribution model and the residual features actually to be encoded, the probability distribution model PDM can be corrected to obtain a corrected probability distribution model PDM_hat, and the probability distribution model PDM_hat can be called a target probability distribution model. In order to make the probability distribution model match the G-Unit process, the probability distribution model PDM can be corrected using a first scaling factor (scaling factor of the G-Unit process) to obtain a corrected target probability distribution model PDM_hat.

[0148] In this embodiment, the input of the probability model correction module is mainly: the first scaling factor scale of the residual feature at the decoding end and the original probability distribution model confirmed by the decoding of Bitstream#1, and the output is the adjusted target probability distribution model PDM_hat. The main purpose of the probability model correction module is to optimize the probability distribution model of the entropy coding of Bitstream#2, and to make the bit rate of the coded bit stream smaller by adjusting the probability distribution parameters of the probability distribution model. For example, by correcting the probability distribution model PDM output by the super prior network, the probability distribution model is made to perceive all scaling changes of the residual feature at the decoding end, and the scaling factor of the residual feature and the probability distribution model decoded by Bitstream#2 are jointly passed into the probability model correction module, and the new target probability distribution model PDM_hat is output. The target probability distribution model PDM_hat is more accurate in describing the distribution of the residual feature than the previous probability distribution model, and the purpose of reducing the bit rate can be achieved.

[0149] Exemplarily, since the target probability distribution model PDM_hat is determined based on probability distribution parameters (such as mean and / or standard deviation), the target probability distribution parameters corresponding to the first scaling factor can be obtained, and the target probability distribution model PDM_hat corresponding to the first scaling factor can be determined based on the target probability distribution parameters. For example, the initial probability distribution parameters can be corrected using the first scaling factor to obtain the corrected target probability distribution parameters.

[0150] For example, the probability distribution model is related to the mean and standard deviation. If the mean of the probability distribution model is 0, it is only necessary to determine the standard deviation of the probability distribution model, and then determine the probability distribution model based on the standard deviation. In this case, the target standard deviation corresponding to the first scaling factor can be obtained. For example, the initial standard deviation can be corrected by the first scaling factor to obtain the corrected target standard deviation, and then the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target standard deviation.

[0151] For another example, if the mean of the probability distribution model is not 0, it is necessary to determine the mean and standard deviation of the probability distribution model, and then determine the probability distribution model based on the mean and standard deviation. In this case, the target mean and target standard deviation corresponding to the first scaling factor can be obtained. For example, the initial mean can be corrected by the first scaling factor to obtain the corrected target mean, and the initial standard deviation can be corrected by the first scaling factor to obtain the corrected target standard deviation. Then, the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target mean and the target standard deviation.

[0152] For another example, if the mean of the probability distribution model is not 0, but the standard deviation of the probability distribution model is 0, it is necessary to determine the mean of the probability distribution model, and then determine the probability distribution model based on the mean. In this case, the target mean corresponding to the first scaling factor can be obtained. For example, the initial mean can be corrected by the first scaling factor to obtain the corrected target mean, and then the target probability distribution model PDM_hat corresponding to the first scaling factor is determined based on the target mean.

[0153] Exemplarily, in order to obtain the mean of the probability distribution model, a context-based prediction can be performed based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the mean mu corresponding to the current block.

[0154] Exemplarily, in order to obtain the standard deviation of the probability distribution model, after obtaining the initial coefficient hyperparameter feature z_hat, the initial coefficient hyperparameter feature z_hat is subjected to an inverse transformation of the coefficient hyperparameter feature to obtain the standard deviation σ corresponding to the current block.

[0155] Exemplarily, the decoding end may obtain the target probability distribution parameters corresponding to the first scaling factor, and determine the target probability distribution model corresponding to the first scaling factor based on the target probability distribution parameters. For example, the decoding end may modify the initial probability distribution parameters based on the first scaling factor to obtain the target probability distribution parameters corresponding to the first scaling factor.

[0156] In a possible implementation, the target probability distribution parameters may include, but are not limited to, a target mean and / or a target standard deviation. Of course, the target mean and the target standard deviation are only examples, and no limitation is imposed on the target probability distribution parameters.

[0157] Exemplarily, if the target probability distribution parameter is the target standard deviation, then the decoding end can decode the first bitstream to obtain the initial coefficient hyperparameter feature corresponding to the current block, and perform an inverse transformation of the initial coefficient hyperparameter feature to obtain the initial standard deviation, and correct the initial standard deviation based on the first scaling factor to obtain the target standard deviation.

[0158] For example, the decoding end may generate a target scaling factor based on the first scaling factor, and correct the initial standard deviation based on the target scaling factor to obtain the target standard deviation, for example, taking the product of the target scaling factor and the initial standard deviation as the target standard deviation. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and the second scaling factor, such as determined based on the product of the first scaling factor and the second scaling factor, or, based on the product value of the first scaling factor and the second scaling factor, a certain offset value is added to obtain the target scaling factor.

[0159] Among them, the second scaling factor is used to represent the mapping relationship between the residual and the standard deviation. For example, the residual and the standard deviation are values ​​of different magnitudes. The magnitude of the standard deviation can be adjusted to the magnitude of the residual through the second scaling factor, that is, the second scaling factor is used as an adjustment coefficient to adjust the magnitude of the standard deviation to the magnitude of the residual, thereby adjusting the residual and the standard deviation to the same magnitude for calculation, so that the residual and the standard deviation can better match the calculation and improve the calculation accuracy.

[0160] The second scaling factor may be preconfigured; or, the second scaling factor may be obtained by decoding a code stream corresponding to the current block, such as obtaining the second scaling factor by decoding a first code stream corresponding to the current block, or obtaining the second scaling factor by decoding a second code stream corresponding to the current block, or obtaining the second scaling factor by decoding other code streams, without limitation.

[0161] Exemplarily, if the target probability distribution parameter is the target standard deviation, then the decoding end can decode the first bitstream to obtain the initial coefficient hyperparameter feature corresponding to the current block, and modify the initial coefficient hyperparameter feature based on the first scaling factor to obtain the target coefficient hyperparameter feature, and perform an inverse transformation of the coefficient hyperparameter feature on the target coefficient hyperparameter feature to obtain the target standard deviation.

[0162] For example, the decoding end can generate a target scaling factor based on the first scaling factor, and modify the initial coefficient hyperparameter feature based on the target scaling factor to obtain the target coefficient hyperparameter feature. For example, the decoding end can use the product of the target scaling factor and the initial coefficient hyperparameter feature as the target coefficient hyperparameter feature. The target scaling factor can be the first scaling factor; or, the target scaling factor can be determined based on the first scaling factor and the third scaling factor, for example, the target scaling factor can be determined based on the product of the first scaling factor and the third scaling factor, or, based on the product value of the first scaling factor and the third scaling factor, a certain bias value is added to obtain the target scaling factor. Among them, the third scaling factor is used to represent the mapping relationship between the residual and the coefficient hyperparameter features. For example, the residual and the coefficient hyperparameter features are values ​​of different magnitudes. The magnitude of the coefficient hyperparameter features can be adjusted to the magnitude of the residual through the third scaling factor. That is, the third scaling factor is used as an adjustment coefficient to adjust the magnitude of the coefficient hyperparameter features to the magnitude of the residual, thereby adjusting the residual and the coefficient hyperparameter features to the same magnitude for calculation, so that the residual and the coefficient hyperparameter features can better match the calculation and improve the calculation accuracy.

[0163] The third scaling factor may be preconfigured; or, the third scaling factor may be obtained by decoding a code stream corresponding to the current block, such as obtaining the third scaling factor by decoding a first code stream corresponding to the current block, or obtaining the third scaling factor by decoding a second code stream corresponding to the current block, or obtaining the third scaling factor by decoding other code streams, without limitation.

[0164] Exemplarily, if the target probability distribution parameter is the target mean, then the decoding end can perform context-based prediction based on the initial coefficient hyperparameter feature z_hat of the current block and the residual feature y_hat of the previous image block to obtain the initial mean corresponding to the current block, and the decoding end can correct the initial mean based on the first scaling factor to obtain the target mean.

[0165] For example, the decoding end may generate a target scaling factor based on the first scaling factor, and modify the initial mean value based on the target scaling factor to obtain the target mean value, for example, the product of the target scaling factor and the initial mean value is used as the target mean value. The target scaling factor may be the first scaling factor; or the target scaling factor may be determined based on the first scaling factor and the second scaling factor, such as determined based on the product of the first scaling factor and the second scaling factor, or, based on the product value of the first scaling factor and the second scaling factor, a certain bias value is added to obtain the target scaling factor.

[0166] Among them, the second scaling factor is used to represent the mapping relationship between the residual and the mean. For example, the residual and the mean are values ​​of different orders of magnitude. The order of magnitude of the mean can be adjusted to the order of magnitude of the residual through the second scaling factor. That is, the second scaling factor is used as an adjustment coefficient to adjust the order of magnitude of the mean to the order of magnitude of the residual, thereby adjusting the residual and the mean to the same order of magnitude for calculation, so that the residual and the mean can better match the calculation and improve the calculation accuracy.

[0167] The second scaling factor may be preconfigured; or, the second scaling factor may be obtained by decoding a code stream corresponding to the current block, such as obtaining the second scaling factor by decoding a first code stream corresponding to the current block, or obtaining the second scaling factor by decoding a second code stream corresponding to the current block, or obtaining the second scaling factor by decoding other code streams, without limitation.

[0168] Example 7: See Figure 8 As shown, it is a flowchart of the decoding method. The decoding end can receive Bitstream#1 (i.e., the first bitstream) and Bitstream#2 (i.e., the second bitstream). The decoding end can parse Bitstream#1 to obtain the initial coefficient hyperparameter feature z_hat. Based on the initial coefficient hyperparameter feature z_hat, the standard deviation of all feature points can be extracted, that is, the initial standard deviation σ of all feature points can be obtained. After the initial standard deviation is corrected by the probability model, the corrected target standard deviation σ_hat can be obtained.

[0169] The decoder can generate a target probability distribution model PDM_hat based on the target standard deviation, and decode Bitstream#2 based on the target probability distribution model PDM_hat to obtain the residual features corresponding to the current block. For example, in the entire decoding process, the default parameter of the target probability distribution model PDM_hat for each residual is σ_hat 2 is a distribution of variance, which is used for entropy decoding of Bitstream#2 to obtain the residual feature r_hat. Therefore, the decoding end can decode Bitstream#2 based on the target probability distribution model PDM_hat to obtain the residual feature r_hat corresponding to the current block.

[0170] The decoding end can parse the initial coefficient hyperparameter feature z_hat from Bitstream#1. Based on the initial coefficient hyperparameter feature z_hat, the current point mean can be extracted. The residual feature r_hat plus the mean can obtain the reconstructed value y_hat.

[0171] Embodiment 8: For the encoding end, the encoding method may include: determining a zero-mean Gaussian distribution The zero-mean Gaussian distribution can be the target probability distribution model, and the residual feature r_hat is obtained by using arithmetic encoding and decoding on the second bitstream (Bitstream#2). The encoder will adaptively learn a distribution representing the probability situation for each feature point based on the image content. Among them, the choice of the probability distribution model is confirmed as a hyperparameter before training, which can be a Gaussian distribution, a mixed Gaussian, etc. There is no restriction on the type of this probability distribution model. For example, it is assumed that each feature point has a zero mean and a variance of σ 2 Normal distribution, in order to estimate the distribution of each feature point, it is necessary to learn a parameter σ for each parameter, that is, the initial standard deviation, and the relevant information of the initial standard deviation will be written into Bitstream#1 in the super prior network. In this embodiment, the initial standard deviation can be synchronously corrected according to the first scaling factor. Therefore, when using the target probability distribution model to predict entropy encoding and decoding, it is also necessary to obtain the target standard deviation σ_hat through the probability model correction module. The target probability distribution model can be expressed as That is, the target probability distribution model is related to mean 0 and target standard deviation σ_hat.

[0172] For example, since the entropy model is not fixed, both the encoder and decoder need to know the probability distribution model parameter σ of each feature point, and the relevant information of σ needs to be transmitted to the decoder. Accurate modeling can reduce the bit rate of latent domain features.

[0173] Example 9: During the encoding process, the residual feature r will undergo various scaling processes, such as the G-unit scaling process, which will cause the distribution of the residual feature r to change. However, in the entire encoding and decoding framework, the input parameter of the generation network of the initial standard deviation σ is the original feature y in the latent domain. In this process, the initial standard deviation σ cannot perceive the change of the residual, resulting in a deviation between the distribution described by the initial standard deviation σ and the actual distribution of the residual. In order to make the initial standard deviation σ able to synchronously perceive the change of the residual, a probability model correction module is added after the initial standard deviation σ to map the change of the residual to the initial standard deviation σ. See Fig. 9A As shown, the input is the initial standard deviation σ and the first scaling factor scale (i.e., the first scaling factor of the G-unit process), and the output is the target standard deviation σ_hat. Obviously, the target standard deviation σ_hat can be determined based on the initial standard deviation σ and the first scaling factor scale, such as the product of the initial standard deviation σ and the first scaling factor scale is the target standard deviation σ_hat.

[0174] In Example 9, the mapping relationship adopts a 1:1 scaling relationship, that is, the first scaling factor of G-Unit is operated once on the initial standard deviation σ in the same way to generate the target standard deviation σ_hat, which can be seen from the following formula: σ_hat = Scale*σ. In this way, the target probability distribution mode can be used. The distribution of the prediction residuals. The first scaling factor Scale may be obtained through training or may be obtained in other ways without training, and there is no limitation on this.

[0175] Embodiment 10: Based on the scaling strategy of the initial standard deviation σ in Embodiment 9, in Embodiment 10, not only the numerical scaling of the residual feature res at the encoding end is considered, but also the difference in the physical meaning of the residual feature and the initial standard deviation σ itself is considered. For example, see Fig. 9B As shown, the input of the probability model correction module includes not only the initial standard deviation σ and the first scaling factor scale (i.e., the first scaling factor of the G-unit process), but also adds a second scaling factor Scale(σ, res) for describing the difference between the original physical meanings of the residual and the standard deviation. The second scaling factor Scale(σ, res) is used to represent the mapping relationship between the residual and the standard deviation. The second scaling factor may be pre-configured; or, the second scaling factor may be decoded from the bitstream corresponding to the current block, and the output is the target standard deviation σ_hat after scaling correction. Obviously, the target standard deviation σ_hat may be determined based on the initial standard deviation σ, the first scaling factor scale, and the second scaling factor Scale(σ, res). For example, the product of the initial standard deviation σ, the first scaling factor scale, and the second scaling factor Scale(σ, res) may be the target standard deviation σ_hat.

[0176] Since there are differences between the residual and the parameter σ of the normal distribution, 1:1 is not the best mapping relationship. Therefore, in order to more accurately perceive the change of the residual to the initial standard deviation σ, see the following formula: σ-hat = Scale(σ, res)*Scale*σ. In addition to being corrected by the first scaling factor scale, the initial standard deviation σ is also corrected by the second scaling factor Scale(σ, res).

[0177] Embodiment 11: In Embodiments 9 and 10, the initial standard deviation σ is directly corrected by the scaling factor scale. In Embodiment 11, the input z_hat of the super-scale decoding network used to generate the initial standard deviation σ is corrected, that is, the initial coefficient hyper-parameter feature z_hat is corrected to obtain the target coefficient hyper-parameter feature In the target coefficient hyperparameter feature After inputting to the super-scale decoding network, the target standard deviation σ_hat can be obtained. For example, in order to make the target standard deviation σ_hat synchronously perceive the transformation of the residual, a probability model correction module is added after the initial coefficient hyperparameter feature z_hat, aiming to map the change of the residual to the initial coefficient hyperparameter feature z_hat. The input is the initial coefficient hyperparameter feature z_hat and the first scaling factor scale (i.e., the first scaling factor of the G-unit process), and the output is the target coefficient hyperparameter feature In the target coefficient hyperparameter feature After inputting into the super-scale decoding network, the target standard deviation σ_hat can be obtained. Obviously, the target coefficient hyperparameter feature can be determined based on the initial coefficient hyperparameter feature z_hat and the first scaling factor scale For example, the product of the initial coefficient hyperparameter feature z_hat and the first scaling factor scale is the target coefficient hyperparameter feature Based on the target coefficient hyperparameter feature Determine the target standard deviation σ_hat.

[0178] In Example 11, the mapping relationship adopts a 1:1 scaling relationship, that is, the first scaling factor of G-Unit is operated once on the initial coefficient hyperparameter feature z_hat in the same way to generate the target coefficient hyperparameter feature The process can be shown in the following formula: In this way, the target probability distribution mode can be The distribution of prediction residuals.

[0179] Example 12: Based on the scaling strategy of the initial coefficient hyperparameter feature z_hat in Example 11, in Example 12, not only the numerical scaling of the residual feature res at the encoding end is considered, but also the difference in physical meaning between the residual feature and the initial coefficient hyperparameter feature z_hat itself is considered. For example, the input of the probability model correction module includes not only the initial coefficient hyperparameter feature z_hat and the first scaling factor scale (i.e., the first scaling factor of the G-unit process), but also adds a third scaling factor Scale(z_hat, res) for describing the difference in the original physical meanings between the residual and coefficient hyperparameter features. The third scaling factor Scale(z_hatres) is used to represent the mapping relationship between the residual and the coefficient hyperparameter features. The third scaling factor can be pre-configured; or, the third scaling factor can be decoded from the bitstream corresponding to the current block, and the output is the target coefficient hyperparameter feature after scaling and correction. Obviously, the target coefficient hyperparameter feature can be determined based on the initial coefficient hyperparameter feature z_hat, the first scaling factor scale and the third scaling factor Scale(z_hatres) For example, the product of z_hat, scale and Scale(z_hat, res) can be the target coefficient hyperparameter feature

[0180] Since the residual and coefficient hyperparameter features are different, 1:1 is not the best mapping relationship. Therefore, in order to more accurately perceive the change of residual to the coefficient hyperparameter feature, see the following formula: The initial coefficient hyperparameter feature z_hat is modified by the third scaling factor Scale(z_hatres) in addition to being modified by the first scaling factor scale.

[0181] Illustratively, the above-mentioned embodiments can be implemented individually or in combination. For example, each embodiment in Embodiment 1 to Embodiment 12 can be implemented individually, and at least two embodiments in Embodiment 1 to Embodiment 12 can be implemented in combination.

[0182] Illustratively, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way.

[0183] Based on the same application concept as the above method, a decoding device is also proposed in the embodiment of the present application. The device is applied to the decoding end, and the device includes: a memory, which is configured to store video data; a decoder, which is configured to implement the decoding method in the above embodiments 1 to 12, that is, the processing flow of the decoding end.

[0184] For example, in one possible implementation, the decoder is configured to implement:

[0185] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0186] Decoding a second bitstream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block;

[0187] A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

[0188] Based on the same application concept as the above method, an encoding device is also proposed in the embodiment of the present application. The device is applied to the encoding end, and the device includes: a memory, which is configured to store video data; an encoder, which is configured to implement the encoding method in the above embodiments 1 to 12, that is, the processing flow of the encoding end.

[0189] For example, in one possible implementation, the encoder is configured to implement:

[0190] Correct the residual features corresponding to the current block to obtain the corrected residual features;

[0191] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0192] The modified residual feature is encoded based on the target probability distribution model to obtain a second bitstream corresponding to the current block, and the second bitstream is sent to a decoding end.

[0193] Based on the same application concept as the above method, the decoding end device (also referred to as a video decoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig. 10A As shown. It includes: a processor 1001 and a machine-readable storage medium 1002, the machine-readable storage medium 1002 stores machine-executable instructions that can be executed by the processor 1001; the processor 1001 is used to execute the machine-executable instructions to implement the decoding method of the above-mentioned embodiments 1-12 of this application. For example, in a possible implementation, the decoding end device is used to implement:

[0194] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0195] Decoding a second bitstream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block;

[0196] A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

[0197] Based on the same application concept as the above method, the encoding end device (also referred to as a video encoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig. 10B As shown. It includes: a processor 1011 and a machine-readable storage medium 1012, the machine-readable storage medium 1012 stores machine-executable instructions that can be executed by the processor 1011; the processor 1011 is used to execute the machine-executable instructions to implement the encoding method of the above-mentioned embodiments 1-12 of the present application. For example, in a possible implementation, the encoding end device is used to implement:

[0198] Acquire a target probability distribution model corresponding to a first scaling factor based on a first bitstream corresponding to the current block;

[0199] Decoding a second bitstream corresponding to the current block based on a target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block;

[0200] A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

[0201] Based on the same application concept as the above method, an embodiment of the present application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the decoding method or encoding method of the above embodiments 1-12 of the present application.

[0202] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented, such as the decoding method or encoding method in the above embodiments.

[0203] Based on the same application concept as the above method, an embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the decoding method or encoding method disclosed in the above example of the present application.

[0204] Based on the same application concept as the above method, a decoding device is also proposed in an embodiment of the present application. The device is applied to a decoding end, and the device includes: an acquisition module, used to acquire a target probability distribution model corresponding to a first scaling factor based on a first code stream corresponding to a current block; a decoding module, used to decode a second code stream corresponding to the current block based on the target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block; and a determination module, used to determine a reconstructed image block corresponding to the current block based on the corrected residual feature.

[0205] Exemplarily, when the acquisition module acquires the target probability distribution model corresponding to the first scaling factor based on the first bitstream corresponding to the current block, it is specifically used to: acquire the target probability distribution parameters corresponding to the first scaling factor based on the first bitstream corresponding to the current block; and determine the target probability distribution model based on the target probability distribution parameters.

[0206] Exemplarily, the target probability distribution parameters include: a target mean and / or a target standard deviation.

[0207] Exemplarily, if the target probability distribution parameter is a target standard deviation, the acquisition module obtains the target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block, and is specifically used to: decode the first bitstream to obtain the initial coefficient hyper-parameter feature corresponding to the current block; perform an inverse transformation of the coefficient hyper-parameter feature on the initial coefficient hyper-parameter feature to obtain the initial standard deviation; and correct the initial standard deviation based on the first scaling factor to obtain the target standard deviation.

[0208] Exemplarily, when the acquisition module corrects the initial standard deviation based on the first scaling factor to obtain the target standard deviation, it is specifically used to: generate a target scaling factor based on the first scaling factor; and correct the initial standard deviation based on the target scaling factor to obtain the target standard deviation.

[0209] Exemplarily, the target scaling factor is the first scaling factor;

[0210] Alternatively, the target scaling factor is determined based on the first scaling factor and the second scaling factor;

[0211] The second scaling factor is used to represent the mapping relationship between the residual and the standard deviation.

[0212] Exemplarily, the second scaling factor is preconfigured; or,

[0213] The second scaling factor is obtained by decoding a code stream corresponding to the current block.

[0214] Exemplarily, if the target probability distribution parameter is a target standard deviation, the acquisition module obtains the target probability distribution parameter corresponding to the first scaling factor based on the first bitstream corresponding to the current block, and is specifically used to: decode the first bitstream to obtain the initial coefficient hyper-parameter feature corresponding to the current block; modify the initial coefficient hyper-parameter feature based on the first scaling factor to obtain the target coefficient hyper-parameter feature; and perform a coefficient hyper-parameter feature inverse transformation on the target coefficient hyper-parameter feature to obtain the target standard deviation.

[0215] Exemplarily, when the acquisition module modifies the initial coefficient hyperparameter feature based on the first scaling factor to obtain the target coefficient hyperparameter feature, it is specifically used to: generate a target scaling factor based on the first scaling factor; and modify the initial coefficient hyperparameter feature based on the target scaling factor to obtain the target coefficient hyperparameter feature.

[0216] Exemplarily, the target scaling factor is the first scaling factor; or,

[0217] The target scaling factor is determined based on the first scaling factor and a third scaling factor;

[0218] The third scaling factor is used to represent the mapping relationship between the residual and the coefficient hyperparameter features.

[0219] Exemplarily, the third scaling factor is preconfigured; or,

[0220] The third scaling factor is obtained by decoding the code stream corresponding to the current block.

[0221] Based on the same application concept as the above method, an encoding device is also proposed in an embodiment of the present application. The device is applied to the encoding end, and the device includes: a correction module, used to correct the residual features corresponding to the current block to obtain the corrected residual features; an acquisition module, used to obtain the target probability distribution model corresponding to the first scaling factor based on the first code stream corresponding to the current block; an encoding module, used to encode the corrected residual features based on the target probability distribution model to obtain the second code stream corresponding to the current block, and send the second code stream to the decoding end.

[0222] Exemplarily, the correction module corrects the residual feature corresponding to the current block to obtain the corrected residual feature, and is specifically used to: correct the residual feature corresponding to the current block based on the first scaling factor to obtain the corrected residual feature.

[0223] Exemplarily, when the acquisition module acquires the target probability distribution model corresponding to the first scaling factor based on the first bitstream corresponding to the current block, it is specifically used to: acquire the target probability distribution parameters corresponding to the first scaling factor based on the first bitstream corresponding to the current block; and determine the target probability distribution model based on the target probability distribution parameters.

[0224] Exemplarily, the encoding module is also used to: perform feature transformation on the current block to obtain image features corresponding to the current block; perform coefficient hyperparameter feature transformation on the image features to obtain initial coefficient hyperparameter features corresponding to the current block, and encode the initial coefficient hyperparameter features to obtain the first code stream.

[0225] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. An image decoding method, characterized in that: The method comprises: Decoding a bitstream corresponding to a current block to obtain an initial coefficient hyperparameter feature corresponding to the current block; performing a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature to obtain an initial standard deviation; correcting the initial standard deviation based on a first scaling factor to obtain a target standard deviation, and determining a target probability distribution parameter corresponding to the first scaling factor based on the target standard deviation; Determining a target probability distribution model based on the target probability distribution parameters; Decoding the bitstream corresponding to the current block based on the target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block; A reconstructed image block corresponding to the current block is determined based on the modified residual feature.

2. The method according to claim 1, characterized in that The target probability distribution parameters also include a target mean.

3. The method according to claim 2, characterized in that The target mean is 0.

4. The method according to claim 1, characterized in that: The correcting the initial standard deviation based on the first scaling factor to obtain a target standard deviation includes: A product value of the initial standard deviation and the first scaling factor is determined as the target standard deviation.

5. An image encoding method, characterized in that: The method comprises: Correct the residual features corresponding to the current block to obtain the corrected residual features; Acquire an initial coefficient hyperparameter feature; perform a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature to obtain an initial standard deviation; correct the initial standard deviation based on a first scaling factor to obtain a target standard deviation, and determine a target probability distribution parameter corresponding to the first scaling factor based on the target standard deviation; Determining a target probability distribution model based on the target probability distribution parameters; The modified residual feature is encoded based on the target probability distribution model to obtain a code stream corresponding to the current block, and the code stream is sent to a decoding end.

6. An image decoding device, characterized in that: The device comprises: An acquisition module is used to decode the bitstream corresponding to the current block to obtain the initial coefficient hyperparameter feature corresponding to the current block; perform coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature to obtain an initial standard deviation; correct the initial standard deviation based on a first scaling factor to obtain a target standard deviation, determine a target probability distribution parameter corresponding to the first scaling factor based on the target standard deviation; and determine a target probability distribution model based on the target probability distribution parameter; A decoding module, configured to decode a bitstream corresponding to the current block based on the target probability distribution model corresponding to the first scaling factor to obtain a corrected residual feature corresponding to the current block; A determination module is used to determine a reconstructed image block corresponding to the current block based on the corrected residual feature.

7. An image encoding device, characterized in that: The device comprises: A correction module, used for correcting the residual features corresponding to the current block to obtain the corrected residual features; An acquisition module is used to acquire an initial coefficient hyperparameter feature; perform a coefficient hyperparameter feature inverse transformation on the initial coefficient hyperparameter feature to obtain an initial standard deviation; correct the initial standard deviation based on a first scaling factor to obtain a target standard deviation, and determine a target probability distribution parameter corresponding to the first scaling factor based on the target standard deviation; and determine a target probability distribution model based on the target probability distribution parameter; The encoding module is used to encode the modified residual features based on the target probability distribution model to obtain a code stream corresponding to the current block, and send the code stream to a decoding end.

8. A decoding device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method according to any one of claims 1 to 4.

9. A coding end device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is configured to execute machine executable instructions to implement the method of claim 5.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by a processor, the method described in any one of claims 1 to 4 is implemented; or, when the computer instructions are executed by a processor, the method described in claim 5 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images