Decoding and coding method, device and equipment

Abstract

The invention provides a decoding method, a coding method, a decoding device, a coding device and equipment. The decoding method comprises the following steps: acquiring feature information of a current image block after compression; obtaining an overfitting parameter corresponding to the current image block; and obtaining a reconstructed image block corresponding to the current image block by using a decoder network based on the compressed feature information and the overfitting parameter. The method can improve coding and decoding performance.

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to a decoding and coding method, device and equipment thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. Complete video encoding can include prediction, transformation, quantization, entropy coding, filtering and other processes. For the prediction process, the prediction process can include intra-frame prediction and inter-frame prediction. Inter-frame prediction refers to the use of the correlation in the video time domain to use the pixels of the adjacent encoded images to predict the current pixel, so as to effectively remove the redundancy in the video time domain. Intra-frame prediction refers to the use of the correlation in the video space domain to use the pixels of the encoded blocks of the current frame image to predict the current pixel, so as to remove the redundancy in the video space domain.

[0003] With the rapid development of deep learning, deep learning has achieved success in many high-level computer vision problems, such as image classification and target detection. Deep learning has also gradually begun to be applied in the field of encoding and decoding, that is, neural networks can be used to encode and decode images. Although the encoding and decoding methods based on neural networks have shown great performance potential, they still have problems such as poor encoding performance, poor decoding performance and high complexity. Summary of the invention

[0004] In view of this, the present application provides a decoding and encoding method, device and equipment thereof to improve encoding performance and decoding performance.

[0005] According to a first aspect of an embodiment of the present application, a decoding method is provided, which is applied to a decoding end, and the method includes:

[0006] Obtaining feature information of the current image block after compression; and,

[0007] Obtaining an overfitting parameter corresponding to the current image block;

[0008] Based on the compressed feature information and the overfitting parameter, a decoder network is used to obtain a reconstructed image block corresponding to the current image block.

[0009] According to a second aspect of an embodiment of the present application, a coding method is provided, which is applied to a coding end, and the method includes:

[0010] Acquire compressed feature information of the current image block, and write the compressed feature information into a code stream corresponding to the current image block; and

[0011] An overfitting parameter corresponding to the current image block is obtained, and the overfitting parameter is written into a code stream corresponding to the current image block.

[0012] According to a third aspect of an embodiment of the present application, a decoding device is provided, including:

[0013] An acquisition unit, used to acquire feature information of a current image block after compression; and to acquire an overfitting parameter corresponding to the current image block;

[0014] A decoding unit is used to obtain a reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameter.

[0015] According to a fourth aspect of an embodiment of the present application, there is provided an encoding device, including:

[0016] An acquisition unit, used to acquire feature information of the current image block after compression;

[0017] An encoding unit, used for writing the compressed feature information into a code stream corresponding to the current image block;

[0018] The acquisition unit is further used to acquire an overfitting parameter corresponding to the current image block;

[0019] The encoding unit is further used to write the overfitting parameter into the code stream corresponding to the current image block.

[0020] According to a fifth aspect of an embodiment of the present application, there is provided a decoding end device, a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0021] The processor is used to execute machine executable instructions to implement the method provided by the first aspect.

[0022] According to a sixth aspect of an embodiment of the present application, there is provided an encoding end device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0023] The processor is used to execute machine executable instructions to implement the method provided by the second aspect.

[0024] According to the seventh aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method provided in the first aspect or the second aspect.

[0025] According to the eighth aspect of the embodiments of the present application, a machine-readable storage medium is provided, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method provided by the first aspect is implemented, or when the computer instructions are executed by a processor, the method provided by the second aspect is implemented.

[0026] It can be seen from the above technical solution that in the embodiment of the present application, by obtaining the compressed feature information of the current image block and obtaining the overfitting parameters corresponding to the current image block, the overfitting parameters can be used to make the output of the decoder network more suitable for the current image block, and then, based on the compressed feature information and the overfitting parameters, the decoder network is used to obtain the reconstructed image block corresponding to the current image block, and a better solution is provided for the current image block in the decoder network through online overfitting, thereby optimizing the image encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A is a schematic diagram of a three-dimensional feature matrix in one embodiment of the present application;

[0028] Figure 1B is a schematic diagram of an end-to-end encoding framework in one implementation of the present application;

[0029] Figure 2 It is a flowchart of a decoding method provided in an embodiment of the present application;

[0030] Figure 3 It is a flowchart of an encoding method provided in an embodiment of the present application;

[0031] Figure 4A It is a schematic diagram of the model structure of a high-speed synthetic transformation network in one embodiment of the present application;

[0032] Figure 4B It is a schematic diagram of the model structure of the base gear synthesis transformation network in one implementation of the present application;

[0033] Figure 5A is a schematic diagram of the structure of the ResAU layer in one embodiment of the present application;

[0034] Figure 5B is a schematic structural diagram of the RB layer in one embodiment of the present application;

[0035] Figure 5C is a schematic structural diagram of the RNAB layer in one embodiment of the present application;

[0036] Figure 6 It is a schematic diagram of various overfitting methods provided in the embodiments of the present application;

[0037] Figure 7It is a flowchart of a decoding method provided in an embodiment of the present application;

[0038] Figure 8 It is a flowchart of a decoding method provided in an embodiment of the present application;

[0039] Fig. 9 It is a flowchart of an encoding method provided in an embodiment of the present application;

[0040] Fig.10 is a schematic diagram of an overfitting model provided in an embodiment of the present application;

[0041] Fig.11 is a schematic diagram of an overfitting model provided in an embodiment of the present application;

[0042] Fig.12 is a schematic diagram of an overfitting model provided in an embodiment of the present application;

[0043] Fig.13 is a schematic diagram of an overfitting model provided in an embodiment of the present application;

[0044] Fig.14 is a schematic diagram of an overfitting model provided in an embodiment of the present application;

[0045] Fig.15 It is a schematic diagram of the hardware structure of a decoding end device provided in an embodiment of the present application;

[0046] Fig.16 It is a schematic diagram of the hardware structure of an encoding terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, depending on the context. In addition, the word "if" used can be interpreted as "at ... ", or "when ... ", or "in response to determination".

[0048] In the embodiments of the present application, a decoding method and an encoding method are proposed, which may involve the following concepts:

[0049] JPEG (Joint Photographic Experts Group): JPEG is a standard for continuous tone static image compression. The file extension can be .jpg or .jpeg. It is a commonly used image file format. JPEG uses a joint coding method of predictive coding (DPCM), discrete cosine transform (DCT) and entropy coding to remove redundant image and color data. It is a lossy compression format that can compress images in a very small storage space, which will cause damage to image data to a certain extent. In particular, when using too high a compression ratio, the quality of the image restored after the final decompression will be reduced. If you pursue high-quality images, JPEG should not use too high a compression ratio.

[0050] Overfitting: Also known as over-learning, its intuitive manifestation is that the algorithm performs well on the training set, but performs poorly on the test set, and has poor generalization performance. Overfitting is caused by the fact that the training data contains sampling errors during the model parameter fitting process, and the complex model during training also fits the sampling errors. The so-called sampling error refers to the deviation between the sample set obtained by sampling and the overall data set.

[0051] Entropy coding: Entropy coding is the coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information of the source (a measure of uncertainty). Entropy coding methods may include but are not limited to: Shannon coding, Huffman coding, and arithmetic coding.

[0052] Neural Network (NN): Neural network refers to artificial neural network. Neural network is a computing model composed of a large number of nodes (or neurons) connected to each other. In a neural network, neuron processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in a neural network can be divided into three categories: input units, output units, and hidden units. Input units receive signals and data from the outside world; output units realize the output of processing results; hidden units are units between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units, and the representation and processing of information are reflected in the connection relationship of processing units. Neural network is a non-programmed, brain-like information processing method. The essence of neural network is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of neural network, and imitate the information processing function of human brain nervous system to different degrees and levels. In the field of video processing, commonly used neural networks may include but are not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0053] Convolutional Neural Network (CNN): Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. The artificial neurons of convolutional neural network can respond to surrounding units within a certain coverage range and have excellent performance in large-scale image processing. The basic structure of convolutional neural network includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer and extracts the local features. Once the local feature is extracted, its positional relationship with other features is also determined. The second is the feature mapping layer (also called activation layer). Each computing layer of the neural network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since the neurons on a mapping surface share weights, the number of free parameters of the network is reduced.

[0054] For example, one of the advantages of convolutional neural networks over image processing algorithms is that they avoid the complex pre-processing of images (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of convolutional neural networks over ordinary neural networks is that ordinary neural networks use a fully connected approach, that is, all neurons from the input layer to the hidden layer are connected, which will result in a huge number of parameters, making network training time-consuming or even difficult to train, while convolutional neural networks avoid this difficulty through local connections, weight sharing and other methods.

[0055] Deconvolution layer: Deconvolution layer is also called transposed convolution layer. The working process of deconvolution layer and convolution layer is very similar. The main difference is that deconvolution layer will make the output larger than the input through padding (of course, it can also remain the same). If stride is 1, it means that the output size is equal to the input size; if stride is N, it means that the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0056] Generalization Ability: Generalization ability refers to the ability of a machine learning algorithm to adapt to fresh samples. The purpose of learning is to learn the rules hidden behind the data pairs. For data outside the learning set with the same rules, the trained network can also give appropriate outputs. This ability can be called generalization ability.

[0057] Feature: The feature involved in this application is a three-dimensional feature matrix of C*W*H, see Figure 1A As shown, it is a schematic diagram of a three-dimensional feature matrix. In the three-dimensional feature matrix, C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be the input of a neural network or the output of a neural network.

[0058] Feature Channel: In a neural network, the input signal can obtain many feature maps after passing through the convolution layer. Each feature map is called a feature channel and contains a part of the information of the input signal.

[0059] Rate-Distortion Optimized (RDO): There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode: J(mode) = D + λ*R, where D represents Distortion. It can usually be measured using the SSE indicator, which refers to the mean square sum of the difference between the reconstructed image block and the source image. In order to consider the cost, the SAD indicator can also be used. SAD refers to the sum of the absolute values ​​of the difference between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc. When selecting a mode, if the rate-distortion principle is used to make a comparison decision on the encoding mode, the best encoding performance can usually be guaranteed.

[0060] A large number of coding tools have been proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the coding tools that can achieve the best coding performance are often different. Therefore, in the encoding process, RDO is usually used to compare the coding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the marker information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to obtain the best coding performance. The decoding end can obtain relevant mode information by directly parsing the flag information, and the complexity impact is small.

[0061] The following is an example to illustrate the process of end-to-end image coding.

[0062] See also Figure 1B , is a schematic diagram of a framework of an end-to-end image encoding (also referred to as image compression) provided in an embodiment of the present application. Of course, Figure 1B This is just an example of an end-to-end image coding framework and does not limit the implementation of end-to-end image coding.

[0063] like Figure 1B As shown, the end-to-end image coding framework may include: analysis transformation network, synthesis transformation network, context-based prediction, quantization, entropy coding, entropy decoding, super coding network, super-scale decoding network and other modules. Among them:

[0064] Analysis transformation network: transforms the image block x to the feature y in the latent domain (from image domain to feature domain), so that all subsequent processes can be operated in the latent domain.

[0065] Exemplarily, an image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the encoding and decoding process for the image block can also be directly used for the image.

[0066] Super coding network: Input the probability distribution parameter σ of the residual r to obtain the super prior information z_hat, which will be written into the bitstream biasstream#1, mainly transmitting the probability model parameter σ of the residual.

[0067] Context-based prediction module: The input of this module includes not only z_hat but also the decoded y_hat. The two are combined to obtain a more accurate mean mu, which is the predicted value of the original feature. The reconstructed residual r_hat is added to obtain the reconstructed y_hat.

[0068] Super-scale decoding network: The output z_hat of the super-coding network is used as the input of the super-scale decoding network in another way to obtain σ. After quantization, σ_hat is obtained, and σ_hat will be used to obtain the quantized residual r_q in conjunction with the second bitstream (Bitstream#2).

[0069] Synthetic transformation module: The reconstructed latent domain feature y_hat is obtained and the reconstructed image block is obtained through the synthetic transformation module.

[0070] Entropy coding: A lossless coding method based on the principle of information entropy, which converts a series of element symbols used to represent a video sequence (such as transform coefficients and mode information) into a binary code stream, removing the statistical redundancy of these video element symbols.

[0071] Entropy decoding: Entropy decoding is the inverse process of entropy coding, which can convert the entropy coded bit stream back to the original symbol sequence.

[0072] Quantization: A "many-to-one" mapping process will cause signal loss. It acts on the residual r and can change the value range of the signal, so that the encoder can give a good approximation of the original signal with a small number of symbols, thereby improving the compression rate. In addition, quantization also acts on σ to obtain σ_hat, so that σ_hat can better estimate the probability distribution of the quantized residual r_q.

[0073] Dequantization: Dequantization is the inverse process of quantization, which is a process of restoring quantized data to its original continuous value.

[0074] based on Figure 1B The framework shown, the process of end-to-end image coding can be as follows:

[0075] After obtaining the current image block x (the current image block x may be the original image block x, i.e., the input image block), the encoder may analyze and transform the current image block x through an analysis transformation network (i.e., a neural network) to obtain an image feature y corresponding to the current image block x. Among them, performing feature transformation on the current image block x through the analysis transformation network means: transforming the current image block x to an image feature y in the latent domain, thereby facilitating all subsequent processes to operate on the latent domain.

[0076] When the encoding end obtains the image feature y, it can determine the probability distribution parameter σ of the image feature residual based on the image feature y.

[0077] On the one hand, the encoder can input the probability distribution parameter σ into the super coding network to obtain the super prior information z_hat, and write z_hat into the bitstream biasstream#1 through entropy coding.

[0078] Exemplarily, after obtaining Bitstream#1 corresponding to the current image block, the encoder may send Bitstream#1 corresponding to the current image block to the decoder. For the processing of Bitstream#1 corresponding to the current image block by the decoder, refer to the subsequent embodiments.

[0079] After obtaining the Bitstream#1 corresponding to the current image block, the encoder may also decode the Bitstream#1, such as by entropy decoding, to obtain the super-prior information z_hat.

[0080] After the encoder obtains the super-prior information z_hat, on the one hand, the super-prior information z_hat can be input into the super-scale decoding network to obtain the probability distribution parameter σ, and quantize σ to obtain σ_hat. On the other hand, the super-prior information z_hat can be input into the context-based prediction module, and the context-based prediction module is used to perform context-based prediction based on the super-prior information z_hat of the current image block and the reconstructed feature y_hat of the previous image block to obtain the predicted value (predicted value of image feature y) mu (i.e., mean mu) corresponding to the current image block.

[0081] On the other hand, the encoding end determines the residual r of the image feature based on the image feature y and the predicted value mu of the image feature y. For example, the difference between the image feature y and the predicted value of the image feature y is determined as the residual of the image feature.

[0082] The encoding end can quantize the residual r to obtain a quantized residual r_q, and perform entropy encoding on the quantized residual r_q based on σ_hat to obtain Bitstream#2 corresponding to the current image block.

[0083] Exemplarily, after obtaining Bitstream#2 corresponding to the current image block, the encoder may send Bitstream#2 corresponding to the current image block to the decoder. For the processing process of the decoder for Bitstream#2 corresponding to the current image block, refer to the subsequent embodiments.

[0084] After obtaining Bitstream#2 corresponding to the current image block, the encoder can also perform entropy decoding on Bitstream#2 based on σ_hat to obtain a quantized residual r_q, and perform inverse quantization to obtain an inverse quantized residual (which can be called a reconstructed residual) r_hat.

[0085] The encoder can obtain the reconstructed feature y_hat of the current image block based on the reconstructed residual r_hat and the predicted value mu of the image feature y, and input the reconstructed feature y_hat of the current image block into the synthesis transformation network to obtain the reconstructed image block corresponding to the current image block.

[0086] The decoding method and encoding method in the embodiments of the present application are described in detail below in combination with several specific embodiments.

[0087] In the embodiment of the present application, a decoding method is proposed, see Figure 2 As shown, it is a flowchart of the decoding method, which can be applied to a decoding end (also called a video decoder), and the method may include:

[0088] Step S200: Obtain feature information of the compressed current image block.

[0089] Step S210: Obtain overfitting parameters corresponding to the current image block.

[0090] In the embodiment of the present application, the model parameters obtained by training in the traditional solution are considered to be an "optimal solution" for the training set as a whole, but they are not necessarily the optimal solution for each training sample (such as an image) in the training set. For other types of images outside the training set, this is even less an "optimal solution".

[0091] Therefore, in order to improve the image encoding and decoding performance, when applying the trained model, the parameters of the trained model can be adjusted through online fitting to obtain a better solution.

[0092] Accordingly, the decoder network can be overfitted online at the encoding end, and the model parameters of the decoder network can be adjusted to obtain model parameters that are more suitable for the current image, that is, to obtain a better solution corresponding to the current image.

[0093] The encoder can write the overfitting parameters determined by online overfitting of the decoder network into the bitstream and send it to the decoding end.

[0094] When the decoding end is decoding the current image block, on the one hand, it can obtain the compressed feature information of the current image block, for example, by parsing the compressed feature information of the current image block from the bitstream corresponding to the current image block; on the other hand, it can obtain the overfitting parameters corresponding to the current image block, for example, by parsing the overfitting parameters corresponding to the current image block from the bitstream corresponding to the current image block.

[0095] Exemplarily, the compressed feature information may represent other information in the information sent by the encoder except for the overfitting parameters.

[0096] Step S220: Based on the compressed feature information and the overfitting parameters, a decoder network is used to obtain a reconstructed image block corresponding to the current image block.

[0097] In an embodiment of the present application, when the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can use the decoder network to obtain the reconstructed image block corresponding to the current image block based on the obtained compressed feature information and the overfitting parameters.

[0098] Exemplarily, the overfitting parameter can be used to achieve overfitting of the decoder network, providing a better solution for the current image block in the decoder network (ie, the decoder network can obtain a better processing result by processing the compressed feature information of the current image block).

[0099] For example, the decoding end can adjust the model parameters of the decoder network based on the overfitting parameters to obtain model parameters of the decoder network that are more suitable for the current image block, and based on the compressed feature information obtained, use the decoder network after the model parameters are adjusted to obtain the reconstructed image block corresponding to the current image block.

[0100] It can be seen that in Figure 2 In the method flow shown, by obtaining the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, the overfitting parameters can be used to make the output of the decoder network more suitable for the current image block. Then, based on the compressed feature information and the overfitting parameters, the decoder network is used to obtain the reconstructed image block corresponding to the current image block. A more optimal solution is provided for the current image block in the decoder network through online overfitting, thereby optimizing the image encoding and decoding performance.

[0101] In some embodiments, the decoder network includes a synthesis transformation network;

[0102] The above-mentioned obtaining of the feature information of the current image block after compression may include:

[0103] Obtaining super-prior information corresponding to the current image block, and obtaining residual probability distribution parameters corresponding to the current image block based on the super-prior information;

[0104] Based on the residual probability distribution parameters, the quantized residual corresponding to the current image block is obtained.

[0105] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0106] Based on the super prior information and the quantized residual, the reconstruction features corresponding to the current image block are obtained;

[0107] The overfitting parameters corresponding to the current image block are obtained, and based on the overfitting parameters and the reconstruction features, a synthetic transformation network is used to obtain a reconstructed image block corresponding to the current image block.

[0108] For example, the decoder network includes a synthetic transformation network. The synthetic transformation network can take the reconstructed features of the image block as input, perform synthetic transformation on the reconstructed features of the image block, and obtain a reconstructed image block corresponding to the image block.

[0109] Accordingly, when the decoder network includes a synthetic transformation network, the decoding end can obtain the super-prior information corresponding to the current image block, and obtain the residual probability distribution parameters corresponding to the current image block based on the obtained super-prior information.

[0110] In one example, the decoding end can parse the super-prior information corresponding to the current image block from the first bitstream corresponding to the current image block (such as Bitstream#1 mentioned above, which can be called an auxiliary bitstream), and use an ultra-scale decoding network to decode the super-prior information to obtain the residual probability distribution parameters corresponding to the current image block.

[0111] When the residual probability distribution parameters corresponding to the current image block are obtained at the decoding end, the quantization parameters corresponding to the current image block can be obtained based on the residual probability distribution parameters.

[0112] In an example, the decoding end may decode the second bitstream (such as the above-mentioned Bitstream#2) corresponding to the current image block based on the residual probability distribution parameter to obtain the quantized residual corresponding to the current image block.

[0113] The decoding end may obtain the reconstruction features corresponding to the current image block based on the super-prior information corresponding to the current image block and the quantization parameter corresponding to the current image block.

[0114] In one example, the decoding end may determine the reconstruction features corresponding to the current image block based on the super-prior information and the quantized residual corresponding to the current image block and using a context-based prediction model.

[0115] For example, the decoding end may perform context-based prediction based on the super-prior information of the current image block and the reconstructed features of the previous image block to obtain a prediction value corresponding to the current image block.

[0116] In addition, the decoding end may dequantize the quantized residual corresponding to the current image block to obtain the reconstructed residual corresponding to the current image block.

[0117] Furthermore, the decoding end can obtain the reconstruction features of the current image block based on the reconstruction residual and the prediction value corresponding to the current image block.

[0118] When the overfitting parameters corresponding to the current image block are obtained, the decoding end can obtain the reconstructed image block corresponding to the current image block by using the synthetic transformation network based on the overfitting parameters corresponding to the current image block and the reconstruction features.

[0119] For example, the decoding end may perform a synthetic transformation on the reconstruction features corresponding to the current image block based on the overfitting parameters corresponding to the current image block to obtain a reconstructed image block corresponding to the current image block.

[0120] In an example, the decoding end may parse the overfitting parameter corresponding to the current image block from the target bitstream corresponding to the current image block.

[0121] Exemplarily, the target code stream may include a first code stream, a second code stream, or a third code stream, that is, the encoder may write the overfitting parameters corresponding to the current image block into the first code stream or the second code stream corresponding to the current image block, or the encoder may write the overfitting parameters corresponding to the current image block into another code stream corresponding to the current image block (that is, the third code stream, used to transmit the overfitting parameters).

[0122] Different embodiments of overfitting parameters are described in detail below.

[0123] Example 1

[0124] The overfitting parameters may include target values ​​for weights of overfitting layers in the decoder network;

[0125] or,

[0126] The overfitting parameter may include a target value of a bias term of an overfitting layer in a decoder network;

[0127] or,

[0128] The overfitting parameters may include weight target values ​​and bias item target values ​​of the overfitting layers in the decoder network;

[0129] The weight target value may include: the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or, a weight offset determined based on the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the weight of the overfitting layer determined by offline training of the decoder network;

[0130] The bias item target value may include: the bias items of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or, the bias item offset determined by the bias item of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the bias item of the overfitting layer determined by offline training of the decoder network.

[0131] Exemplarily, in order to reduce the amount of data of overfitting parameters, the weights or bias items of the overfitting layer in the decoder network may be overfitted.

[0132] In one example, considering that the bias term in the convolutional layer is arguably the most important (more sensitive to a given instance) and is less common than the convolutional weights (i.e., weights), overfitting the bias term of the overfitting layer of the decoder network can reduce the amount of data for the overfitting parameters while ensuring the overfitting effect.

[0133] In another example, when the number of parameters in the original network layer is small, overfitting weight parameters, or overfitting bias items and weights may also be selected.

[0134] In one example, all network layers in the decoder network may be determined as overfitting layers to optimize the overfitting effect.

[0135] In another example, some network layers in the decoder network may be determined as overfitting layers to reduce the amount of data of overfitting parameters.

[0136] Exemplarily, the overfitting parameter may include a bias item target value of an overfitting layer in the decoder network, or, the overfitting parameter may include a weight target value of an overfitting layer in the decoder network, or, the overfitting parameter may include a bias item target value and a weight target value of an overfitting layer in the decoder network.

[0137] In one example, the weight target value may include: the weights of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block.

[0138] The bias item target value may include: bias items of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block.

[0139] That is, when the encoder determines the weights and / or bias items of the overfitting layer in the decoder network using an online overfitting method, the weights and / or bias items of the overfitting layer in the decoder network can be sent to the decoder as overfitting parameters.

[0140] In another example, the weight target value may include: a weight offset determined based on the weights of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the weight of the overfitting layer determined by offline training of the decoder network.

[0141] The bias item target value may include: the bias items of each overfitting layer determined based on online training of the overfitting layer in the decoder network using the current image block, and the bias item offset determined by the bias item of the overfitting layer determined by offline training of the decoder network.

[0142] That is, when the encoder uses an online overfitting method to determine the weights and / or bias items of the overfitting layers in the decoder network, it can determine the weight offset and / or bias item offset of each overfitting layer in the decoder network based on the determined weights and / or bias items of each overfitting layer in the decoder network, and the weights and / or bias items of the overfitting layers determined by offline training of the decoder network, and send the weight offset and / or bias item offset of each overfitting layer in the decoder network as overfitting parameters to the decoder.

[0143] Accordingly, the decoding end can assign weights and / or bias items of the overfitting layer in the decoder network based on the weight target value and / or bias item target value; input the compressed feature information into the assigned decoder network to determine the reconstructed image block corresponding to the current image block.

[0144] Exemplarily, when the encoder uses the current image block to perform online training on the overfitting layer in the decoder network, the encoder can determine the loss value based on the difference between the reconstructed image block output by the decoder network and the current image block, and based on the loss value, optimize and update the relevant parameters (weights and / or bias items) of the overfitting layer in the decoder network.

[0145] Exemplarily, the greater the difference between the reconstructed image block output by the decoder network and the current image block, the greater the loss value.

[0146] In one example, the decoder network includes a synthetic transformer network; and the overfitting layer in the decoder network includes an overfitting layer in the synthetic transformer network.

[0147] Accordingly, the overfitting parameter may include a weight target value of an overfitting layer in the synthetic transformation network;

[0148] or,

[0149] The overfitting parameters include the bias item values ​​of the overfitting layers in the synthetic transformer network;

[0150] or,

[0151] The overfitting parameters include the weight target value and the bias item target value of the overfitting layer in the synthetic transformation network;

[0152] The weight target value may include: the weight of each overfitting layer determined by online training of the overfitting layer in the synthesis transformation network using the current image block; or, a weight offset determined based on the weight of each overfitting layer determined by online training of the overfitting layer in the synthesis transformation network using the current image block and the weight of the overfitting layer determined by offline training of the synthesis transformation network;

[0153] The bias item target value may include: the bias items of each overfitting layer determined by online training of the overfitting layer in the synthesis transformation network using the current image block; or, the bias item offset determined by the bias item of each overfitting layer determined by online training of the overfitting layer in the synthesis transformation network using the current image block and the bias item of the overfitting layer determined by offline training of the synthesis transformation network.

[0154] Accordingly, the above-mentioned method of obtaining the reconstructed image block corresponding to the current image block by using the decoder network based on the compressed feature information and the overfitting parameter may include:

[0155] Based on the super-prior information and quantized residual corresponding to the current image block, obtaining the reconstruction feature corresponding to the current image block; and,

[0156] Assigning weights and / or bias items of an overfitting layer in a synthetic transformation network based on a weight target value and / or a bias item target value;

[0157] The reconstructed features are input into the valued synthetic transformation network to determine the reconstructed image block corresponding to the current image block.

[0158] Exemplarily, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include super-prior information corresponding to the current image block and a quantized residual corresponding to the current image block.

[0159] When the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, on the one hand, it can obtain the reconstructed features corresponding to the current image block based on the super prior information and the quantization residual.

[0160] On the other hand, the weights and / or bias items of the overfitting layers in the synthetic transformation network may be assigned based on the weight target value and / or the bias item target value.

[0161] For example, when the overfitting parameter may include a weight target value of an overfitting layer in the synthetic transformation network, the weight of the overfitting layer in the synthetic transformation network may be assigned based on the weight target value.

[0162] For another example, when the overfitting parameter may include a target value of a bias item of an overfitting layer in the synthetic transformation network, the bias item of the overfitting layer in the synthetic transformation network may be assigned a value based on the target value of the bias item.

[0163] For another example, when the overfitting parameters may include a weight target value and a bias item target value of an overfitting layer in a synthetic transformation network, the weight and bias item of the overfitting layer in the synthetic transformation network may be assigned based on the weight target value and the bias item target value.

[0164] Exemplarily, taking the assignment of bias items of overfitting layers in a synthetic transformation network as an example, when the bias item target value includes the bias items of each overfitting layer determined by online training of the overfitting layer in the synthetic transformation network using the current image block, the decoding end can set the value of the bias item of each overfitting layer in the synthetic transformation network to the corresponding bias item target value.

[0165] In the case where the bias item target value includes the bias items of each overfitting layer determined based on online training of the overfitting layer in the synthesis transformation network using the current image block, and the bias item offset determined by the bias item of the overfitting layer determined by offline training of the synthesis transformation network, the decoding end can determine the actual value of the bias item of each overfitting layer in the synthesis transformation network based on the bias item of the overfitting layer determined by offline training of the synthesis transformation network and the bias item target value of each overfitting layer, and assign values ​​to the bias items of each overfitting layer in the synthesis transformation network.

[0166] For example, assuming that for any overfitting layer, the bias item offset = the bias item of the overfitting layer determined by online training of each overfitting layer in the synthesis transformation network using the current image block - the bias item of the overfitting layer determined by offline training of the synthesis transformation network, then when the decoding end assigns a value to the bias item of the overfitting layer, for any overfitting layer, the actual value of the bias item = the bias item of the overfitting layer determined by offline training of the synthesis transformation network + the bias item offset of the overfitting layer.

[0167] It should be noted that in the embodiment of the present application, the online overfitting of the decoder network is not limited to being achieved through online overfitting of the synthetic transformation network, but can also be achieved by online overfitting of other networks or modules, for example, overfitting of the prediction module (such as a context-based prediction module). The specific implementation can refer to the relevant implementation of online overfitting of the synthetic transformation network, which will not be described in detail in the embodiment of the present application.

[0168] Example 2

[0169] The above-mentioned overfitting parameters may include target parameters of the first type of additional modules corresponding to each overfitting layer in the decoder network respectively, and the target parameters of the first type of additional modules are determined by the encoder embedding the first type of additional modules in parallel for each overfitting layer in the decoder network, and performing online training on relevant parameters of the first type of additional modules using the current image block;

[0170] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0171] In the decoder network, a first-type additional module is embedded in parallel for each overfitting layer, and relevant parameters of the first-type additional module are assigned values ​​based on target parameters of the first-type additional module;

[0172] The compressed feature information is input and embedded into the first type of additional module, and the decoder network after assignment determines the reconstructed image block corresponding to the current image block.

[0173] Exemplarily, online overfitting of the decoder network can also be achieved by embedding additional modules (referred to as first-type additional modules in this article) in parallel for each overfitting layer, and performing online training on the decoder network embedded with the first-type additional modules.

[0174] Exemplarily, the encoder may embed the first type of additional module in parallel for each overfitting layer in the decoder network, and use the current image block to perform online training on relevant parameters of the first type of additional module to obtain target parameters of the first type of additional module.

[0175] Exemplarily, during the online training of the relevant parameters of the first type of additional modules, the relevant parameters of the decoder network (such as weights, bias terms, etc.) can be kept unchanged, and the relevant parameters of the first type of additional modules embedded in parallel are trained.

[0176] Exemplarily, for any overfitting layer in which a first-type additional module is embedded in parallel, the output of the layer can be determined based on the output of the overfitting layer after processing the input and the output of the first-type additional module after processing the input. For example, it can be the sum of the two outputs.

[0177] Exemplarily, during the training process of the relevant parameters of the first type of additional module, a loss value can be determined based on the difference between the reconstructed image block output by the decoder network in which the first type of additional module is embedded and the current image block, and the relevant parameters of the first type of additional module are optimized and updated based on the loss value.

[0178] Exemplarily, the greater the difference between the reconstructed image block output by the decoder network embedded with the first type of additional module and the current image block, the greater the loss value.

[0179] Accordingly, when the decoding end obtains the feature information of the current image after compression and the overfitting parameters, the first type of additional modules can be embedded in parallel for each overfitting layer in the decoder network, and the relevant parameters of the first type of additional modules can be assigned based on the target parameters of the first type of additional modules; the compressed feature information is input into the decoder network embedded in the first type of additional modules and after the assignment, to determine the reconstructed image block corresponding to the current image block.

[0180] In one example, the decoder network may include a synthetic transformer network; and the overfitting layer in the decoder network includes an overfitting layer in the synthetic transformer network.

[0181] Exemplarily, when the decoder network includes a synthetic transformation network, a more optimal solution of the synthetic transformation network corresponding to the current image block can be found by performing online overfitting on the synthetic transformation network.

[0182] Exemplarily, the overfitting parameters may include target parameters of the first type of additional modules corresponding to each overfitting layer in the synthetic transformation network, respectively. The target parameters of the first type of additional modules are determined by the encoding end embedding the first type of additional modules in parallel for each overfitting layer in the synthetic transformation network, and performing online training on relevant parameters of the first type of additional modules using the current image block.

[0183] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0184] Based on the super-prior information and quantized residual corresponding to the current image block, obtaining the reconstruction feature corresponding to the current image block; and,

[0185] In the synthetic transformation network, a first-type additional module is embedded in parallel for each overfitting layer, and relevant parameters of the first-type additional module are assigned based on target parameters of the first-type additional module;

[0186] The reconstructed feature input is embedded into the first type of additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0187] Exemplarily, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include super-prior information corresponding to the current image block and a quantized residual corresponding to the current image block.

[0188] When the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, on the one hand, it can obtain the reconstructed features corresponding to the current image block based on the super prior information and the quantization residual.

[0189] On the other hand, the first type of additional modules may be embedded in parallel for each overfitting layer in the synthetic transformation network, and the relevant parameters of the first type of additional modules may be assigned based on the target parameters of the first type of additional modules.

[0190] The decoding end can embed the reconstruction feature input into the first type of additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0191] Example 3

[0192] The overfitting parameters include weight target values ​​of first-type overfitting layers in the decoder network, and target parameters of first-type additional modules corresponding to respective second-type overfitting layers in the decoder network;

[0193] or,

[0194] The overfitting parameters include target values ​​of bias items of the first type of overfitting layers in the decoder network, and target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network;

[0195] or,

[0196] The overfitting parameters include weight target values ​​and bias item target values ​​of the first type of overfitting layers in the decoder network, and target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network;

[0197] The weight target value includes: the weight of each first type of overfitting layer determined by online training of a decoder network embedded with the first type of additional module using the current image block; or, a weight offset determined based on the weight of each first type of overfitting layer determined by online training of a decoder network embedded with the first type of additional module using the current image block and the weight of the first type of overfitting layer determined by offline training of the decoder network;

[0198] The bias item target value includes: the bias item of each first type of overfitting layer determined by online training of a decoder network embedded in the first type of additional module using the current image block; or, based on the bias item of each first type of overfitting layer determined by online training of a decoder network embedded in the first type of additional module using the current image block, and the bias item offset determined by the bias item of the first type of overfitting layer determined by offline training of the decoder network;

[0199] The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and performing online training on the relevant parameters of the first type of additional modules using the current image block.

[0200] Exemplarily, for the overfitting layer in the decoder network, different methods can be used to implement online overfitting to improve the flexibility of overfitting implementation.

[0201] Exemplarily, for some overfitting layers (which may be referred to as first type overfitting layers), online overfitting may be achieved by online training of relevant parameters (weights and / or bias items) of the first type overfitting layers.

[0202] For another part of the overfitting layers (which may be referred to as the second type of overfitting layers), online overfitting may be achieved by embedding the first type of additional modules in parallel and performing online training on the relevant parameters of the first type of additional modules.

[0203] Exemplarily, for the first type of overfitting layer, its corresponding overfitting parameters can refer to the relevant description in Example 1.

[0204] Exemplarily, the encoding end can embed the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and use the current image block to perform online training on the relevant parameters of each first type of overfitting layer in the decoder network, as well as the relevant parameters of each first type of additional module, to determine the overfitting parameters (weight target values ​​and / or bias item target values) corresponding to the current image block for each first type of overfitting layer in the decoder network, as well as the overfitting parameters corresponding to each first type of additional module (target parameters of the first type of additional modules).

[0205] Exemplarily, during the online training of relevant parameters of the first type of overfitting layer and relevant parameters of the first type of additional module, a loss value can be determined based on the difference between the reconstructed image block output by the decoder network in which the first type of additional module is embedded and the current image block, and the relevant parameters of the first type of overfitting layer and relevant parameters of the first type of additional module are optimized and updated based on the loss value.

[0206] Exemplarily, the greater the difference between the reconstructed image block output by the decoder network embedded with the first type of additional module and the current image block, the greater the loss value.

[0207] Accordingly, when the decoding end obtains the compressed feature information of the current image block and the overfitting parameters, it can assign the weights and / or bias items of the first type of overfitting layer in the decoder network based on the weight target value and / or bias item target value included in the overfitting parameters; and, in the decoder network, embed the first type of additional modules in parallel for each second type of overfitting layer, and assign relevant parameters of the first type of additional modules based on the target parameters of the first type of additional modules, input the compressed feature information into the embedded first type of additional modules, and determine the reconstructed image block corresponding to the current image block in the decoder network after the assignment.

[0208] In one example, the decoder network includes a synthetic transform network; the first type of overfitting layer in the decoder network may include the first type of overfitting layer in the synthetic transform network, and the second type of overfitting layer in the decoder network includes the second type of overfitting layer in the synthetic transform network.

[0209] Exemplarily, when the decoder network includes a synthetic transformation network, a more optimal solution of the synthetic transformation network corresponding to the current image block can be found by performing online overfitting on the synthetic transformation network.

[0210] Exemplarily, the overfitting parameters include weight target values ​​of the first type of overfitting layers in the synthetic transformation network, and target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the synthetic transformation network;

[0211] or,

[0212] The overfitting parameters include target values ​​of bias items of the first type of overfitting layers in the synthetic transform network, and target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the synthetic transform network;

[0213] or,

[0214] The overfitting parameters include weight target values ​​and bias item target values ​​of the first type of overfitting layers in the synthetic transformation network, and target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the synthetic transformation network;

[0215] The weight target value includes: the weight of each first type of overfitting layer determined by online training of a synthetic transformation network embedded in the first type of additional module using the current image block; or, a weight offset determined based on the weight of each first type of overfitting layer determined by online training of a synthetic transformation network embedded in the first type of additional module using the current image block and the weight of the first type of overfitting layer determined by offline training of the synthetic transformation network;

[0216] The bias item target value includes: the bias item of each first type overfitting layer determined by online training of the synthesis transformation network embedded in the first type additional module using the current image block; or, based on the bias item of each first type overfitting layer determined by online training of the synthesis transformation network embedded in the first type additional module using the current image block, and the bias item offset determined by the bias item of the first type overfitting layer determined by offline training of the synthesis transformation network;

[0217] The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the synthetic transformation network, and performing online training on relevant parameters of the first type of additional modules using the current image block.

[0218] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0219] Based on the super-prior information and quantized residual corresponding to the current image block, obtaining the reconstruction feature corresponding to the current image block; and,

[0220] Based on the weight target value and / or the bias target value, weights and / or bias items of the first type of overfitting layer in the synthetic transformation network are assigned; and, in the synthetic transformation network, first type additional modules are embedded in parallel for each second type of overfitting layer, and relevant parameters of the first type additional modules are assigned based on the target parameters of the first type additional modules;

[0221] The reconstructed feature input is embedded into the first type of additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0222] Exemplarily, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include super-prior information corresponding to the current image block and a quantized residual corresponding to the current image block.

[0223] When the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, on the one hand, it can obtain the reconstructed features corresponding to the current image block based on the super prior information and the quantization residual.

[0224] On the other hand, the weights and / or bias items of the first type of overfitting layer in the synthetic transformation network can be assigned based on the weight target value and / or the bias item target value; and the first type of additional module can be embedded in parallel for each second type of overfitting layer in the synthetic transformation network, and the relevant parameters of the first type of additional module can be assigned based on the target parameters of the first type of additional module.

[0225] Furthermore, the decoding end can embed the reconstruction feature input into the first type of additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0226] In one example, the first type additional module may be a low-rank matrix.

[0227] Exemplarily, in order to reduce the amount of data of overfitting parameters and improve online training efficiency, the first type of additional module may be a low rank matrix. The target parameter of the first type of module may be a target parameter of a low rank matrix.

[0228] In one example, when a network (such as the above-mentioned decoder network or synthetic transformation network) includes different types of overfitting layers, such as the above-mentioned first type of overfitting layer and the second type of overfitting layer, the decoding end needs to obtain the overfitting layer type identifier corresponding to the current image block in the process of obtaining the reconstructed image block of the current image block.

[0229] Exemplarily, the overfitting layer type identifier is used to indicate that the overfitting layer is a first type overfitting layer or a second type overfitting layer.

[0230] Example 4

[0231] The overfitting parameters include target inputs of the second type additional modules corresponding to each overfitting layer in the decoder network respectively. The target inputs of the second type additional modules are determined by the encoder embedding the second type additional modules in parallel for each overfitting layer in the decoder network and performing online training on the inputs of the second type additional modules; the relevant parameters of the second type additional modules are determined through offline training.

[0232] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0233] embedding a second type of additional module in parallel for each overfitting layer in the decoder network;

[0234] The compressed feature information is used as the input of the overfitting layer in the decoder network, and the target input is used as the input of the second type of additional module. The decoder network embedded with the second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

[0235] Exemplarily, in order to reduce the amount of data of overfitting parameters, the online overfitting of the decoder network can be achieved by embedding the second type of additional modules in parallel for each overfitting layer in the decoder network, and performing online training on the input of the second type of additional modules, so as to find a better solution for the decoder corresponding to the current image block.

[0236] Exemplarily, the encoder can embed the second type of additional module in parallel for each overfitting layer in the decoder network, and use the current image block to perform online training on the input of the second type of additional module to obtain the target input of the second type of additional module, and use the target input as the overfitting parameter to send it to the decoder.

[0237] Exemplarily, during online training of the input of the second type of additional module, a loss value can be determined based on the difference between the reconstructed image block output by the decoder network in which the second type of additional module is embedded and the current image block, and the input of the second type of additional module can be optimized based on the loss value.

[0238] Exemplarily, the greater the difference between the reconstructed image block output by the decoder network embedded with the second type of additional module and the current image block, the greater the loss value.

[0239] Accordingly, when the decoding end obtains the compressed feature information corresponding to the current image block and the overfitting parameters, it can embed the second type of additional modules in parallel for each overfitting layer in the decoder network, use the compressed feature information as the input of the overfitting layer in the decoder network, and use the marked input as the input of the second type of additional module, and use the decoder network embedded with the second type of additional module to determine the reconstructed image block corresponding to the current image block.

[0240] In one example, the decoder network includes a synthetic transformer network; the overfitting layer in the decoder network may include an overfitting layer in the synthetic transformer network.

[0241] The overfitting parameters may include target inputs of second-type additional modules corresponding to each overfitting layer in the synthetic transformation network respectively. The target inputs of the second-type additional modules are determined by the encoding end embedding the second-type additional modules in parallel for each overfitting layer in the synthetic transformation network, and performing online training on the inputs of the second-type additional modules; the relevant parameters of the second-type additional modules are determined through offline training.

[0242] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0243] Based on the super-prior information and quantized residual corresponding to the current image block, obtaining the reconstruction feature corresponding to the current image block; and,

[0244] Embedding a second type of additional module in parallel for each overfitting layer in the synthetic transformation network;

[0245] The reconstructed features are used as inputs of the overfitting layer in the synthetic transformation network, and the target input is used as inputs of the second type of additional module. The synthetic transformation network embedded in the second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

[0246] Exemplarily, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include super-prior information corresponding to the current image block and a quantized residual corresponding to the current image block.

[0247] When the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, on the one hand, it can obtain the reconstructed features corresponding to the current image block based on the super prior information and the quantization residual.

[0248] On the other hand, the second type of additional modules can be embedded in parallel for each overfitting layer in the synthetic transformation network.

[0249] Furthermore, the decoding end can use the reconstructed features as the input of the overfitting layer in the synthetic transformation network, and use the target input as the input of the second type of additional module, and use the synthetic transformation network embedded in the second type of additional module to determine the reconstructed image block corresponding to the current image block.

[0250] In one example, in the above embodiment (any one of Embodiment 1 to Embodiment 4), the overfitting layer includes part of the network layers or all the network layers in the network.

[0251] Exemplarily, the overfitting layer in the above embodiment may include some network layers in the network to reduce the amount of data of overfitting parameters. Or, the overfitting layer in the above embodiment may include all network layers in the network to optimize the online overfitting effect.

[0252] As an example, in the case where the overfitting layer includes part of the network layers in the network, the above method may further include:

[0253] Obtain an overfitting layer identifier corresponding to the current image block; wherein the overfitting layer identifier is used to indicate an overfitting layer in the network;

[0254] The overfitting layer identifier includes the layer sequence number of the overfitting layer, or the number of overfitting layers N; wherein the number of overfitting layers N is used to indicate that the first N layer sequence numbers in the layer sequence number list determined offline are the layer sequence numbers of the overfitting layers.

[0255] Exemplarily, in the case where the overfitting layer includes a part of the network layers in the network, the encoder may also send the overfitting layer identifier corresponding to the current image block to the decoder.

[0256] When the decoding end obtains the reconstructed image block of the current image block, the overfitting layer in the network can be determined by obtaining the overfitting layer identifier corresponding to the current image block.

[0257] In one example, the overfitting layer identifier may include the layer sequence number of the overfitting layer, that is, the encoding end may send the layer sequence numbers of each overfitting layer corresponding to the current image block to the decoding end, and the decoding end may determine the overfitting layer in the network based on the layer sequence numbers of each overfitting layer corresponding to the current image block.

[0258] In another example, the overfitting layer identifier includes the number N of overfitting layers; wherein the number N of overfitting layers is used to indicate that the first N layer numbers in the layer number list determined offline are the layer numbers of overfitting layers.

[0259] Exemplarily, the encoding end and the decoding end may pre-negotiate a layer sequence number list, in which the layer sequence number list is generated by sorting the sequence numbers of each network layer in the network according to a certain rule.

[0260] For example, the layer sequence number list may be generated by arranging the layer sequence numbers of the network layers in the network in ascending order.

[0261] For another example, the layer sequence number list may be generated by arranging the energy of each network layer in the network in descending order.

[0262] Exemplarily, the energy of each network layer in the network layer can be determined by training each network layer in the network and calculating the change energy of the network layer, that is, the sum of the squares of the differences between the trained parameters and the original parameters.

[0263] Exemplarily, when the encoder and decoder can pre-negotiate a layer number list, the encoder can determine the network layers corresponding to the first N layer numbers in the layer number list as overfitting layers, and send the number N of overfitting layers to the decoder.

[0264] The decoding end can set the first N layer numbers in the layer number list as the layer numbers of the overfitting layers according to the number N of overfitting layers, thereby determining the overfitting layers in the network.

[0265] It should be noted that the overfitting layer in the network may also be a fixed layer determined offline, for example, determined in advance by negotiation between the encoding end and the decoding end. In this case, there is no need to additionally send the overfitting layer identifier.

[0266] Example 5

[0267] The overfitting parameters include a target input of the additional network. The target input of the additional network is determined by the encoder by online training of the input of the additional network based on the output of the decoder network and the output of the additional network. The relevant parameters of the additional network are determined by offline training.

[0268] The method of obtaining the reconstructed image block corresponding to the current image block by using a decoder network based on the compressed feature information and the overfitting parameter may include:

[0269] The compressed feature information is used as the input of the overfitting layer in the decoder network, and the target input is used as the input of the additional network. The output of the decoder network and the output of the additional network are used to determine the reconstructed image block corresponding to the current image block.

[0270] Exemplarily, online overfitting of the decoder network can be achieved by adding an additional network that is relatively independent of the decoding network and performing online training on the input of the additional network.

[0271] Exemplarily, the encoder can perform online training on the input of the additional network based on the output of the decoder network and the output of the additional network, determine the target input of the additional network, and send the target input as an overfitting parameter to the decoder.

[0272] Accordingly, when the decoding end obtains the compressed feature information of the current image block and the overfitting parameters, it can use the compressed feature information as the input of the overfitting layer in the decoder network, and use the target input as the input of the additional network, and use the output of the decoder network and the output of the additional network to determine the reconstructed image block corresponding to the current image block.

[0273] In one example, the decoder network includes a synthesis transform network.

[0274] The above-mentioned method based on the compressed feature information and the overfitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block may include:

[0275] Based on the super prior information and the quantized residual, the reconstruction features corresponding to the current image block are obtained;

[0276] The reconstructed features are used as inputs of the overfitting layer in the synthetic transformation network, and the target input is used as input of the additional network. The output of the synthetic transformation network and the output of the additional network are used to determine the reconstructed image block corresponding to the current image block.

[0277] Exemplarily, in the case where the decoder network includes a synthetic transformation network, the compressed feature information may include super-prior information corresponding to the current image block and a quantized residual corresponding to the current image block.

[0278] When the decoder obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can obtain the reconstructed features corresponding to the current image block based on the super prior information and the quantization residual.

[0279] The decoding end can use the reconstructed features as the input of the overfitting layer in the synthetic transformation network, and use the target input as the input of the additional network, and use the output of the synthetic transformation network and the output of the additional network to determine the reconstructed image block corresponding to the current image block.

[0280] In some embodiments, the decoding method provided in the embodiments of the present application may further include:

[0281] Obtain an overfitting scheme identifier corresponding to the current image block; wherein different overfitting scheme identifiers correspond to different overfitting schemes;

[0282] The above-mentioned obtaining of the overfitting parameter corresponding to the current image block may include:

[0283] Based on the overfitting scheme identifier, an overfitting parameter corresponding to the current image block is parsed from a target bitstream corresponding to the current image block; wherein the target bitstream includes a first bitstream, a second bitstream or a third bitstream.

[0284] Exemplarily, in the process of determining overfitting parameters, the encoding end can provide multiple candidate overfitting schemes, for example, at least two overfitting schemes in the above-mentioned embodiments 1 to 5. The encoding end can respectively determine the overfitting parameters corresponding to different candidate overfitting schemes, and select an optimal overfitting scheme, and send the identifier of the optimal overfitting scheme and the overfitting parameters corresponding to the overfitting scheme to the decoding end.

[0285] Accordingly, the decoding end can obtain the overfitting scheme identifier corresponding to the current image block, determine the overfitting scheme currently used based on the overfitting scheme identifier, and then, based on the overfitting scheme identifier, parse the overfitting parameters corresponding to the current image block from the target code stream corresponding to the current image block.

[0286] Exemplarily, the target code stream may include the first code stream, the second code stream or the third code stream.

[0287] Exemplarily, when the decoding end determines the overfitting scheme to be adopted and obtains the corresponding overfitting parameters, the specific implementation of obtaining the reconstructed image block corresponding to the current image block based on the obtained overfitting parameters can be found in the relevant description in the above embodiments.

[0288] In the embodiment of the present application, a coding method is proposed, see Figure 3 As shown, it is a schematic diagram of the process of the encoding method, which can be applied to an encoding end (also called a video encoder), and the method may include:

[0289] Step S300: Obtain compressed feature information of the current image block, and write the compressed feature information into a code stream corresponding to the current image block.

[0290] In the embodiment of the present application, during the process of encoding the current image block, the encoding end may obtain compressed feature information of the current image block, and write the compressed feature information into a code stream corresponding to the current image block.

[0291] Exemplarily, the encoding end may send the code stream to the decoding end, and the decoding end performs image decoding processing according to the method described in the above embodiment.

[0292] In one example, the encoding end can obtain the probability distribution parameters of the residual of the current image block. On the one hand, based on the probability distribution parameters, it can obtain the super prior information corresponding to the current image block and write the super prior information into the bitstream; on the other hand, the encoding end can quantize the probability distribution parameters and write the quantized residual into the bitstream based on the quantized probability distribution parameters.

[0293] Step S310: Obtain overfitting parameters corresponding to the current image block, and write the overfitting parameters into a bitstream corresponding to the current image block.

[0294] In an embodiment of the present application, in order to improve the image encoding and decoding performance, when the trained model is applied, the parameters of the trained model can be adjusted through an online fitting method to obtain a better solution.

[0295] Accordingly, the decoder network can be overfitted online at the encoding end, and the model parameters of the decoder network can be adjusted to obtain model parameters that are more suitable for the current image, that is, to obtain a better solution corresponding to the current image.

[0296] The encoder can write the overfitting parameters determined by online overfitting of the decoder network into the bitstream and send it to the decoder.

[0297] Exemplarily, the specific implementation of determining the overfitting parameter at the encoding end can refer to the relevant description in the above embodiment, and the embodiments of the present application will not be described in detail here.

[0298] It should be noted that since the structure of the encoder network includes the structure of the decoder network, the encoding end will also perform the above-mentioned "based on the compressed feature information and the over-fitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block" during the image encoding process. The specific implementation process can refer to the relevant description in the method embodiment of the above-mentioned decoding end side.

[0299] In order to enable those skilled in the art to better understand the technical solutions provided in the embodiments of the present application, the technical solutions provided in the embodiments of the present application are described below with reference to specific examples.

[0300] The decoder network includes a synthetic transformation network, and an online overfitting method is performed on the synthetic transformation network to find a better solution corresponding to the current image block for the synthetic transformation network.

[0301] The model structure of the synthetic transformation network is first explained below.

[0302] See also Figure 4A and Figure 4B According to the complexity of the model structure, the model structures of the two commonly used synthetic transformation networks can be respectively as follows Figure 4A and Figure 4B shown.

[0303] like Figure 4A As shown in FIG. 1 , it is a schematic diagram of the model structure of the high-speed synthetic transformation network. For the high-speed synthetic transformation network, it includes ResBlock layer (i.e., RB layer), ResBlock layer, Conv-1 layer, Crop layer, ResAU layer, Conv-1 layer, Crop layer, ResAU layer, Conv layer, PixelShuffle layer, RNAB layer, Crop layer, ResAU layer, Conv-1 layer, and Crop layer. Among them, the ResBlock layer is a residual block. The structural diagram of the ResBlock layer can be found in Figure 5B As shown. Conv -1 The layer is a deconvolution layer, which is used to upsample the input data or restore the original size. (3, 3, C, C) means that the height and width of the convolution kernel are 3 pixels. The first C represents the number of channels of the input feature map, and the second C represents the number of channels of the output feature map. The upward arrow of 2 indicates that the feature is upsampled by 2 times. The Crop layer is a cropping layer, which is used to implement the cropping operation of the feature. (H in , W in , 2, 2) in H in and W in The height and width of the input feature map are shown in Figure 2. The last two 2s indicate that the cropped area is 2*2. The ResAU layer is an activation layer. The structure diagram of the ResAU layer can be found in Figure 5AAs shown. In (3, 3, noTAHN), the first 3 indicates that the number of channels of the input feature map is 3, the second 3 indicates that the size of the convolution kernel is 3*3, and noTAHN indicates that the TAHN (Temporal Adaptive Hard Negative Mining) function is not used in the attention module. The Conv layer is a convolution layer, which is used to implement the convolution operation of the feature. The PixelShuffle layer is a pixel shuffling layer, which implements upsampling by rearranging channels and reconstructing at the pixel level. In (4C, C), 4C indicates that the number of channels of the input feature map is 4C, C indicates that the number of channels of the output feature map is C, and the upward arrow of 2 indicates that the feature is upsampled by a factor of 2. The RNAB layer is a residual non-local attention block. The structural diagram of the RNAB layer can be seen in Figure 5C As shown, the RNAB layer may include a RB layer, a Conv layer, and a sigmoid layer.

[0304] like Figure 4B As shown in the figure, it is a schematic diagram of the model structure of the base gear synthesis transformation network. For the base gear synthesis transformation network, it includes the LighResBlock layer, Conv -1 Layer, Crop layer, ResAU layer, Conv -1 Layer, Crop layer, ResAU layer, Conv layer, ResAU layer, Conv layer, PixelShuffle layer, Crop layer. Among them, LighResBlock is a lightweight residual block, and 3×3 means that the size of the convolution kernel is 3x3.

[0305] See also Figure 5A As shown, the ResAU layer is an activation layer. The structural diagram of the ResAU layer can be found in Figure 5A As shown in the figure, LeakyReLU represents the activation operation, Conv 1*1 represents the convolution operation using a 1*1 convolution kernel to implement the feature, and tanh represents the hyperbolic tangent operation.

[0306] See also Figure 5B , the RB (Residual Blocks) layer is a residual block, and the structural diagram of the RB layer can be shown as Figure 5B As shown, the RB layer may include a Conv layer, a LeakyReLU and a Conv layer, and a 3*3 convolution kernel is used in the Conv layer.

[0307] See also Figure 5C , the RNAB layer is a residual non-local attention block, and the structural diagram of the RNAB layer can be shown as Figure 5CAs shown, the RNAB layer may include a RB layer, a Conv layer, and a sigmoid layer. In the Conv layer, the upward arrow of 2 indicates that the feature is upsampled by a factor of 2. The sigmoid layer is used to perform feature mapping on the feature, such as using a sigmoid function to map the feature to between 0 and 1.

[0308] In order to ensure consistency between encoding and decoding, the "better solution" after overfitting on the encoding end needs to be written into the bitstream and passed to the decoding end.

[0309] Exemplarily, the overfitting methods at the encoding end may include four categories:

[0310] The first type is to relearn the synthesis transformation module online without adding any additional modules, and transfer the "better solution" (weight W1 and / or bias item B1) of the parameters (weight W and / or bias item B) in the synthesis transformation module to the decoding end, and obtain the "better solution" of the final result through the "better solution" of the parameters;

[0311] The second type is to embed an additional module (such as the first type of additional module mentioned above) in parallel in the current synthesis transformation module, train the additional module online, transfer the relevant parameters (weight W1) of the additional module to the decoding end, and add the learned "solution" (x1) of the additional module to the solution of the original synthesis transformation module to obtain a "better solution" of the final result;

[0312] The third type is to embed additional modules (such as the second type of additional modules mentioned above) in parallel in the current synthetic transformation module. The additional modules are trained offline, and the input of the network is trained online. The input is passed to change the output of the synthetic transformation network, and the "solution" (x1) of the learned additional module is added to the solution of the original synthetic transformation module to obtain a "better solution" for the final result.

[0313] The fourth type is to add an additional network, train the additional network offline, train the input of the additional network online, pass the input, thereby changing the output of the synthetic transformation module, and adding the learned "solution" (x1) of the additional network to the solution of the original synthetic transformation module to obtain a "better solution" for the final result.

[0314] Schematic diagrams of various overfitting methods can be shown as Figure 6 As shown, among them, the fourth category.

[0315] The implementation processes of the decoding end and the encoding end are described below respectively.

[0316] Example 6

[0317] A decoding method flow may include:

[0318] like Figure 7As shown, the decoding end will receive one or two code streams (ie, Bitstream#1; or, Bitstream#1 and Bitstream#2).

[0319] Step 1, parse the Bitstream#1 code stream to obtain the feature information of the compressed image;

[0320] Step 2, decoding from Bitstream#1 or Bitstream#2 to obtain overfitting parameters;

[0321] Step 3: Input the compressed feature information and overfitting parameters into the decoder network to obtain the reconstructed image.

[0322] Example 7

[0323] Taking the above decoder network including a synthetic transformation network as an example, a decoding method flow may include:

[0324] like Figure 8 As shown, the decoding end will receive two or three code streams (ie, Bitstream#1 and Bitstream#2; or, Bitstream#1, Bitstream#2 and Bitstream3).

[0325] Step 1, parse and obtain z_hat from Bitstream#1 (auxiliary bitstream), and obtain the parameters of the probability distribution model of the residual (ie, probability distribution parameters) σ through z_hat.

[0326] Step 2: Perform entropy decoding on Bitsstream#2 and combine it with σ to obtain the quantized residual r_q.

[0327] Step 3: The quantized residual r_q and z_hat are passed through a context-based prediction module to obtain the reconstructed feature y_hat.

[0328] Step 4, decode the overfitting parameters from Bitsstream#1, Bitsstream#2 or Bitsstream#3, and use the overfitting parameters to synthesize the transformation module, the additional module or the additional network, and input the reconstruction feature y_hat to obtain the reconstructed image x_hat.

[0329] Embodiment 8,

[0330] Taking the above decoder network including a synthetic transformation network as an example, an encoding method may include:

[0331] like Fig. 9As shown, the encoder can obtain the probability distribution parameter σ of the residual of the current image block. On the one hand, based on the probability distribution parameter σ, the super prior information z_hat corresponding to the current image block is obtained and z_hat is written into Bitstream#1. On the other hand, the encoder can quantize the probability distribution parameter and write the quantized residual r_q into the bitstream Bitstream#2 based on the quantized probability distribution parameter. The overfitting parameter acts on the synthesis transformation module, additional module or additional network, and is written into Bitstream#1 or Bitstream#2, or Bitstream#3.

[0332] The overfitting solutions at the encoding end and the decoding end are described below in conjunction with the embodiments.

[0333] Example 9: Encoding end overfitting solution 1

[0334] Overfit the bias term (B) of the current synthetic transformation network. Since the bias term in the convolutional layer can be said to be the most important (more sensitive to a given instance) and is less than the convolution weight (i.e., weight), the bias term can be overfitted. If the number of network layers is too large (such as the number of network layers exceeds the preset layer threshold), the bit rate of overfitting the bias terms of all network layers will also be unbearable. Therefore, you can choose to overfit the bias terms of some layers (i.e., overfitting layers).

[0335] For example, there are two ways to select the overfitting layer:

[0336] First, the first layer of the network is mainly related to more general features, and the last layer is related to details. Therefore, in order to improve the reconstruction details, overfitting is limited to the bias terms of the last L convolutional layers.

[0337] Second, since some synthetic transformation networks are more complex and have jump connections, some of the previous network layers may still be related to details. By training all network layers of the synthetic transformation network, the change energy of the network layer is calculated, that is, the sum of the squares of the differences between the parameters of the trained network layer and the original parameters. The layers with larger energy are the selected L overfitting convolutional layers.

[0338] Exemplarily, when L is selected, the overfitting layer is determined accordingly. In order to reduce the computational complexity, the overfitting model (NET) is the first layer in the selected L-layer network layer to the last layer of the synthetic transformation network, such as Fig.10As shown, then use the Adam (Adaptive Moment Estimation) optimizer in PyTorch (an open-source machine learning library based on Python), set the learning rate to 0.003, and perform 100 online iterations. If better coding performance than the original network parameters is obtained (i.e., smaller loss, where lambda = 5, 16, 35, 110, 220, corresponding to the choices when bpp = 0.06, 0.12, 0.25, 0.50, 0.75 respectively), then write the difference of the bias terms of the L layers learned online into the bitstream and transfer it to the decoding end. The specific online overfitting algorithm is shown in Algorithm 1.

[0339]

[0340] Among them, mssim(X_hat, X) is to compare the structural similarity between X_hat and X (i.e., the reconstructed image block and the original image block). mssim (Mean Structural Similarity Index) is a metric for measuring image similarity and is used to compare the structural similarity between two images.

[0341] loss < loss_best: loss_best = loss, B_best = B, F = 1 means that when the loss calculated during the iteration is less than the current best loss (i.e., loss_best, with the initial value being the original loss), update the current best loss to the loss calculated during this iteration, update the bias terms of the overfitting layer to the bias terms determined during this iteration, and set the value of the algorithm start flag F to 1 (i.e., adopt the online overfitting algorithm).

[0342] It should be noted that when the value of the algorithm start flag F is 0, it can be determined that the online overfitting algorithm is not adopted. In this case, an offline-trained synthesis transformation network can be used to generate the reconstructed image block, and there is no need to use overfitting parameters (nor transfer overfitting parameters).

[0343] Example 10. Overfitting Scheme 2 at the Encoding End

[0344] Add an additional module to achieve overfitting. Use a low-rank matrix to assist the original synthesis transformation network to achieve overfitting, that is, add a low-rank matrix in parallel for each selected overfitting layer. By online learning the low-rank matrix, change the output of the overfitting layer in the original synthesis transformation network, and thus achieve the overfitting of the entire synthesis transformation network.

[0345] Exemplarily, the selection method of the overfitting layer is the same as that of Scheme 1, except that the parameters that need to be written into the bitstream are the parameters of the low-rank matrix, and the relevant parameters of each network layer in the synthetic transformation network remain unchanged.

[0346] Exemplarily, the schematic diagram of the overfitting model NET of this scheme is as follows Fig.11 As shown, the specific online overfitting algorithm is shown in Algorithm 2.

[0347]

[0348] Embodiment 11, encoding end overfitting solution 3

[0349] Add additional modules to achieve overfitting, and use parallel small networks to achieve overfitting. The parallel small network needs to be trained offline. The purpose of this network is to assist the original synthetic network to achieve overfitting by learning the input of the network online (less data). The parallel network is different from the original network in each layer.

[0350] Exemplarily, this solution may not need to select an overfitting layer, and all layers have a parallel layer that needs to be trained offline in advance.

[0351] Exemplarily, the schematic diagram of the overfitting model NET of this scheme is as follows Fig.12 As shown, the specific online overfitting algorithm is shown in Algorithm 3.

[0352]

[0353] Embodiment 12: Overfitting Solution 4 at the Encoding End

[0354] An additional network is added to achieve overfitting, wherein the additional network needs to be trained offline. The purpose of the additional network is to assist the original synthetic network in achieving overfitting by learning the input of the network online (less data).

[0355] Exemplarily, the schematic diagram of the overfitting model NET of this scheme is as follows Fig.13 As shown, the specific online overfitting algorithm is similar to Algorithm 3.

[0356] It should be noted that the number of network layers of the additional network can be the same as or different from the number of network layers of the original synthetic transformation network ( Fig.13 Take the same as an example).

[0357] Example 13: Overfitting Solution 5 at the Encoding End

[0358] This solution is a combination of Solution 1 and Solution 2, that is, some layers (such as the first type of overfitting layer mentioned above) use the bias item of the overfitting original network layer, and some layers (such as the second type of overfitting layer mentioned above) use additional modules of low-rank matrices to assist overfitting. The schematic diagram of the overfitting model NET of this solution is as follows: Fig.14 shown.

[0359] Embodiment 14, encoding end overfitting solution 6

[0360] This scheme is a selection of multiple schemes, that is, schemes 1-5 (that is, candidate overfitting schemes include schemes 1-5) are tried at the encoding end, and the final scheme is determined based on the comparison of the loss size of their common loss function.

[0361] Exemplarily, the encoding end may send the overfitting scheme identifier and the corresponding overfitting parameters corresponding to the final scheme to the decoding end.

[0362] It should be noted that in the embodiments of the present application, for any overfitting scheme, there may be multiple different combinations. For example, for scheme 1, there are multiple options for the overfitting layer, and various options can be determined by full traversal, but the decoding end method of this combination is consistent.

[0363] Embodiment 15, decoding end overfitting solution 1 (corresponding to encoding end overfitting solution 1)

[0364] According to different methods, the overfitting layer in the original network layer is selected, and the parameters in the overfitting layer (i.e., the overfitting parameters) are transferred. You can choose to transfer the bias item B (i.e., the above-mentioned bias item target value) or the weight W (i.e., the above-mentioned weight target value), or you can choose to transfer W and B at the same time.

[0365] Encode the required information into the code stream, and the relevant information syntax includes:

[0366] 1. Identification of overfitting layers (choose one of the following methods)

[0367] Method 1: Directly pass the layer number list layer_list of the overfitting layer.

[0368] Example: layer_list = [9, 10], that is, the layer numbers of the overfitting layers are 9 and 10 respectively.

[0369] Method 2: When the synthetic transformation network is fixed, a layer number list layer_list is determined offline. The determination method can refer to the relevant description in the encoding end overfitting scheme 1. It can be arranged in sequence or by energy. At this time, the number of layers layer_num can be passed, and the codec can determine the overfitting layer according to layer_num.

[0370] Example: layer_list = [3, 5, 4, 2, 1, 0], layernum = 3, then the overfitting layer is [3, 5, 4].

[0371] 2. Overfitting parameters (choose one of the following methods)

[0372] Method 1: Transfer the weight W, or bias term B, or weight W and bias term B of each overfitting layer, and directly assign values ​​according to the overfitting parameters during decoding.

[0373] Method 2: Transfer the difference of the correlation coefficients of each overfitting layer (i.e., the above-mentioned bias offset and / or weight offset) W_offset, or, B_offset, or, W_offset and B_offset. When decoding, use the original parameters of the synthetic transformation network (properly trained offline) plus the difference obtained by decoding to determine the final network parameters.

[0374] 3. Other optional parameters. If both the encoding and decoding ends confirm the rules in advance, these parameters can be omitted.

[0375] Parameter 1: If entropy coding is used to encode the overfitting parameters of each layer, it is necessary to pass the distribution coefficient list sigma_list of the parameters of each layer or a distribution coefficient sigma of all parameters;

[0376] Parameter 2: num_list, the number list of parameters for each layer;

[0377] Parameter 3: Identifiers of weight coefficients and bias coefficients.

[0378] Embodiment 16, decoding end overfitting solution 2 (corresponding to encoding end overfitting solution 2)

[0379] Select the overfitting layer in the original network layer according to different methods, and transfer the parameters of the low-rank matrix. Encode the required information into the bitstream, and the relevant information syntax includes:

[0380] 1. Identification of overfitted layers (same as Example 15).

[0381] 2. Related parameters of the low-rank matrix. For example, pass the low-rank matrices B and A.

[0382] 3. Other optional parameters. If both the encoding and decoding ends confirm the rules in advance, these parameters can be omitted:

[0383] Parameter 1: rank list of low-rank matrices rank_list. If all layers use the same rank, only one rank needs to be passed.

[0384] Parameter 2: If entropy coding is used to encode the low-rank matrices of each layer, it is necessary to pass the distribution coefficient list sigma_list of the parameters of each layer or a distribution coefficient sigma of all parameters;

[0385] Parameter 3: Pass the number list num_list of parameters for each layer.

[0386] Embodiment 17, decoding end overfitting solution 3 (corresponding to encoding end overfitting solution 3)

[0387] Pass the input parameters of the additional module (such as the target input above). Encode the required information into the bitstream, and the relevant information syntax includes:

[0388] 1. Input related parameters: pass y2.

[0389] 2. Other optional parameters. If both the encoding and decoding ends confirm the rules in advance, these parameters can be omitted:

[0390] Parameter 1: If entropy coding is used to encode y2, the distribution coefficient sigma needs to be passed

[0391] Parameter 2: The number of parameters num passed to y2.

[0392] Embodiment 18, decoding end overfitting solution 4 (corresponding to encoding end overfitting solution 4)

[0393] Pass the input parameters of the additional network (such as the target input above). Encode the required information into the bitstream, and the relevant information syntax includes:

[0394] 1. Input related parameters: pass y2.

[0395] 2. Other optional parameters. If both the encoding and decoding ends confirm the rules in advance, these parameters can be omitted:

[0396] Parameter 1: If entropy coding is used to encode y2, the distribution coefficient sigma needs to be passed

[0397] Parameter 2: The number of parameters num passed to y2.

[0398] Embodiment 19, decoding end overfitting solution 5 (corresponding to encoding end overfitting solution 5)

[0399] Select the overfitting layer in the original network layer according to different methods, and transfer the relevant parameters and low-rank matrix of the overfitting layer. Encode the required information into the bitstream, and the relevant information syntax includes:

[0400] 1. Identification of overfitting layer (same as Example 15).

[0401] 2. The identifier type of the network layer and the low-rank matrix in the overfitting original network (that is, the above-mentioned overfitting layer type identifier, used to indicate that the overfitting layer is the first type of overfitting layer or the second type of overfitting layer).

[0402] For example: type = 0, overfitting the original network layer (that is, the overfitting layer is the first type of overfitting layer; type = 1, overfitting and low-rank matrix (that is, the overfitting layer is the second type of overfitting layer).

[0403] 3. Overfitting the original network parameters and low-rank matrix parameters (refer to Examples 15 and 16).

[0404] 4. Other optional transmission parameters: If both the encoding end and the decoding end confirm the rules in advance, these parameters may not be transmitted (please refer to Examples 15 and 16).

[0405] Embodiment 20, decoding end overfitting solution 6 (corresponding to encoding end overfitting solution 6)

[0406] If you select one of the examples 1 to 4, you need to pass the relevant information syntax according to the above example, and only need to pass an additional scheme identifier index.

[0407] For example, index=1 means using scheme 1, and the passed parameters are the relevant parameters of the overfitting layer in the original synthetic transformation network.

[0408] It should be noted that in the embodiments of the present application, the above-mentioned embodiments can also be combined with each other to obtain new embodiments, and these new embodiments should also fall within the protection scope of the present application.

[0409] In addition, in the embodiments of the present application, the specific implementation of the decoding end embodiment and the specific implementation of the encoding end embodiment can refer to each other.

[0410] For example, since the encoder network structure includes the structure of the decoder network, the encoding end will also perform the above-mentioned "based on the compressed feature information and the over-fitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block" during the image encoding process. The specific implementation process can refer to the relevant description in the method embodiment of the above-mentioned decoding end side.

[0411] Based on the same application concept as the above method, the decoding end device (also referred to as a video decoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig.15The system comprises: a processor 1501 and a machine-readable storage medium 1502, wherein the machine-readable storage medium 1502 stores machine-executable instructions that can be executed by the processor 1501; the processor 1501 is used to execute the machine-executable instructions to implement the decoding method described in the above embodiment of the present application.

[0412] Based on the same application concept as the above method, the encoding end device (also referred to as a video encoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig.16 The system comprises: a processor 1601 and a machine-readable storage medium 1602, wherein the machine-readable storage medium 1602 stores machine-executable instructions that can be executed by the processor 1601; the processor 1601 is used to execute the machine-executable instructions to implement the encoding method described in the above embodiment of the present application.

[0413] Based on the same application concept as the above method, an embodiment of the present application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the decoding method or encoding method disclosed in the above example of the present application.

[0414] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented, such as the decoding method or encoding method disclosed in the above example.

[0415] Based on the same application concept as the above method, an embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the decoding method or encoding method disclosed in the above example of the present application.

[0416] Based on the same application concept as the above method, a decoding device is also proposed in an embodiment of the present application. The decoding device can be applied to a decoding end. The decoding device may include:

[0417] An acquisition unit, used to acquire feature information of a current image block after compression; and to acquire an overfitting parameter corresponding to the current image block;

[0418] A decoding unit is used to obtain a reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameter.

[0419] In one possible implementation, the decoder network includes a synthesis transformation network;

[0420] The acquisition unit is specifically used to acquire super a priori information corresponding to the current image block, and acquire residual probability distribution parameters corresponding to the current image block based on the super a priori information; based on the residual probability distribution parameters, acquire the quantized residual corresponding to the current image block;

[0421] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super-prior information and the quantization residual; based on the overfitting parameters and the reconstruction features, using the synthetic transformation network, to obtain the reconstructed image block corresponding to the current image block.

[0422] In one possible implementation, the overfitting parameter includes a weight target value of an overfitting layer in the decoder network;

[0423] or,

[0424] The overfitting parameters include target values ​​of bias items of overfitting layers in the decoder network;

[0425] or,

[0426] The overfitting parameters include weight target values ​​and bias item target values ​​of the overfitting layers in the decoder network.

[0427] The decoding unit is specifically configured to assign a weight and / or a bias item of an overfitting layer in the decoder network based on the weight target value and / or the bias item target value;

[0428] The compressed feature information is input into the decoder network after the assignment to determine the reconstructed image block corresponding to the current image block.

[0429] In a possible implementation, the weight target value includes: the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or, a weight offset determined based on the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the weight of the overfitting layer determined by offline training of the decoder network;

[0430] The bias item target value includes: the bias items of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or, the bias item offset determined by the bias item of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the bias item of the overfitting layer determined by offline training of the decoder network.

[0431] In a possible implementation manner, the overfitting parameters include target parameters of first-type additional modules corresponding to each overfitting layer in the decoder network, respectively, and the target parameters of the first-type additional modules are determined by the encoder by embedding the first-type additional modules in parallel for each overfitting layer in the decoder network, and performing online training on relevant parameters of the first-type additional modules using the current image block;

[0432] The decoding unit is specifically used to embed the first type of additional modules in parallel for each overfitting layer in the decoder network, and assign relevant parameters of the first type of additional modules based on target parameters of the first type of additional modules; input the compressed feature information into the first type of additional modules, and the decoder network after the assignment determines the reconstructed image block corresponding to the current image block.

[0433] In a possible implementation, the overfitting parameters include weight target values ​​of first-type overfitting layers in the decoder network, and target parameters of first-type additional modules corresponding to respective second-type overfitting layers in the decoder network;

[0434] or,

[0435] The overfitting parameters include target values ​​of bias items of a first type of overfitting layer in the decoder network, and target parameters of first type of additional modules corresponding to each second type of overfitting layer in the decoder network;

[0436] or,

[0437] The overfitting parameters include weight target values ​​and bias item target values ​​of a first type of overfitting layer in the decoder network, and target parameters of first type additional modules corresponding to each second type of overfitting layer in the decoder network.

[0438] The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and performing online training on relevant parameters of the first type of additional modules using the current image block;

[0439] The decoding unit is specifically used to assign weights and / or bias items of the first type of overfitting layer in the decoder network based on the weight target value and / or the bias item target value; and, embed the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and assign relevant parameters of the first type of additional modules based on the target parameters of the first type of additional modules; input the compressed feature information into the first type of additional modules, embed them into the decoder network after the assignment, and determine the reconstructed image block corresponding to the current image block.

[0440] In a possible implementation manner, the weight target value includes: the weight of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block; or, a weight offset determined based on the weight of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block and the weight of the first type of overfitting layer determined by offline training of the decoder network;

[0441] The bias item target value includes: the bias items of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block; or, the bias item offset determined by the bias item of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block and the bias item of the first type of overfitting layer determined by offline training of the decoder network.

[0442] In a possible implementation manner, the overfitting parameters include target inputs of the second type additional modules corresponding to the respective overfitting layers in the decoder network, and the target inputs of the second type additional modules are determined by the encoder by embedding the second type additional modules in parallel for the respective overfitting layers in the decoder network, and performing online training on the inputs of the second type additional modules; and relevant parameters of the second type additional modules are determined by offline training;

[0443] The decoding unit is specifically used to embed the second type of additional modules in parallel for each overfitting layer in the decoder network; use the compressed feature information as the input of the overfitting layer in the decoder network, and use the target input as the input of the second type of additional module, and use the decoder network embedded with the second type of additional module to determine the reconstructed image block corresponding to the current image block.

[0444] In a possible implementation, the overfitting parameter includes a target input of the additional network, the target input of the additional network is determined by the encoder performing online training on the input of the additional network based on the output of the decoder network and the output of the additional network, and the relevant parameters of the additional network are determined by offline training;

[0445] The decoding unit is specifically used to use the compressed feature information as the input of the overfitting layer in the decoder network, and use the target input as the input of the additional network, and use the output of the decoder network and the output of the additional network to determine the reconstructed image block corresponding to the current image block.

[0446] In a possible implementation manner, the acquisition unit is specifically configured to parse the super-prior information corresponding to the current image block from the first bitstream corresponding to the current image block; decode the super-prior information using a super-scale decoding network to obtain a residual probability distribution parameter corresponding to the current image block;

[0447] The acquisition unit is further specifically configured to decode the second bitstream corresponding to the current image block based on the residual probability distribution parameter to obtain a quantized residual corresponding to the current image block;

[0448] The decoding unit is further specifically used to determine the reconstruction features corresponding to the current image block based on the super-prior information and the quantized residual using a context-based prediction model.

[0449] In a possible implementation, the acquisition unit is specifically configured to parse the overfitting parameter corresponding to the current image block from a target bitstream corresponding to the current image block; wherein the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

[0450] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes an overfitting layer in the synthetic transform network;

[0451] The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block;

[0452] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super-prior information and the quantization residual; and, based on the weight target value and / or the bias item target value, assign the weights and / or bias items of the overfitting layer in the synthetic transformation network; input the reconstruction features into the synthetic transformation network after assignment, and determine the reconstructed image block corresponding to the current image block.

[0453] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes an overfitting layer in the synthetic transform network;

[0454] The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block;

[0455] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super prior information and the quantization residual; and to embed the first type of additional modules in parallel for each overfitting layer in the synthetic transformation network, and assign relevant parameters of the first type of additional modules based on the target parameters of the first type of additional modules; the reconstruction features are input into the first type of additional modules, and the synthetic transformation network after the assignment determines the reconstructed image block corresponding to the current image block.

[0456] In a possible implementation, the decoder network includes a synthetic transform network; the first type of overfitting layer in the decoder network includes the first type of overfitting layer in the synthetic transform network, and the second type of overfitting layer in the decoder network includes the second type of overfitting layer in the synthetic transform network;

[0457] The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block;

[0458] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super prior information and the quantization residual; and, based on the weight target value and / or the bias item target value, assign the weight and / or bias item of the first type of overfitting layer in the synthetic transformation network; and, in the synthetic transformation network, embed the first type of additional modules in parallel for each second type of overfitting layer, and assign relevant parameters of the first type of additional modules based on the target parameters of the first type of additional modules; the reconstruction features are input into the first type of additional modules, and the synthetic transformation network after the assignment determines the reconstructed image block corresponding to the current image block.

[0459] In a possible implementation, the first type of additional module is a low-rank matrix.

[0460] In a possible implementation, the acquisition unit is further used to obtain an overfitting layer type identifier corresponding to the current image block; wherein the overfitting layer type identifier is used to indicate that the overfitting layer is the first type overfitting layer or the second type overfitting layer.

[0461] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes an overfitting layer in the synthetic transform network;

[0462] The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block;

[0463] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super prior information and the quantization residual; and to embed the second type of additional modules in parallel for each overfitting layer in the synthetic transformation network; use the reconstruction features as the input of the overfitting layer in the synthetic transformation network, and use the target input as the input of the second type of additional module, and use the synthetic transformation network embedded with the second type of additional module to determine the reconstructed image block corresponding to the current image block.

[0464] In a possible implementation, the overfitting layer includes part of the network layers or all of the network layers in the network.

[0465] In a possible implementation, the acquisition unit is further configured to, when the overfitting layer includes a part of the network layers in the network, acquire an overfitting layer identifier corresponding to the current image block; wherein the overfitting layer identifier is used to indicate the overfitting layer in the network;

[0466] The overfitting layer identifier includes the layer sequence number of the overfitting layer, or the number of overfitting layers N; wherein the number of overfitting layers N is used to indicate that the first N layer sequence numbers in the layer sequence number list determined offline are the layer sequence numbers of the overfitting layers.

[0467] In one possible implementation, the decoder network includes a synthesis transformation network;

[0468] The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block;

[0469] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image block based on the super prior information and the quantization residual; use the reconstruction features as the input of the overfitting layer in the synthetic transformation network, and use the target input as the input of the additional network, and use the output of the synthetic transformation network and the output of the additional network to determine the reconstructed image block corresponding to the current image block.

[0470] In a possible implementation manner, the acquisition unit is further configured to acquire an overfitting scheme identifier corresponding to the current image block; wherein different overfitting scheme identifiers correspond to different overfitting schemes;

[0471] The acquisition unit is further specifically configured to parse the overfitting parameter corresponding to the current image block from the target bitstream corresponding to the current image block based on the overfitting scheme identifier; wherein the target bitstream includes the first bitstream, the second bitstream or the third bitstream.

[0472] In a possible implementation manner, the acquisition unit is further configured to acquire a distribution coefficient list of each overfitting layer parameter corresponding to the current image block or the distribution coefficients of all parameters when the encoding end uses entropy coding to encode the overfitting parameter;

[0473] and / or,

[0474] In a case where the overfitting parameters include network parameters, obtaining the number of parameters corresponding to each overfitting layer corresponding to the current image block;

[0475] and / or,

[0476] In a case where the overfitting parameters include parameters of a low-rank matrix, obtaining the rank of the low-rank matrix corresponding to the current image block;

[0477] and / or,

[0478] In a case where the overfitting parameters include input parameters, the number of input parameters corresponding to the current image block is obtained.

[0479] Based on the same application concept as the above method, an encoding device is also proposed in an embodiment of the present application. The encoding device can be applied to an encoding end. The encoding device may include:

[0480] Acquire compressed feature information of the current image block, and write the compressed feature information into a code stream corresponding to the current image block; and

[0481] An overfitting parameter corresponding to the current image block is obtained, and the overfitting parameter is written into a code stream corresponding to the current image block.

[0482] It should be noted that since the encoder network structure includes the structure of the decoder network, the encoding end will also perform the above-mentioned "based on the compressed feature information and the over-fitting parameters, using the decoder network to obtain the reconstructed image block corresponding to the current image block" during the image encoding process. The specific implementation process can refer to the relevant description in the above-mentioned decoding end side embodiment.

[0483] Correspondingly, the above encoding unit may also have related functions of the above decoding unit.

[0484] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The above is only an embodiment of the present application and is not intended to limit the present application.

[0485] For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A decoding method, characterized in that: The method comprises: Obtaining feature information of the current image block after compression; and, Obtaining an overfitting parameter corresponding to the current image block; Based on the compressed feature information and the overfitting parameter, a decoder network is used to obtain a reconstructed image block corresponding to the current image block.

2. The method according to claim 1, characterized in that The decoder network includes a synthetic transform network; The step of obtaining the compressed feature information of the current image block includes: Acquire super a priori information corresponding to the current image block, and acquire residual probability distribution parameters corresponding to the current image block based on the super a priori information; Based on the residual probability distribution parameter, obtaining a quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; Based on the overfitting parameter and the reconstruction feature, the synthetic transformation network is used to obtain a reconstructed image block corresponding to the current image block.

3. The method according to claim 1 or 2, characterized in that: The overfitting parameters include weight target values ​​of overfitting layers in the decoder network; or, The overfitting parameters include target values ​​of bias items of overfitting layers in the decoder network; or, The overfitting parameters include weight target values ​​and bias item target values ​​of the overfitting layer in the decoder network; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Assigning weights and / or bias items of an overfitting layer in the decoder network based on the weight target value and / or bias item target value; The compressed feature information is input into the decoder network after the assignment to determine the reconstructed image block corresponding to the current image block.

4. The method according to claim 3, characterized in that The weight target value includes: the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or a weight offset determined based on the weight of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the weight of the overfitting layer determined by offline training of the decoder network; The bias item target value includes: the bias items of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block; or, the bias item offset determined by the bias item of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image block and the bias item of the overfitting layer determined by offline training of the decoder network.

5. The method according to claim 1 or 2, characterized in that: The overfitting parameters include target parameters of first-type additional modules corresponding to each overfitting layer in the decoder network, respectively, and the target parameters of the first-type additional modules are determined by the encoder embedding the first-type additional modules in parallel for each overfitting layer in the decoder network, and performing online training on relevant parameters of the first-type additional modules using the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: embedding the first type of additional modules in parallel for each overfitting layer in the decoder network, and assigning relevant parameters of the first type of additional modules based on target parameters of the first type of additional modules; The compressed feature information is input and embedded into the first type additional module, and the decoder network after assignment determines the reconstructed image block corresponding to the current image block.

6. The method according to claim 1 or 2, characterized in that: The overfitting parameters include weight target values ​​of first-type overfitting layers in the decoder network, and target parameters of first-type additional modules corresponding to respective second-type overfitting layers in the decoder network; or, The overfitting parameters include target values ​​of bias items of a first type of overfitting layer in the decoder network, and target parameters of first type additional modules corresponding to respective second type of overfitting layers in the decoder network; or, The overfitting parameters include weight target values ​​and bias item target values ​​of a first type of overfitting layer in the decoder network, and target parameters of first type additional modules corresponding to each second type of overfitting layer in the decoder network; The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and performing online training on relevant parameters of the first type of additional modules using the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Assigning values ​​to weights and / or bias items of a first type of overfitting layer in the decoder network based on the weight target value and / or the bias item target value; and, embedding the first type of additional module in parallel for each second type of overfitting layer in the decoder network, and assigning values ​​to relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; The compressed feature information is input and embedded into the first type additional module, and the decoder network after assignment determines the reconstructed image block corresponding to the current image block.

7. The method according to claim 6, characterized in that The weight target value includes: the weight of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block; or, a weight offset determined based on the weight of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block and the weight of the first type of overfitting layer determined by offline training of the decoder network; The bias item target value includes: the bias items of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block; or, the bias item offset determined by the bias item of each first type of overfitting layer determined by online training of the decoder network embedded in the first type of additional module using the current image block and the bias item of the first type of overfitting layer determined by offline training of the decoder network.

8. The method according to claim 1 or 2, characterized in that: The overfitting parameters include target inputs of the second type additional modules respectively corresponding to the overfitting layers in the decoder network, and the target inputs of the second type additional modules are determined by the encoder embedding the second type additional modules in parallel for each overfitting layer in the decoder network and performing online training on the inputs of the second type additional modules; and the relevant parameters of the second type additional modules are determined by offline training; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: embedding the second type of additional modules in parallel for each overfitting layer in the decoder network; The compressed feature information is used as the input of the overfitting layer in the decoder network, and the target input is used as the input of the second type of additional module. The decoder network embedded with the second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

9. The method according to claim 1 or 2, characterized in that: The overfitting parameter includes a target input of the additional network, the target input of the additional network is determined by the encoder performing online training on the input of the additional network based on the output of the decoder network and the output of the additional network, and the relevant parameters of the additional network are determined by offline training; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: The compressed feature information is used as the input of the overfitting layer in the decoder network, and the target input is used as the input of the additional network. The output of the decoder network and the output of the additional network are used to determine the reconstructed image block corresponding to the current image block.

10. The method according to claim 2, characterized in that The obtaining of the super prior information corresponding to the current image block, and obtaining the residual probability distribution parameter corresponding to the current image block based on the super prior information, includes: Parsing the super-prior information corresponding to the current image block from the first bitstream corresponding to the current image block; Decoding the super-prior information using a super-scale decoding network to obtain residual probability distribution parameters corresponding to the current image block; The obtaining, based on the residual probability distribution parameter, a quantized residual corresponding to the current image block, comprises: Based on the residual probability distribution parameter, decoding a second bitstream corresponding to the current image block to obtain a quantized residual corresponding to the current image block; The obtaining, based on the super-prior information and the quantized residual, a reconstruction feature corresponding to the current image block includes: Based on the super-prior information and the quantized residual, a context-based prediction model is used to determine a reconstruction feature corresponding to the current image block.

11. The method according to claim 1 or 2, characterized in that: The obtaining the overfitting parameter corresponding to the current image block includes: The overfitting parameter corresponding to the current image block is parsed from the target bitstream corresponding to the current image block; wherein the target bitstream includes a first bitstream, a second bitstream or a third bitstream.

12. The method according to claim 3, characterized in that The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super-prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; and Based on the weight target value and / or the bias item target value, assigning a weight and / or a bias item of an overfitting layer in the synthetic transformation network; The reconstructed features are input into the synthetic transformation network after the assignment to determine the reconstructed image block corresponding to the current image block.

13. The method according to claim 5, characterized in that The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super-prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; and In the synthetic transformation network, the first type of additional modules are embedded in parallel for each overfitting layer, and relevant parameters of the first type of additional modules are assigned values ​​based on target parameters of the first type of additional modules; The reconstructed feature input is embedded into the first type additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

14. The method according to claim 6, characterized in that The decoder network includes a synthetic transform network; the first type of overfitting layer in the decoder network includes the first type of overfitting layer in the synthetic transform network, and the second type of overfitting layer in the decoder network includes the second type of overfitting layer in the synthetic transform network; The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; as well as, Based on the weight target value and / or the bias target value, weights and / or bias items of the first type of overfitting layer in the synthetic transformation network are assigned; and, in the synthetic transformation network, the first type of additional modules are embedded in parallel for each second type of overfitting layer, and relevant parameters of the first type of additional modules are assigned based on the target parameters of the first type of additional modules; The reconstructed feature input is embedded into the first type additional module, and the synthetic transformation network after assignment determines the reconstructed image block corresponding to the current image block.

15. The method according to any one of claims 5 to 6, 13 to 14, characterized in that: The first type of additional module is a low-rank matrix.

16. The method according to claim 6 or 14, characterized in that The method further comprises: Obtain an overfitting layer type identifier corresponding to the current image block; wherein the overfitting layer type identifier is used to indicate that the overfitting layer is the first type overfitting layer or the second type overfitting layer.

17. The method according to claim 8, characterized in that The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super-prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; and embedding the second type of additional modules in parallel for each overfitting layer in the synthetic transformation network; The reconstructed feature is used as the input of the overfitting layer in the synthetic transformation network, and the target input is used as the input of the second type of additional module. The synthetic transformation network embedded with the second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

18. The method according to any one of claims 3 to 7, 12 to 14 and 17, characterized in that: The overfitting layer includes part of the network layers or all of the network layers in the network.

19. The method according to claim 18, characterized in that In the case where the overfitting layer includes a portion of the network layers in the network, the method further includes: Obtaining an overfitting layer identifier corresponding to the current image block; wherein the overfitting layer identifier is used to indicate an overfitting layer in a network; The overfitting layer identifier includes the layer sequence number of the overfitting layer, or the number of overfitting layers N; wherein the number of overfitting layers N is used to indicate that the first N layer sequence numbers in the layer sequence number list determined offline are the layer sequence numbers of the overfitting layers.

20. The method according to claim 9, characterized in that The decoder network includes a synthetic transform network; The compressed feature information includes the super-prior information corresponding to the current image block and the quantized residual corresponding to the current image block; The obtaining, based on the compressed feature information and the overfitting parameter, a reconstructed image block corresponding to the current image block by using a decoder network includes: Based on the super prior information and the quantized residual, obtaining a reconstruction feature corresponding to the current image block; The reconstructed features are used as inputs of the overfitting layer in the synthetic transformation network, and the target input is used as inputs of the additional network. The reconstructed image block corresponding to the current image block is determined by utilizing the output of the synthetic transformation network and the output of the additional network.

21. The method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining an overfitting scheme identifier corresponding to the current image block; wherein different overfitting scheme identifiers correspond to different overfitting schemes; The obtaining the overfitting parameter corresponding to the current image block includes: Based on the overfitting scheme identifier, an overfitting parameter corresponding to the current image block is parsed from a target bitstream corresponding to the current image block; wherein the target bitstream includes a first bitstream, a second bitstream or a third bitstream.

22. The method according to claim 1 or 2, characterized in that The method further comprises: When the encoding end uses entropy coding to encode the overfitting parameters, obtaining a distribution coefficient list of the overfitting layer parameters corresponding to the current image block or the distribution coefficients of all parameters; and / or, In a case where the overfitting parameters include network parameters, obtaining the number of parameters corresponding to each overfitting layer corresponding to the current image block; and / or, In a case where the overfitting parameters include parameters of a low-rank matrix, obtaining the rank of the low-rank matrix corresponding to the current image block; and / or, In a case where the overfitting parameters include input parameters, the number of input parameters corresponding to the current image block is obtained.

23. A coding method, characterized in that: Applied to the encoding end, the method includes: Acquire compressed feature information of the current image block, and write the compressed feature information into a code stream corresponding to the current image block; and An overfitting parameter corresponding to the current image block is obtained, and the overfitting parameter is written into a code stream corresponding to the current image block.

24. A decoding device, characterized in that: include: An acquisition unit, used for acquiring feature information of the current image block after compression; And, obtaining an overfitting parameter corresponding to the current image block; A decoding unit is used to obtain a reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameter.

25. An encoding device, characterized in that: include: An acquisition unit, used for acquiring feature information of the current image block after compression; An encoding unit, used for writing the compressed feature information into a code stream corresponding to the current image block; The acquisition unit is further used to acquire an overfitting parameter corresponding to the current image block; The encoding unit is further used to write the overfitting parameter into the code stream corresponding to the current image block.

26. A decoding end device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method described in any one of claims 1-22.

27. A coding end device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is configured to execute machine executable instructions to implement the method of claim 23.

28. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method described in any one of claims 1 to 22, or to implement the method described in claim 23.

29. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by the processor, the method described in any one of claims 1 to 22 is implemented, or when the computer instructions are executed by the processor, the method described in claim 23 is implemented.

Images