A decoding and encoding method, apparatus and equipment

Abstract

This application provides a decoding and encoding method, apparatus, and device. The decoding method includes: acquiring compressed feature information of a current image block; acquiring overfitting parameters corresponding to the current image block; and acquiring a reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameters. This method can improve encoding and decoding performance.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding technology, and in particular to a decoding and encoding method, apparatus and device thereof. Background Technology

[0002] To save space, video images are encoded before transmission. Complete video encoding includes processes such as prediction, transform, quantization, entropy coding, and filtering. The prediction process can be divided into intra-frame prediction and inter-frame prediction. Inter-frame prediction utilizes temporal correlation to predict the current pixel using pixels from neighboring encoded images, effectively removing temporal redundancy. Intra-frame prediction utilizes spatial correlation to predict the current pixel using pixels from the encoded blocks of the current frame, removing spatial redundancy.

[0003] With the rapid development of deep learning, it has achieved success in many high-level computer vision problems, such as image classification and object detection. Deep learning is also gradually being applied in the field of encoding and decoding, where neural networks can be used to encode and decode images. Although neural network-based encoding and decoding methods have shown great performance potential, they still suffer from problems such as poor encoding performance, poor decoding performance, and high complexity. Summary of the Invention

[0004] In view of this, this application provides a decoding and encoding method, apparatus and device, to improve encoding and decoding performance.

[0005] According to a first aspect of the embodiments of this application, a decoding method is provided, applied at a decoding end, the method comprising:

[0006] Obtain the compressed feature information of the current image patch; and,

[0007] Obtain the overfitting parameters corresponding to the current image patch;

[0008] Based on the compressed feature information and the overfitting parameters, the reconstructed image block corresponding to the current image block is obtained using a decoder network.

[0009] According to a second aspect of the embodiments of this application, an encoding method is provided, applied at an encoding end, the method comprising:

[0010] Obtain the compressed feature information of the current image block, and write the compressed feature information into the bitstream corresponding to the current image block; and,

[0011] Obtain the overfitting parameters corresponding to the current image block, and write the overfitting parameters into the bitstream corresponding to the current image block.

[0012] According to a third aspect of the embodiments of this application, a decoding apparatus is provided, comprising:

[0013] The acquisition unit is used to acquire the compressed feature information of the current image block; and to acquire the overfitting parameters corresponding to the current image block.

[0014] The decoding unit is used to obtain the reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameters.

[0015] According to a fourth aspect of the embodiments of this application, an encoding apparatus is provided, comprising:

[0016] The acquisition unit is used to acquire the compressed feature information of the current image block;

[0017] The encoding unit is used to write the compressed feature information into the bitstream corresponding to the current image block;

[0018] The acquisition unit is further configured to acquire the overfitting parameters corresponding to the current image block;

[0019] The encoding unit is also used to write the overfitting parameters into the bitstream corresponding to the current image block.

[0020] According to a fifth aspect of the embodiments of this application, a decoding end device, a processor, and a machine-readable storage medium are provided, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor;

[0021] The processor is used to execute machine-executable instructions to implement the method provided in the first aspect.

[0022] According to a sixth aspect of the embodiments of this application, an encoding end device is provided, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0023] The processor is used to execute machine-executable instructions to implement the method provided in the second aspect.

[0024] According to a seventh aspect of the present application, an electronic device is provided, comprising: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute the machine-executable instructions to implement the method provided in the first aspect or the second aspect.

[0025] According to an eighth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein a plurality of computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, they implement the method provided in the first aspect, or when the computer instructions are executed by a processor, they implement the method provided in the second aspect.

[0026] As can be seen from the above technical solutions, in the embodiments of this application, by obtaining the compressed feature information of the current image block and the overfitting parameter corresponding to the current image block, the overfitting parameter can be used to make the output of the decoder network more suitable for the current image block. Then, based on the compressed feature information and the overfitting parameter, the decoder network is used to obtain the reconstructed image block corresponding to the current image block. The online overfitting method provides a better solution for the current image block in the decoder network, thereby optimizing the image encoding and decoding performance. Attached Figure Description

[0027] Figure 1A This is a schematic diagram of a three-dimensional feature matrix in one embodiment of this application;

[0028] Figure 1B This is a schematic diagram of an end-to-end coding framework in one embodiment of this application;

[0029] Figure 2 This is a flowchart illustrating a decoding method provided in an embodiment of this application;

[0030] Figure 3 This is a flowchart illustrating an encoding method provided in an embodiment of this application;

[0031] Figure 4A This is a schematic diagram of the model structure of the high-gear synthesis transformation network in one embodiment of this application;

[0032] Figure 4B This is a schematic diagram of the model structure of the base level synthesis transformation network in one embodiment of this application;

[0033] Figure 5A This is a schematic diagram of the ResAU layer in one embodiment of this application;

[0034] Figure 5B This is a schematic diagram of the RB layer structure in one embodiment of this application;

[0035] Figure 5C This is a schematic diagram of the RNAB layer structure in one embodiment of this application;

[0036] Figure 6 These are schematic diagrams illustrating various overfitting methods provided in the embodiments of this application;

[0037] Figure 7This is a flowchart illustrating a decoding method provided in an embodiment of this application;

[0038] Figure 8 This is a flowchart illustrating a decoding method provided in an embodiment of this application;

[0039] Figure 9 This is a flowchart illustrating an encoding method provided in an embodiment of this application;

[0040] Figure 10 This is a schematic diagram of an overfitting model provided in an embodiment of this application;

[0041] Figure 11 This is a schematic diagram of an overfitting model provided in an embodiment of this application;

[0042] Figure 12 This is a schematic diagram of an overfitting model provided in an embodiment of this application;

[0043] Figure 13 This is a schematic diagram of an overfitting model provided in an embodiment of this application;

[0044] Figure 14 This is a schematic diagram of an overfitting model provided in an embodiment of this application;

[0045] Figure 15 This is a schematic diagram of the hardware structure of a decoding device provided in an embodiment of this application;

[0046] Figure 16 This is a schematic diagram of the hardware structure of an encoding end device provided in an embodiment of this application. Detailed Implementation

[0047] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” used in the embodiments and claims of this application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any or all possible combinations including one or more of the associated listed items. It should be understood that although the terms first, second, third, etc., may be used to describe various information in the embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of the embodiments of this application, and similarly, second information may also be referred to as first information, depending on the context. Furthermore, the word “if” as used can be interpreted as “when,” “in response to a determination,” or “when…”.

[0048] This application proposes a decoding method and an encoding method, which may involve the following concepts:

[0049] JPEG (Joint Photographic Experts Group): JPEG is a standard for compressing continuous-tone still images. Files with the extension .jpg or .jpeg are commonly used image file formats. JPEG uses a joint coding method of predictive coding (DPCM), discrete cosine transform (DCT), and entropy coding to remove redundant image and color data. It is a lossy compression format, capable of compressing images into a very small storage space, but it inevitably causes some damage to the image data. Especially when using excessively high compression ratios, the quality of the decompressed image will be reduced. If high-quality images are desired, JPEG should not be used with excessively high compression ratios.

[0050] Overfitting, also known as overlearning, is characterized by an algorithm performing well on the training set but poorly on the test set, exhibiting poor generalization performance. Overfitting occurs because the training data contains sampling errors, and the complex model incorporates these errors into the model's parameter fitting process. Sampling error refers to the deviation between the sampled data and the overall dataset.

[0051] Entropy coding: Entropy coding is a coding process that follows the principle of entropy without losing any information. Information entropy is the average amount of information in the source (a measure of uncertainty). Entropy coding methods can include, but are not limited to, Shannon coding, Huffman coding, and arithmetic coding.

[0052] Neural Networks (NNs): Neural networks are artificial neural networks, a computational model composed of numerous interconnected nodes (or neurons). In a neural network, neurons can represent different objects, such as features, letters, concepts, or meaningful abstract patterns. The types of processing units in a neural network can be divided into three categories: input units, output units, and hidden units. Input units receive signals and data from the external world; output units output the processed results; hidden units are located between the input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units; the representation and processing of information are reflected in the connections between processing units. Neural networks are a non-programmed, brain-like information processing method. The essence of a neural network is to obtain a parallel and distributed information processing function through the transformations and dynamic behavior of the network, mimicking the information processing function of the human brain's nervous system to varying degrees and levels. In the field of video processing, commonly used neural networks include, but are not limited to, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and fully connected networks.

[0053] Convolutional Neural Networks (CNNs) are a type of feedforward neural network and one of the most representative network structures in deep learning. The artificial neurons in a CNN can respond to surrounding units within a certain coverage area, exhibiting excellent performance in large-scale image processing. The basic structure of a CNN consists of two layers: a feature extraction layer (also called a convolutional layer), where the input of each neuron is connected to the local receptive field of the previous layer, extracting features from that local area. Once these local features are extracted, their positional relationship with other features is determined. The second layer is a feature mapping layer (also called an activation layer). Each computational layer of the neural network consists of multiple feature maps, each a plane where all neurons have equal weights. Feature mapping structures can use functions such as the Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, and GDN function as activation functions for the convolutional network. Furthermore, because neurons on a single feature map share weights, the number of free parameters in the network is reduced.

[0054] For example, one advantage of convolutional neural networks (CNNs) over image processing algorithms is that they avoid complex preprocessing steps (such as extracting artificial features) and can directly input the original image for end-to-end learning. Another advantage of CNNs over ordinary neural networks is that ordinary neural networks use fully connected layers, meaning all neurons from the input layer to the hidden layer are connected. This results in a huge number of parameters, making network training time-consuming or even difficult. CNNs, however, avoid this difficulty through local connectivity and weight sharing.

[0055] Deconvolution: Also known as transposed convolution, deconvolution layers work similarly to convolutional layers. The main difference is that deconvolution layers use padding to make the output larger than the input (though they can also remain the same). If the stride is 1, the output size equals the input size; if the stride is N, the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0056] Generalization ability: Generalization ability refers to the ability of a machine learning algorithm to adapt to new samples. The goal of learning is to learn the patterns hidden behind data pairs. The trained network can also give appropriate outputs for data outside the learning set that have the same pattern. This ability can be called generalization ability.

[0057] Features: The features involved in this application are a three-dimensional feature matrix of size C*W*H, see [link to relevant documentation]. Figure 1A The diagram shows a schematic of a three-dimensional feature matrix. In the three-dimensional feature matrix, C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be either the input or the output of a neural network.

[0058] Feature Channel: In a neural network, the input signal passes through a convolutional layer to obtain many feature maps. Each feature map is called a feature channel, which contains a part of the information of the input signal.

[0059] Rate-Distortion Optimized (RDO) principle: Two main metrics for evaluating coding efficiency are bitrate and PSNR (Peak Signal-to-Noise Ratio). A smaller bitrate results in a higher compression ratio, and a higher PSNR results in better reconstructed image quality. In mode selection, the decision formula is essentially a comprehensive evaluation of these two metrics. For example, the cost corresponding to a mode is: J(mode) = D + λ*R, where D represents Distortion, typically measured using the SSE metric (Sum of Mean Squares of Differences between the Reconstructed Image Patch and the Source Image). Alternatively, the SAD metric (Sum of Absolute Differences between the Reconstructed Image Patch and the Source Image) can be used to consider the cost. λ is a Lagrange multiplier, and R is the actual number of bits required to encode the image patch in that mode, including the total number of bits needed to encode mode information, motion information, residuals, etc. Using the rate-distortion principle to compare and decide on coding modes during mode selection usually ensures optimal coding performance.

[0060] Numerous encoding tools have been proposed for various modules at the encoding end, and each tool often has multiple modes. The optimal encoding tool for different video sequences often differs. Therefore, during encoding, Restricted Directive (RDO) is typically used to compare the encoding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information is transmitted by encoding marker information in the bitstream. While this method introduces higher encoding complexity, it can adaptively select the optimal mode combination for different content, achieving the best encoding performance. The decoding end can obtain the relevant mode information by directly parsing the marker information, with minimal impact on complexity.

[0061] The following example illustrates the end-to-end image encoding process.

[0062] Please see Figure 1B This is a schematic diagram of an end-to-end image coding (also known as image compression) framework provided in an embodiment of this application. Of course, Figure 1B This is just one example of an end-to-end image coding framework, and does not impose any restrictions on the implementation of end-to-end image coding.

[0063] like Figure 1B As shown, this end-to-end image coding framework may include modules such as: analysis transform network, synthesis transform network, context-based prediction, quantization, entropy coding, entropy decoding, super coding network, and superscale decoding network. Among them:

[0064] Analysis of the transformation network: Transforms image patch x to feature y in the latent domain (image domain to feature domain), facilitating all subsequent processes to operate in the latent domain.

[0065] For example, an image can be divided into one image block or multiple image blocks. If the image is divided into one image block, then the current image block x can also be an image. That is, the encoding and decoding process for image blocks can also be directly applied to images.

[0066] Super-encoded network: Input the probability distribution parameters σ of the residual r, obtain the super-prior information z_hat, and write z_hat into the bitstream biasstream#1, which mainly transmits the probability model parameters σ of the residual.

[0067] Context-based prediction module: The input of this module includes not only z_hat, but also the decoded y_hat. The two are combined as input to obtain a more accurate mean mu, which is the predicted value of the original feature. The reconstructed y_hat is obtained by adding the reconstruction residual r_hat.

[0068] Superscale decoding network: The output z_hat of the super-encoding network is used as the input of the superscale decoding network in another path to obtain σ. After quantization, σ_hat is obtained. σ_hat will be used to combine with the second bitstream (Bitstream#2) to obtain the quantization residual r_q.

[0069] Synthesis Transformation Module: The obtained reconstructed latent domain features y_hat are processed by the synthesis transformation module to obtain reconstructed image blocks.

[0070] Entropy coding: a lossless coding method based on the principle of information entropy, which transforms a series of element symbols used to represent a video sequence (such as transform coefficients and mode information) into a binary code stream, removing the statistical redundancy of these video element symbols.

[0071] Entropy decoding: Entropy decoding is the inverse process of entropy encoding, which can convert the bit stream that has been entropy encoded back into the original symbol sequence.

[0072] Quantization, a many-to-one mapping process, introduces signal loss and affects the residual r. It alters the signal's value range, allowing the encoder to approximate the original signal with fewer symbols, thus improving compression ratio. Furthermore, quantization also affects σ, yielding σ_hat, which is used to better estimate the probability distribution of the quantization residual r_q.

[0073] Dequantization: Dequantization is the inverse process of quantization, which is a process of restoring quantized data to its original continuous value.

[0074] based on Figure 1B The end-to-end image encoding process within the framework shown can be as follows:

[0075] After obtaining the current image block x (which can be the original image block x, i.e., the input image block), the encoding end can perform analysis and transformation on the current image block x through an analysis and transformation network (i.e., a neural network) to obtain the image features y corresponding to the current image block x. Specifically, performing feature transformation on the current image block x through the analysis and transformation network means transforming the current image block x to image features y in the latent domain, thereby facilitating all subsequent processes to be performed in the latent domain.

[0076] Given the image features y, the encoder can determine the probability distribution parameter σ of the image feature residuals based on the image features y.

[0077] On the one hand, the encoding end can input the probability distribution parameter σ into the super-coding network to obtain the prior information z_hat, and write z_hat into the bitstream biasstream#1 through entropy encoding.

[0078] For example, after obtaining the Bitstream#1 corresponding to the current image block, the encoding end can send the Bitstream#1 corresponding to the current image block to the decoding end. For the processing of the Bitstream#1 corresponding to the current image block by the decoding end, please refer to the following embodiments.

[0079] After obtaining Bitstream#1 corresponding to the current image block, the encoding end can also decode Bitstream#1, such as entropy decoding, to obtain the prior information z_hat.

[0080] After obtaining the prior information z_hat, the encoder can, on the one hand, input the prior information z_hat into the superscale decoding network to obtain the probability distribution parameter σ, and then quantize σ to obtain σ_hat. On the other hand, the prior information z_hat can be input into the context-based prediction module. Using the context-based prediction module, based on the prior information z_hat of the current image patch and the reconstructed features y_hat of the previous image patches, context-based prediction is performed to obtain the predicted value (predicted value of image feature y) mu (i.e., the mean mu) corresponding to the current image patch.

[0081] On the other hand, the encoder determines the residual r of the image features based on the image features y and the predicted value mu of the image features y. For example, the difference between the image features y and the predicted value of the image features y is determined as the residual of the image features.

[0082] The encoder can quantize the residual r to obtain the quantized residual r_q, and perform entropy encoding on the quantized residual r_q based on σ_hat to obtain the Bitstream#2 corresponding to the current image block.

[0083] For example, after obtaining the Bitstream#2 corresponding to the current image block, the encoding end can send the Bitstream#2 corresponding to the current image block to the decoding end. For the processing of the Bitstream#2 corresponding to the current image block by the decoding end, please refer to the following embodiments.

[0084] After obtaining Bitstream#2 corresponding to the current image block, the encoding end can also perform entropy decoding on Bitstream#2 based on σ_hat to obtain the quantization residual r_q, and then perform dequantization to obtain the dequantized residual (which can be called the reconstruction residual) r_hat.

[0085] The encoder can obtain the reconstruction feature y_hat of the current image block based on the reconstruction residual r_hat and the predicted value mu of the image feature y. The reconstruction feature y_hat of the current image block is then input into the synthesis transform network to obtain the reconstructed image block corresponding to the current image block.

[0086] The decoding and encoding methods in the embodiments of this application will be described in detail below with reference to several specific examples.

[0087] This application proposes a decoding method in its embodiments; see [link to documentation]. Figure 2 The diagram shown illustrates the flowchart of this decoding method, which can be applied to the decoding end (also known as a video decoder). This method may include:

[0088] Step S200: Obtain the compressed feature information of the current image block.

[0089] Step S210: Obtain the overfitting parameters corresponding to the current image patch.

[0090] In this embodiment, it is considered that the model parameters obtained by training in the traditional scheme are an "optimal solution" for the training set as a whole, but not necessarily an optimal solution for each training sample (such as an image) in the training set. For other types of images outside the training set, this is even less of an "optimal solution".

[0091] Therefore, in order to improve image encoding and decoding performance, when applying the trained model, the parameters of the trained model can be adjusted through online fitting to obtain a better solution.

[0092] Correspondingly, online overfitting can be performed on the decoder network at the encoder end to adjust the model parameters of the decoder network and obtain model parameters that are more suitable for the current image, that is, to obtain a better solution for the current image.

[0093] The encoder can write the overfitting parameters determined by online overfitting of the decoder network into the bitstream and send it to the decoder.

[0094] During the decoding process of the current image block, the decoding end can obtain the compressed feature information of the current image block, for example, by parsing the compressed feature information of the current image block from the bitstream corresponding to the current image block; on the other hand, it can obtain the overfitting parameters corresponding to the current image block, for example, by parsing the overfitting parameters corresponding to the current image block from the bitstream corresponding to the current image block.

[0095] For example, the compressed feature information can characterize other information in the information sent by the encoder, excluding the overfitting parameters.

[0096] Step S220: Based on the compressed feature information and the overfitting parameters, use the decoder network to obtain the reconstructed image block corresponding to the current image block.

[0097] In this embodiment of the application, when the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can use the decoder network to obtain the reconstructed image block corresponding to the current image block based on the obtained compressed feature information and the overfitting parameters.

[0098] For example, overfitting parameters can be used to overfit the decoder network, providing a better solution for the current image patch in the decoder network (even if the decoder network can obtain a better processing result by processing the compressed feature information of the current image patch).

[0099] For example, the decoder can adjust the model parameters of the decoder network based on the overfitting parameters to obtain model parameters of the decoder network that are more suitable for the current image patch. Based on the obtained compressed feature information, the decoder network with adjusted model parameters can be used to obtain the reconstructed image patch corresponding to the current image patch.

[0100] It can be seen that, in Figure 2 In the method flow shown, the compressed feature information of the current image block is obtained, and the overfitting parameter corresponding to the current image block is obtained. The overfitting parameter can be used to make the output of the decoder network more suitable for the current image block. Then, based on the compressed feature information and the overfitting parameter, the decoder network is used to obtain the reconstructed image block corresponding to the current image block. The online overfitting method provides a better solution for the current image block in the decoder network, thereby optimizing the image encoding and decoding performance.

[0101] In some embodiments, the decoder network includes a synthesis transform network;

[0102] The above-mentioned acquisition of the compressed feature information of the current image block may include:

[0103] Obtain the prior information corresponding to the current image patch, and obtain the residual probability distribution parameters corresponding to the current image patch based on the prior information;

[0104] Based on the residual probability distribution parameters, the quantization residual corresponding to the current image patch is obtained.

[0105] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0106] Based on prior information and quantization residuals, the reconstructed features corresponding to the current image patch are obtained;

[0107] Obtain the overfitting parameters corresponding to the current image patch, and based on the overfitting parameters and reconstruction features, use a synthetic transformation network to obtain the reconstructed image patch corresponding to the current image patch.

[0108] For example, consider a decoder network that includes a synthetic transform network. The synthetic transform network can take the reconstructed features of an image patch as input, perform a synthetic transform on the reconstructed features of the image patch, and obtain the reconstructed image patch corresponding to the image patch.

[0109] Accordingly, when the decoder network includes a synthesis transform network, the decoder can obtain the prior information corresponding to the current image block and obtain the residual probability distribution parameters corresponding to the current image block based on the obtained prior information.

[0110] In one example, the decoder can parse the prior information corresponding to the current image patch from the first bitstream (such as Bitstream#1 above, which can be called the auxiliary bitstream) and use the superscale decoding network to decode the prior information to obtain the residual probability distribution parameters corresponding to the current image patch.

[0111] If the residual probability distribution parameters corresponding to the current image block are obtained at the decoding end, the quantization parameters corresponding to the current image block can be obtained based on these residual probability distribution parameters.

[0112] In one example, the decoder can decode the second bitstream (such as Bitstream#2 above) corresponding to the current image block based on the residual probability distribution parameters to obtain the quantization residual corresponding to the current image block.

[0113] The decoder can obtain the reconstructed features of the current image patch based on the prior information and the quantization parameters of the current image patch.

[0114] In one example, the decoder can use a context-based prediction model to determine the reconstructed features corresponding to the current image patch based on the prior information and quantization residuals corresponding to the current image patch.

[0115] For example, the decoder can perform context-based prediction based on the prior information of the current image patch and the reconstruction features of the previous image patches to obtain the predicted value corresponding to the current image patch.

[0116] In addition, the decoding end can dequantize the quantization residual corresponding to the current image block to obtain the reconstruction residual corresponding to the current image block.

[0117] Furthermore, the decoding end can obtain the reconstruction features of the current image patch based on the reconstruction residual and the predicted value corresponding to the current image patch.

[0118] Once the overfitting parameters corresponding to the current image patch are obtained, the decoding end can use the synthetic transformation network to obtain the reconstructed image patch corresponding to the current image patch based on the overfitting parameters and reconstruction features corresponding to the current image patch.

[0119] For example, the decoding end can synthesize and transform the reconstructed features corresponding to the current image patch based on the overfitting parameters corresponding to the current image patch to obtain the reconstructed image patch corresponding to the current image patch.

[0120] In one example, the decoder can parse the overfitting parameters corresponding to the current image block from the target bitstream corresponding to the current image block.

[0121] For example, the target bitstream may include a first bitstream, a second bitstream, or a third bitstream. That is, the encoding end can write the overfitting parameters corresponding to the current image block into the first bitstream or the second bitstream corresponding to the current image block, or the encoding end can write the overfitting parameters corresponding to the current image block into another bitstream (i.e., the third bitstream, used to transmit the overfitting parameters).

[0122] The following provides a detailed description of different implementations of the overfitting parameters.

[0123] Example 1

[0124] Overfitting parameters can include the target weight values ​​of the overfitting layers in the decoder network;

[0125] or,

[0126] Overfitting parameters can include the bias values ​​of the overfitting layers in the decoder network;

[0127] or,

[0128] Overfitting parameters can include the target weight values ​​and the target bias values ​​of the overfitting layers in the decoder network;

[0129] The target weight values ​​may include: the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch; or, the weight offset determined based on the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch and the weights of the overfitted layers determined by offline training of the decoder network.

[0130] The bias term values ​​may include: bias terms for each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch; or, bias term offsets determined by comparing the bias terms for each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch with the bias terms for the overfitted layers determined by offline training of the decoder network.

[0131] For example, in order to reduce the amount of data for overfitting parameters, one can choose to overfit the weights or biases of the overfitting layers in the decoder network.

[0132] In one example, considering that the bias terms in the convolutional layers are arguably the most important (more sensitive to a given instance) and fewer than the convolutional weights (i.e., weights), overfitting the bias terms of the overfitting layers of the decoder network can reduce the amount of data for the overfitting parameters while maintaining the overfitting effect.

[0133] In another example, when the number of parameters in the original network layer is small, overfitting weight parameters, or overfitting bias terms and weights, can also be chosen.

[0134] In one example, all network layers in the decoder network can be identified as overfitting layers to optimize the overfitting effect.

[0135] In another example, some network layers in the decoder network can be identified as overfitting layers to reduce the amount of data for overfitting parameters.

[0136] For example, the overfitting parameter may include the bias target value of the overfitting layer in the decoder network, or the overfitting parameter may include the weight target value of the overfitting layer in the decoder network, or the overfitting parameter may include the bias target value and the weight target value of the overfitting layer in the decoder network.

[0137] In one example, the target weight values ​​may include the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch.

[0138] The bias term values ​​may include: bias terms for each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch.

[0139] That is, when the encoder determines the weights and / or biases of the overfitted layers in the decoder network using an online overfitting method, it can send the weights and / or biases of the overfitted layers in the decoder network as overfitting parameters to the decoder.

[0140] In another example, the target weight value may include a weight offset determined based on the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch and the weights of the overfitted layers determined by offline training of the decoder network.

[0141] The bias term values ​​may include: the bias terms of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch, and the bias term offset determined by offline training of the overfitted layers in the decoder network.

[0142] That is, when the encoder determines the weights and / or biases of the overfitted layers in the decoder network using online overfitting, it can determine the weight offsets and / or bias offsets of each overfitted layer in the decoder network based on the determined weights and / or biases of each overfitted layer in the decoder network, as well as the weights and / or biases of the overfitted layers determined by offline training of the decoder network. The encoder then sends the weight offsets and / or bias offsets of each overfitted layer in the decoder network as overfitting parameters to the decoder.

[0143] Accordingly, the decoder can assign values ​​to the weights and / or biases of the overfitted layers in the decoder network based on the target weight values ​​and / or the target bias values; and input the compressed feature information into the assigned decoder network to determine the reconstructed image block corresponding to the current image block.

[0144] For example, during the online training of the overfitted layer in the decoder network using the current image patch, the encoder can determine the loss value based on the difference between the reconstructed image patch output by the decoder network and the current image patch, and optimize and update the relevant parameters (weights and / or bias terms) of the overfitted layer in the decoder network based on the loss value.

[0145] For example, the greater the difference between the reconstructed image patch output by the decoder network and the current image patch, the greater the loss value.

[0146] In one example, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network.

[0147] Accordingly, the overfitting parameters may include the target weight values ​​of the overfitting layers in the synthetic transform network;

[0148] or,

[0149] Overfitting parameters include the bias values ​​of the overfitting layers in the synthetic transform network;

[0150] or,

[0151] Overfitting parameters include the target weight values ​​and the target bias values ​​of the overfitting layers in the synthetic transform network;

[0152] The target weight values ​​may include: the weights of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch; or, a weight offset determined based on the weights of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch and the weights of the overfitted layers determined by offline training of the synthetic transform network.

[0153] The bias term values ​​may include: bias terms of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch; or, bias term offset determined by comparing the bias terms of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch with the bias terms of the overfitted layers determined by offline training of the synthetic transform network.

[0154] Accordingly, the above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on compressed feature information and overfitting parameters can include:

[0155] Based on the prior information and quantization residual corresponding to the current image patch, obtain the reconstructed features corresponding to the current image patch; and,

[0156] Assign values ​​to the weights and / or biases of the overfitted layers in the synthetic transformation network based on the target values ​​of the weights and / or the target values ​​of the biases.

[0157] The reconstructed features are input into the synthetic transformation network to determine the reconstructed image block corresponding to the current image block.

[0158] For example, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include the prior information corresponding to the current image patch and the quantization residual corresponding to the current image patch.

[0159] Once the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can, on the one hand, obtain the reconstructed features corresponding to the current image block based on the prior information and the quantization residual.

[0160] On the other hand, the weights and / or biases of the overfitted layers in the synthetic transformation network can be assigned values ​​based on the target weight values ​​and / or the target bias values.

[0161] For example, if the overfitting parameters can include the target weight values ​​of the overfitting layers in the synthetic transformation network, the weights of the overfitting layers in the synthetic transformation network can be assigned based on these target weight values.

[0162] For example, if the overfitting parameters can include the bias term values ​​of the overfitting layer in the synthetic transform network, the bias term of the overfitting layer in the synthetic transform network can be assigned a value based on these bias term values.

[0163] For example, if the overfitting parameters can include the target values ​​of the weights and biases of the overfitting layer in the synthetic transformation network, the weights and biases of the overfitting layer in the synthetic transformation network can be assigned values ​​based on these target values ​​and biases.

[0164] For example, taking the assignment of bias terms to overfitted layers in a synthetic transform network as an example, if the bias term labels include the bias terms of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch, the decoding end can set the value of the bias term of each overfitted layer in the synthetic transform network to the corresponding bias term label.

[0165] When the bias term labels include the bias terms of each overfitted layer determined by online training of the overfitted layers in the synthetic transform network using the current image patch, and the bias term offset determined by the bias terms of the overfitted layers determined by offline training of the synthetic transform network, the decoding end can determine the actual value of the bias terms of each overfitted layer in the synthetic transform network based on the bias terms of the overfitted layers determined by offline training of the synthetic transform network, and the bias term labels of each overfitted layer, and assign values ​​to the bias terms of each overfitted layer in the synthetic transform network.

[0166] For example, suppose that for any overfitting layer, the bias term offset = the bias term of the overfitting layer determined by online training of each overfitting layer in the synthetic transform network using the current image patch - the bias term of the overfitting layer determined by offline training of the synthetic transform network. Then, when the decoder assigns a value to the bias term of the overfitting layer, for any overfitting layer, the actual value of the bias term = the bias term of the overfitting layer determined by offline training of the synthetic transform network + the bias term offset of the overfitting layer.

[0167] It should be noted that, in the embodiments of this application, online overfitting of the decoder network is not limited to online overfitting of the synthesis transform network. It can also be achieved by online overfitting of other networks or modules. For example, overfitting of the prediction module (such as a context-based prediction module) can be performed. For specific implementation, please refer to the relevant implementation of online overfitting of the synthesis transform network. This embodiment of the application will not elaborate on it here.

[0168] Example 2

[0169] The overfitting parameters mentioned above may include target parameters of the first type of additional modules corresponding to each overfitting layer in the decoder network. The target parameters of the first type of additional modules are determined by the encoder embedding the first type of additional modules in parallel for each overfitting layer in the decoder network and by online training of the relevant parameters of the first type of additional modules using the current image patch.

[0170] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0171] In the decoder network, first-type additional modules are embedded in parallel for each overfitting layer, and the relevant parameters of the first-type additional modules are assigned values ​​based on the target parameters of the first-type additional modules.

[0172] The compressed feature information is input into the first type of additional module and the assigned decoder network to determine the reconstructed image block corresponding to the current image block.

[0173] For example, online overfitting of the decoder network can also be achieved by embedding additional modules (referred to as first-type additional modules in this paper) in parallel for each overfitting layer and training the decoder network with the first-type additional modules embedded online.

[0174] For example, the encoder can embed a first type of additional module in parallel for each overfitting layer in the decoder network, and use the current image patch to train the relevant parameters of the first type of additional module online to obtain the target parameters of the first type of additional module.

[0175] For example, during the online training of the relevant parameters of the first type of additional module, the relevant parameters of the decoder network (such as weights, bias terms, etc.) can be kept unchanged, and the relevant parameters of the first type of additional module embedded in parallel can be trained.

[0176] For example, for any overfitting layer that has a first type of additional module embedded in parallel, the output of the layer can be jointly determined based on the output of the overfitting layer after processing the input, and the output of the first type of additional module after processing the input. For example, it can be the sum of the two outputs.

[0177] For example, during the training of the relevant parameters of the first type of additional module, a loss value can be determined based on the difference between the reconstructed image patch output by the decoder network with the first type of additional module embedded and the current image patch, and the relevant parameters of the first type of additional module can be optimized and updated based on the loss value.

[0178] For example, the greater the difference between the reconstructed image patch output by the decoder network with the first type of additional module and the current image patch, the greater the loss value.

[0179] Accordingly, when the decoding end obtains the compressed feature information of the current image and the overfitting parameters, it can embed the first type of additional module in parallel for each overfitting layer in the decoder network, and assign values ​​to the relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; input the compressed feature information into the decoder network embedded in the first type of additional module and assigned values, and determine the reconstructed image block corresponding to the current image block.

[0180] In one example, the decoder network may include a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network.

[0181] For example, when the decoder network includes a synthetic transform network, a better solution for the synthetic transform network corresponding to the current image patch can be found by performing online overfitting on the synthetic transform network.

[0182] For example, the overfitting parameters may include target parameters of a first type of additional module corresponding to each overfitting layer in the synthetic transform network. The target parameters of the first type of additional module are determined by the encoder by embedding the first type of additional module in parallel for each overfitting layer in the synthetic transform network, and by training the relevant parameters of the first type of additional module online using the current image patch.

[0183] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0184] Based on the prior information and quantization residual corresponding to the current image patch, obtain the reconstructed features corresponding to the current image patch; and,

[0185] In the synthetic transformation network, a first type of additional module is embedded in parallel for each overfitting layer, and the relevant parameters of the first type of additional module are assigned values ​​based on the target parameters of the first type of additional module;

[0186] The reconstructed features are embedded into the first type of additional module, and the synthesized transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0187] For example, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include the prior information corresponding to the current image patch and the quantization residual corresponding to the current image patch.

[0188] Once the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can, on the one hand, obtain the reconstructed features corresponding to the current image block based on the prior information and the quantization residual.

[0189] On the other hand, a first type of additional module can be embedded in parallel for each overfitting layer in the synthetic transformation network, and the relevant parameters of the first type of additional module can be assigned values ​​based on the target parameters of the first type of additional module.

[0190] The decoding end can embed the reconstructed features into the first type of additional module and assign values ​​to the synthetic transformation network to determine the reconstructed image block corresponding to the current image block.

[0191] Example 3

[0192] The overfitting parameters include the target weight values ​​of the first type of overfitting layer in the decoder network, and the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network.

[0193] or,

[0194] The overfitting parameters include the bias target values ​​of the first type of overfitting layer in the decoder network, and the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network.

[0195] or,

[0196] The overfitting parameters include the target weight values ​​and the target bias values ​​of the first type of overfitting layer in the decoder network, as well as the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network.

[0197] The target weight values ​​include: the weights of each first-type overfitting layer determined by online training of the decoder network with embedded first-type additional modules using the current image patch; or, the weight offset determined based on the weights of each first-type overfitting layer determined by online training of the decoder network with embedded first-type additional modules using the current image patch and the weights of the first-type overfitting layers determined by offline training of the decoder network.

[0198] The bias term values ​​include: bias terms of each first type of overfitting layer determined by online training of the decoder network with embedded first type of additional modules using the current image patch; or, bias term offset determined by the bias terms of each first type of overfitting layer determined by online training of the decoder network with embedded first type of additional modules using the current image patch, and the bias terms of the first type of overfitting layer determined by offline training of the decoder network.

[0199] The target parameters are determined by the encoder embedding first-type additional modules in parallel for each second-type overfitting layer in the decoder network, and by training the relevant parameters of the first-type additional modules online using the current image patch.

[0200] For example, for overfitting layers in a decoder network, online overfitting can be implemented in different ways to improve the flexibility of overfitting implementation.

[0201] For example, for a partially overfitted layer (which can be called a first-type overfitted layer), online overfitting can be achieved by training the relevant parameters (weights and / or bias terms) of the first-type overfitted layer online.

[0202] For the other part of the overfitting layer (which can be called the second type of overfitting layer), online overfitting can be achieved by embedding the first type of additional modules in parallel and training the relevant parameters of the first type of additional modules online.

[0203] For example, for the first type of overfitting layer, the corresponding overfitting parameters can be found in the relevant description in Example 1.

[0204] For example, the encoder can embed first-type additional modules in parallel for each second-type overfitting layer in the decoder network, and use the current image patch to train the relevant parameters of each first-type overfitting layer in the decoder network and the relevant parameters of each first-type additional module online to determine the overfitting parameters (target weight values ​​and / or target bias values) of each first-type overfitting layer in the decoder network and the overfitting parameters (target parameters of the first-type additional modules) of each first-type additional module for the current image patch.

[0205] For example, during the online training of the relevant parameters of the first type of overfitting layer and the relevant parameters of the first type of additional module, a loss value can be determined based on the difference between the reconstructed image patch output by the decoder network with the first type of additional module embedded and the current image patch. Based on the loss value, the relevant parameters of the first type of overfitting layer and the relevant parameters of the first type of additional module are optimized and updated.

[0206] For example, the greater the difference between the reconstructed image patch output by the decoder network with the first type of additional module and the current image patch, the greater the loss value.

[0207] Accordingly, after obtaining the compressed feature information of the current image block and the overfitting parameters, the decoder can assign values ​​to the weights and / or biases of the first type of overfitting layer in the decoder network based on the target weight values ​​and / or bias values ​​included in the overfitting parameters; and, in the decoder network, embed the first type of additional modules in parallel for each second type of overfitting layer, and assign values ​​to the relevant parameters of the first type of additional modules based on the target parameters of the first type of additional modules, input the compressed feature information into the decoder network embedded in the first type of additional modules, and determine the reconstructed image block corresponding to the current image block.

[0208] In one example, the decoder network includes a synthetic transform network; a first type of overfitting layer in the decoder network may include a first type of overfitting layer in the synthetic transform network, and a second type of overfitting layer in the decoder network includes a second type of overfitting layer in the synthetic transform network.

[0209] For example, when the decoder network includes a synthetic transform network, a better solution for the synthetic transform network corresponding to the current image patch can be found by performing online overfitting on the synthetic transform network.

[0210] For example, the overfitting parameters include the target weight values ​​of the first type of overfitting layer in the synthetic transformation network, and the target parameters of the first type of additional modules corresponding to each of the second type of overfitting layers in the synthetic transformation network.

[0211] or,

[0212] The overfitting parameters include the bias target values ​​of the first type of overfitting layer in the synthetic transformation network, and the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the synthetic transformation network.

[0213] or,

[0214] The overfitting parameters include the target weight values ​​and the target bias values ​​of the first type of overfitting layer in the synthetic transformation network, as well as the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the synthetic transformation network.

[0215] The target weight values ​​include: the weights of each first-type overfitting layer determined by online training of the synthetic transformation network with embedded first-type additional modules using the current image patch; or, the weight offset determined based on the weights of each first-type overfitting layer determined by online training of the synthetic transformation network with embedded first-type additional modules using the current image patch and the weights of the first-type overfitting layers determined by offline training of the synthetic transformation network.

[0216] The bias term values ​​include: bias terms of each first-type overfitting layer determined by online training of the synthetic transformation network with embedded first-type additional modules using the current image patch; or, bias term offset determined by the bias terms of each first-type overfitting layer determined by online training of the synthetic transformation network with embedded first-type additional modules using the current image patch, and the bias terms of the first-type overfitting layer determined by offline training of the synthetic transformation network.

[0217] The target parameters are determined by the encoder by embedding first-type additional modules in parallel for each second-type overfitting layer in the synthetic transformation network, and by training the relevant parameters of the first-type additional modules online using the current image patch.

[0218] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0219] Based on the prior information and quantization residual corresponding to the current image patch, obtain the reconstructed features corresponding to the current image patch; and,

[0220] Based on the target values ​​of the weights and / or the target values ​​of the biases, the weights and / or biases of the first type of overfitting layer in the synthetic transformation network are assigned values; and, for each second type of overfitting layer in the synthetic transformation network, the first type of additional modules are embedded in parallel, and the relevant parameters of the first type of additional modules are assigned values ​​based on the target parameters of the first type of additional modules.

[0221] The reconstructed features are embedded into the first type of additional module, and the synthesized transformation network after assignment determines the reconstructed image block corresponding to the current image block.

[0222] For example, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include the prior information corresponding to the current image patch and the quantization residual corresponding to the current image patch.

[0223] Once the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can, on the one hand, obtain the reconstructed features corresponding to the current image block based on the prior information and the quantization residual.

[0224] On the other hand, the weights and / or biases of the first type of overfitting layer in the synthetic transformation network can be assigned values ​​based on the target weight values ​​and / or the target bias values; and, a first type of additional module can be embedded in parallel for each second type of overfitting layer in the synthetic transformation network, and the relevant parameters of the first type of additional module can be assigned values ​​based on the target parameters of the first type of additional module.

[0225] Furthermore, the decoding end can embed the reconstructed feature input into the first type of additional module and the assigned synthetic transformation network to determine the reconstructed image block corresponding to the current image block.

[0226] In one example, the first type of additional module mentioned above can be a low-rank matrix.

[0227] For example, to reduce the amount of data for overfitting parameters and improve online training efficiency, the first type of additional module mentioned above can be a low-rank matrix. The target parameter of the first type of module can be the target parameter of the low-rank matrix.

[0228] In one example, if the network (such as the decoder network or synthesis transform network mentioned above) includes different types of overfitting layers, such as the first type of overfitting layer and the second type of overfitting layer mentioned above, the decoder also needs to obtain the overfitting layer type identifier corresponding to the current image block during the process of obtaining the reconstructed image block of the current image block.

[0229] For example, the overfitting layer type identifier is used to indicate whether the overfitting layer is a first type of overfitting layer or a second type of overfitting layer.

[0230] Example 4

[0231] The overfitting parameters include the target inputs of the second type of additional modules corresponding to each overfitting layer in the decoder network. The target inputs of the second type of additional modules are determined by the encoder embedding the second type of additional modules in parallel for each overfitting layer in the decoder network and training the inputs of the second type of additional modules online. The relevant parameters of the second type of additional modules are determined by training offline.

[0232] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0233] In the decoder network, a second type of additional module is embedded in parallel for each overfitted layer;

[0234] The compressed feature information is used as the input to the overfitting layer in the decoder network, and the target input is used as the input to the second type of additional module. The decoder network with embedded second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

[0235] For example, in order to reduce the amount of data for overfitting parameters, the decoder network can be overfitted online by embedding a second type of additional module in parallel into each overfitting layer in the decoder network and training the input of the second type of additional module online, thereby finding a better solution for the decoder corresponding to the current image patch.

[0236] For example, the encoder can embed a second type of additional module in parallel for each overfitting layer in the decoder network, and use the current image patch to train the input of the second type of additional module online to obtain the target input of the second type of additional module. The target input is then used as the overfitting parameter and sent to the decoder.

[0237] For example, during online training of the input to the second type of additional module, a loss value can be determined based on the difference between the reconstructed image patch output by the decoder network with the second type of additional module embedded and the current image patch, and the input to the second type of additional module can be optimized based on the loss value.

[0238] For example, the greater the difference between the reconstructed image patch output by the decoder network with the second type of additional module and the current image patch, the greater the loss value.

[0239] Accordingly, when the decoding end obtains the compressed feature information corresponding to the current image block and the overfitting parameters, it can embed a second type of additional module in parallel for each overfitting layer in the decoder network. The compressed feature information is used as the input of the overfitting layer in the decoder network, and the labeled input is used as the input of the second type of additional module. The decoder network with the embedded second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

[0240] In one example, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network may include the overfitting layer in the synthetic transform network.

[0241] The overfitting parameters may include the target inputs of the second type of additional modules corresponding to each overfitting layer in the synthetic transform network. The target inputs of the second type of additional modules are determined by the encoder by embedding the second type of additional modules in parallel for each overfitting layer in the synthetic transform network and by training the inputs of the second type of additional modules online. The relevant parameters of the second type of additional modules are determined by training offline.

[0242] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0243] Based on the prior information and quantization residual corresponding to the current image patch, obtain the reconstructed features corresponding to the current image patch; and,

[0244] In the synthetic transformation network, a second type of additional module is embedded in parallel for each overfitting layer;

[0245] Using the reconstructed features as input to the overfitting layer in the synthetic transformation network and the target input as input to the second type of additional module, the synthetic transformation network with embedded second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

[0246] For example, when the decoder network includes a synthetic transform network and the overfitting layer in the decoder network includes a synthetic transform network, the compressed feature information may include the prior information corresponding to the current image patch and the quantization residual corresponding to the current image patch.

[0247] Once the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can, on the one hand, obtain the reconstructed features corresponding to the current image block based on the prior information and the quantization residual.

[0248] On the other hand, a second type of additional module can be embedded in parallel for each overfitting layer in the synthetic transformation network.

[0249] Furthermore, the decoding end can use the reconstructed features as the input to the overfitting layer in the synthetic transformation network and the target input as the input to the second type of additional module. By using the synthetic transformation network with the second type of additional module embedded, the reconstructed image block corresponding to the current image block can be determined.

[0250] In one example, the overfitting layer in the above embodiments (any one of Embodiments 1 to 4) includes some or all of the network layers in the network.

[0251] For example, the overfitting layer in the above embodiments may include some network layers to reduce the amount of data for the overfitting parameters. Alternatively, the overfitting layer in the above embodiments may include all network layers to optimize the online overfitting effect.

[0252] As an example, in cases where the overfitting layer includes a portion of the network layers, the above method may further include:

[0253] Obtain the overfitting layer identifier corresponding to the current image patch; where the overfitting layer identifier is used to indicate the overfitting layer in the network;

[0254] The overfitting layer identifier includes the layer number of the overfitting layer, or the number of overfitting layers N; where the number of overfitting layers N is used to indicate that the first N layer numbers in the offline determined layer number list are overfitting layers.

[0255] For example, in the case where the overfitting layer includes a portion of the network layers, the encoder can also send the overfitting layer identifier corresponding to the current image patch to the decoder.

[0256] During the process of obtaining the reconstructed image block of the current image block, the decoding end can identify the overfitting layer in the network by obtaining the overfitting layer identifier corresponding to the current image block.

[0257] In one example, the overfitting layer identifier can include the layer number of the overfitting layer. That is, the encoder can send the layer number of each overfitting layer corresponding to the current image patch to the decoder. The decoder can determine the overfitting layer in the network based on the layer number of each overfitting layer corresponding to the current image patch.

[0258] In another example, the overfitting layer identifier includes the number of overfitting layers N; where the number of overfitting layers N is used to indicate the layer numbers of the first N layers in the offline determined layer number list that are overfitting layers.

[0259] For example, the encoder and decoder can pre-negotiate a list of layer numbers, which is generated by sorting the numbers of each network layer in the network according to certain rules.

[0260] For example, the list of layer numbers can be generated by arranging the layer numbers of each network layer in ascending order.

[0261] For example, the list of layer numbers can be generated by arranging the energy of each network layer in descending order.

[0262] For example, the energy of each network layer can be determined by training each network layer in the network and calculating the change energy of the network layer, that is, the sum of squares of the differences between the trained parameters and the original parameters.

[0263] For example, if the encoder and decoder can negotiate the layer number list in advance, the encoder can determine the network layers corresponding to the first N layer numbers in the layer number list as overfitting layers and send the number N of overfitting layers to the decoder.

[0264] The decoder can determine the overfitting layers in the network by selecting the first N layer numbers in the layer number list as the overfitting layers based on the number N of overfitting layers.

[0265] It should be noted that the overfitting layer in the network can also be a fixed layer determined offline, for example, determined in advance by the encoder and decoder. In this case, it is not necessary to send the overfitting layer identifier separately.

[0266] Example 5

[0267] The overfitting parameters include the target input of the supplementary network, which is determined by the encoder through online training based on the output of the decoder network and the output of the supplementary network. The relevant parameters of the supplementary network are determined through offline training.

[0268] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on compressed feature information and overfitting parameters may include:

[0269] The compressed feature information is used as the input to the overfitting layer in the decoder network, and the target input is used as the input to the additional network. The output of the decoder network and the output of the additional network are used to determine the reconstructed image patch corresponding to the current image patch.

[0270] For example, online overfitting of the decoder network can be achieved by adding an additional network that is relatively independent of the decoder network and training the input of the additional network online.

[0271] For example, the encoder can train the input of the supplementary network online based on the output of the decoder network and the output of the supplementary network, determine the target input of the supplementary network, and send the target input as an overfitting parameter to the decoder.

[0272] Accordingly, after obtaining the compressed feature information of the current image block and the overfitting parameters, the decoder can use the compressed feature information as the input to the overfitting layer in the decoder network and the target input as the input to the additional network. Using the output of the decoder network and the output of the additional network, the decoder can determine the reconstructed image block corresponding to the current image block.

[0273] In one example, the decoder network includes a synthesis transform network.

[0274] The above-mentioned method of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network, based on compressed feature information and overfitting parameters, may include:

[0275] Based on prior information and quantization residuals, the reconstructed features corresponding to the current image patch are obtained;

[0276] Using the reconstructed features as input to the overfitting layer in the synthetic transform network and the target input as input to the additional network, the reconstructed image patch corresponding to the current image patch is determined by using the output of the synthetic transform network and the output of the additional network.

[0277] For example, when the decoder network includes a synthesis transform network, the compressed feature information may include the prior information corresponding to the current image patch and the quantization residual corresponding to the current image patch.

[0278] Once the decoding end obtains the compressed feature information of the current image block and the overfitting parameters corresponding to the current image block, it can obtain the reconstructed features corresponding to the current image block based on the prior information and the quantization residual.

[0279] The decoder can use the reconstructed features as input to the overfitting layer in the synthetic transform network and the target input as input to the additional network. By using the output of the synthetic transform network and the output of the additional network, the reconstructed image patch corresponding to the current image patch can be determined.

[0280] In some embodiments, the decoding method provided in this application may further include:

[0281] Obtain the overfitting scheme identifier corresponding to the current image patch; different overfitting scheme identifiers correspond to different overfitting schemes.

[0282] The above method for obtaining the overfitting parameters corresponding to the current image patch may include:

[0283] Based on the overfitting scheme identifier, the overfitting parameters corresponding to the current image block are parsed from the target bitstream corresponding to the current image block; wherein, the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

[0284] For example, during the process of determining overfitting parameters, the encoder can provide multiple candidate overfitting schemes, such as at least two overfitting schemes in embodiments 1 to 5 above. The encoder can determine the overfitting parameters corresponding to different candidate overfitting schemes, select an optimal overfitting scheme, and send the identifier of the optimal overfitting scheme and the overfitting parameters corresponding to the overfitting scheme to the decoder.

[0285] Accordingly, the decoding end can obtain the overfitting scheme identifier corresponding to the current image block, determine the overfitting scheme currently used based on the overfitting scheme identifier, and then parse the overfitting parameters corresponding to the current image block from the target bitstream corresponding to the current image block based on the overfitting scheme identifier.

[0286] For example, the target bitstream may include a first bitstream, a second bitstream, or a third bitstream.

[0287] For example, after determining the overfitting scheme and obtaining the corresponding overfitting parameters, the decoding end can obtain the reconstructed image block corresponding to the current image block based on the obtained overfitting parameters. The specific implementation of this can be found in the relevant description in the above embodiments.

[0288] This application proposes an encoding method in its embodiments; see [link to relevant documentation]. Figure 3 The diagram shown illustrates the flowchart of this encoding method, which can be applied to the encoding end (also known as a video encoder). This method may include:

[0289] Step S300: Obtain the compressed feature information of the current image block and write the compressed feature information into the bitstream corresponding to the current image block.

[0290] In this embodiment of the application, during the encoding process of the current image block, the encoding end can obtain the compressed feature information of the current image block and write the compressed feature information into the bitstream corresponding to the current image block.

[0291] For example, the encoding end can send the bitstream to the decoding end, which then performs image decoding processing in accordance with the manner described in the above embodiments.

[0292] In one example, the encoder can obtain the probability distribution parameters of the residual of the current image patch. On the one hand, based on the probability distribution parameters, it can obtain the prior information corresponding to the current image patch and write the prior information into the bitstream. On the other hand, the encoder can quantize the probability distribution parameters and write the quantized residual into the bitstream based on the quantized probability distribution parameters.

[0293] Step S310: Obtain the overfitting parameters corresponding to the current image block and write the overfitting parameters into the bitstream corresponding to the current image block.

[0294] In this embodiment of the application, in order to improve the image encoding and decoding performance, when applying the trained model, the parameters of the trained model can be adjusted by online fitting to obtain a better solution.

[0295] Correspondingly, online overfitting can be performed on the decoder network at the encoder end to adjust the model parameters of the decoder network and obtain model parameters that are more suitable for the current image, that is, to obtain a better solution for the current image.

[0296] The encoder can write the overfitting parameters determined by online overfitting of the decoder network into the bitstream and send it to the decoder.

[0297] For example, the specific implementation of determining the overfitting parameters at the encoding end can be found in the relevant descriptions in the above embodiments, and will not be repeated here in the embodiments of this application.

[0298] It should be noted that since the structure of the encoder network includes the structure of the decoder network, the encoder will also perform the above-mentioned operation of "obtaining the reconstructed image block corresponding to the current image block using the decoder network based on the compressed feature information and the overfitting parameters" during the image encoding process. The specific implementation process can be found in the relevant description in the above-mentioned method embodiment on the decoder side.

[0299] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, the technical solutions provided in the embodiments of this application are described below with reference to specific examples.

[0300] The decoder network includes a synthesis transform network. By performing online overfitting on the synthesis transform network, a better solution for the current image patch is found.

[0301] The model structure of the synthetic transformation network will be explained below.

[0302] Please see Figure 4A and Figure 4B Based on the complexity of the model structure, the model structures of the two commonly used synthetic transformation networks can be respectively as follows: Figure 4A and Figure 4B As shown.

[0303] like Figure 4A The diagram shown illustrates the model structure of the high-level synthesis transform network. The high-level synthesis transform network consists of the following layers in sequence: ResBlock (RB layer), ResBlock layer, Conv-1 layer, Crop layer, ResAU layer, Conv-1 layer, Crop layer, ResAU layer, Conv layer, PixelShuffle layer, RNAB layer, Crop layer, ResAU layer, Conv-1 layer, and Crop layer. The ResBlock layer represents the residual block. A structural diagram of the ResBlock layer can be found in [reference needed]. Figure 5B As shown. Conv -1 The first layer is a deconvolutional layer, used to upsample the input data or restore its original size. (3, 3, C, C) means the height and width of the convolutional kernel are 3 pixels. The first C represents the number of channels in the input feature map, and the second C represents the number of channels in the output feature map. The upward arrow of 2 indicates that the feature is upsampled by a factor of 2. The Crop layer is a cropping layer used to perform feature cropping operations. (H) in W in H in ,2,2) in and W in These represent the height and width of the input feature map, respectively. The two 2s indicate that the cropped region is 2x2. The ResAU layer is an activation layer; a schematic diagram of the ResAU layer can be found in [link to diagram]. Figure 5AAs shown in (3, 3, noTAHN), the first 3 indicates that the number of channels in the input feature map is 3, the second 3 indicates that the size of the convolution kernel is 3*3, and noTAHN indicates that the TAHN (Temporal Adaptive Hard Negative Mining) function is not used in the attention module. The Conv layer is a convolutional layer used to perform convolution operations on the features. The PixelShuffle layer is a pixel resampling layer that achieves upsampling by rearranging channels and reconstructing pixels at the pixel level. In (4C, C), 4C indicates that the number of channels in the input feature map is 4C, C indicates that the number of channels in the output feature map is C, and the up arrow of 2 indicates that the features are upsampled by a factor of 2. The RNAB layer is a residual nonlocal attention block. A schematic diagram of the RNAB layer can be found in [reference needed]. Figure 5C As shown, the RNAB layer may include the RB layer, the Conv layer, and the sigmoid layer.

[0304] like Figure 4B The diagram shown illustrates the model structure of the base gear synthesis transform network. The base gear synthesis transform network consists of a LightResBlock layer, a Conv layer, and so on. -1 Layer, Crop layer, ResAU layer, Conv -1 The convolutional layers are: Layer 1, Crop layer, ResAU layer, Conv layer, ResAU layer, Conv layer, PixelShuffle layer, and Crop layer. LightResBlock is a lightweight residual block, and 3×3 indicates that the convolutional kernel size is 3x3.

[0305] Please see Figure 5A As shown, the ResAU layer is the activation layer. A schematic diagram of the ResAU layer structure can be found in [reference needed]. Figure 5A As shown, LeakyReLU represents the activation operation, Conv 1*1 represents the convolution operation of features using a 1*1 convolution kernel, and tanh represents the hyperbolic tangent operation.

[0306] Please see Figure 5B The RB (Residual Blocks) layer consists of residual blocks. A schematic diagram of the RB layer structure can be seen as follows: Figure 5B As shown, the RB layer can include a Conv layer, LeakyReLU, and a Conv layer, with a 3*3 convolution kernel used in the Conv layer.

[0307] Please see Figure 5C The RNAB layer is a residual nonlocal attention block. A schematic diagram of the RNAB layer structure can be seen as follows: Figure 5CAs shown, the RNAB layer can include an RB layer, a Conv layer, and a sigmoid layer. In the Conv layer, the up arrow indicating a 2x upsampling of the features. The sigmoid layer is used for feature mapping, such as using the sigmoid function to map features to a range between 0 and 1.

[0308] To ensure consistency between encoding and decoding, the "better solution" after overfitting at the encoding end needs to be written into the bitstream and passed to the decoding end.

[0309] For example, overfitting methods at the encoder end can include four categories:

[0310] The first type does not add any additional modules, but learns the synthesis transformation module online, and passes the "better solution" (weight W1 and / or bias B1) of the parameters (weights W and / or bias terms B) in the synthesis transformation module to the decoding end, and obtains the "better solution" of the final result through the "better solution" of the parameters.

[0311] The second type involves embedding additional modules (such as the first type of additional modules mentioned above) in parallel within the current synthesis transformation module. The additional modules are trained online, and their relevant parameters (weights W1) are passed to the decoding end. The learned "solution" (x1) of the additional modules is added to the solution of the original synthesis transformation module to obtain a "better solution" for the final result.

[0312] The third type involves embedding additional modules (such as the second type of additional modules mentioned above) in parallel with the current synthesis transformation module. The additional modules are trained offline, and the input of the network is trained online. The input is passed to change the output of the synthesis transformation network. The learned "solution" (x1) of the additional modules is added to the solution of the original synthesis transformation module to obtain a "better solution" for the final result.

[0313] The fourth type involves adding an additional network. This additional network is trained offline, and its input is trained online. This input is then passed to change the output of the synthesis transformation module. The learned "solution" (x1) of the additional network is added to the solution of the original synthesis transformation module to obtain a "better solution" for the final result.

[0314] Schematic diagrams of various overfitting methods can be seen as follows: Figure 6 As shown, the fourth category.

[0315] The implementation process of the decoding and encoding ends will be explained below.

[0316] Example 6

[0317] A decoding method process may include:

[0318] like Figure 7As shown, the decoding end will receive one or two bitstream segments (i.e., Bitstream#1; or, Bitstream#1 and Bitstream#2).

[0319] Step 1: Parse the compressed image feature information from Bitstream#1.

[0320] Step 2: Decode the overfitting parameters from Bitstream#1 or Bitsstream#2;

[0321] Step 3: Input the compressed feature information and overfitting parameters into the decoder network to obtain the reconstructed image.

[0322] Example 7

[0323] Taking the aforementioned decoder network, including the synthesis transform network, as an example, a decoding method flow may include:

[0324] like Figure 8 As shown, the decoding end will accept two or three bitstream segments (i.e., Bitstream#1 and Bitstream#2; or, Bitstream#1, Bitstream#2 and Bitstream3).

[0325] Step 1: Parse z_hat from Bitstream#1 (auxiliary bitstream) and obtain the parameters (i.e. probability distribution parameters) σ of the residual probability distribution model through z_hat.

[0326] Step 2: Perform entropy decoding from Bitsstream#2 and obtain the quantization residual r_q by combining σ.

[0327] Step 3: The quantized residuals r_q and z_hat are used by the context-based prediction module to obtain the reconstructed feature y_hat.

[0328] Step 4: Decode the overfitting parameters from Bitsstream#1, Bitsstream#2, or Bitsstream#3. These overfitting parameters are applied to the synthesis transform module, additional module, or additional network. Input the reconstruction feature y_hat to obtain the reconstructed image x_hat.

[0329] Example 8

[0330] Taking the aforementioned decoder network, including the synthesis transform network, as an example, an encoding method may include:

[0331] like Figure 9As shown, the encoder can obtain the probability distribution parameter σ of the residual of the current image patch. On one hand, based on this probability distribution parameter σ, it obtains the super-prior information z_hat corresponding to the current image patch and writes z_hat into Bitstream#1. On the other hand, the encoder can quantize this probability distribution parameter and write the quantized residual r_q into Bitstream#2 based on the quantized probability distribution parameter. Overfitting parameters are applied to the synthesis transform module, additional module, or additional network and written into Bitstream#1, Bitstream#2, or Bitstream#3.

[0332] The overfitting schemes for the encoding and decoding ends are explained below with reference to specific examples.

[0333] Example 9, Overfitting Scheme 1 at the Encoding End

[0334] Overfitting the bias term (B) of the current synthetic transform network can be problematic. Since the bias term in the convolutional layer is arguably the most important (more sensitive to a given instance) and less important than the convolutional weights, overfitting the bias term is acceptable. However, if the number of network layers is too high (e.g., exceeding a preset threshold), overfitting the bias term of all network layers will result in unsustainable bitrate. Therefore, it is preferable to overfit the bias term of only a subset of layers (i.e., the overfitted layers).

[0335] For example, there are two ways to select the overfitting layer:

[0336] First, the first layer of the network is mainly related to more general features, while the last layer is related to details. Therefore, in order to improve the reconstruction of details, the bias term of overfitting to the last L convolutional layers is limited.

[0337] Second, since some synthetic transformation networks are complex and have skip connections, some of the earlier network layers may still be related to details. By training all the network layers of the synthetic transformation network, the change energy of the network layers is calculated, which is the sum of the squares of the differences between the parameters of the trained network layers and the original parameters. The layers with larger energy are the L convolutional layers selected for overfitting.

[0338] For example, once L is selected, the overfitting layers are determined accordingly. To reduce computational complexity, the overfitting model (NET) is defined as the layer from the first layer of the selected L-layer network to the last layer of the synthetic transformation network, such as... Figure 10As shown, then use the Adam (Adaptive Moment Estimation) optimizer in PyTorch (an open-source machine learning library based on Python), set the learning rate to 0.003, and perform 100 online iterations. If better coding performance than the original network parameters is obtained (i.e., smaller loss, where lambda = 5, 16, 35, 110, 220, corresponding to the choices when bpp = 0.06, 0.12, 0.25, 0.50, 0.75 respectively), then write the difference of the bias terms of the L layers learned online into the code stream and transfer it to the decoding end. The specific online overfitting algorithm is shown in Algorithm 1.

[0339]

[0340] Among them, mssim(X_hat, X) is used to compare the structural similarity between X_hat and X (i.e., the reconstructed image block and the original image block). mssim (Mean Structural Similarity Index) is a metric for measuring image similarity and is used to compare the structural similarity between two images.

[0341] loss < loss_best: loss_best = loss, B_best = B, F = 1 means that when the loss calculated during the iteration is less than the current best loss (i.e., loss_best, with the initial value being the original loss), update the current best loss to the loss calculated during this iteration, update the bias terms of the overfitting layer to the bias terms determined during this iteration, and set the value of the algorithm start flag F to 1 (i.e., adopt the online overfitting algorithm).

[0342] It should be noted that when the value of the algorithm start flag F is 0, it can be determined that the online overfitting algorithm is not adopted. In this case, an offline-trained synthesis transformation network can be used to generate the reconstructed image block, and there is no need to use overfitting parameters (nor transfer overfitting parameters).

[0343] Example 10, Encoding End Overfitting Scheme 2

[0344] Add an additional module to achieve overfitting, and use a low-rank matrix to assist the original synthesis transformation network to achieve overfitting. That is, add a low-rank matrix in parallel for each selected overfitting layer, and change the output of the overfitting layer in the original synthesis transformation network by online learning the low-rank matrix, thereby achieving overfitting of the entire synthesis transformation network.

[0345] For example, the selection method for the overfitting layer is the same as that in Scheme 1, except that the parameters to be written into the bitstream are the parameters of the low-rank matrix, while the relevant parameters of each network layer in the synthetic transformation network remain unchanged.

[0346] For example, a schematic diagram of the overfitting model NET of this scheme is shown below. Figure 11 As shown in Algorithm 2, the specific online overfitting algorithm is as follows.

[0347]

[0348] Example 11, Overfitting Scheme 3 at the Encoding End

[0349] Add an additional module to achieve overfitting, using a parallel mini-network to achieve overfitting. The parallel mini-network needs to be trained offline. The purpose of this network is to assist the original synthetic network in achieving overfitting by learning the input of the network online (with less data) (each layer of the parallel network is different from the original network).

[0350] For example, this approach does not require selecting overfitting layers; all layers have a parallel layer and require offline training in advance.

[0351] For example, a schematic diagram of the overfitting model NET of this scheme is shown below. Figure 12 As shown in the figure, the specific online overfitting algorithm is shown in Algorithm 3.

[0352]

[0353] Example 12, Overfitting Scheme 4 at the Encoding End

[0354] An additional network is added to achieve overfitting. This additional network needs to be trained offline. The purpose of this additional network is to assist the original synthetic network in achieving overfitting by learning the input of the network online (with less data).

[0355] For example, a schematic diagram of the overfitting model NET of this scheme is shown below. Figure 13 As shown, the specific online overfitting algorithm is similar to that of Algorithm 3.

[0356] It should be noted that the number of network layers in the additional network can be the same as or different from the number of network layers in the original synthetic transform network. Figure 13 (The same applies to China and Israel).

[0357] Example 13, Overfitting Scheme 5 at the Encoding End

[0358] This scheme combines Scheme 1 and Scheme 2. Specifically, some layers (such as the first type of overfitting layer mentioned above) use bias terms from the original network layer to overfit, while other layers (such as the second type of overfitting layer mentioned above) use additional modules of low-rank matrices to assist overfitting. A schematic diagram of the overfitting model NET for this scheme is shown below. Figure 14 As shown.

[0359] Example 14, Overfitting Scheme 6 at the Encoding End

[0360] This approach involves selecting from multiple options. Specifically, options 1-5 (i.e., candidate overfitting options include options 1-5) are attempted at the encoding stage, and the final option is determined by comparing the magnitude of their common loss function.

[0361] For example, the encoding end can send the overfitting scheme identifier and the corresponding overfitting parameters corresponding to the final scheme to the decoding end.

[0362] It should be noted that in the embodiments of this application, there may be multiple different combinations for any overfitting scheme. For example, for scheme 1, there are multiple choices for the overfitting layer, and each choice can be determined by full traversal, but the decoding method of this combination is consistent.

[0363] Example 15: Overfitting Scheme 1 at the Decoding End (corresponding to Overfitting Scheme 1 at the Encoding End)

[0364] Depending on the method used, the overfitting layer in the original network can be selected, and the parameters in the overfitting layer (i.e., the overfitting parameters) can be passed. You can choose to pass the bias term B (i.e., the target value of the bias term mentioned above) or the weight W (i.e., the target value of the weight mentioned above), or you can choose to pass both W and B.

[0365] The required information is encoded into a bitstream, and the relevant information syntax includes:

[0366] 1. Identification of overfitted layers (choose one of the following methods)

[0367] Method 1: Directly pass the layer index list layer_list of the overfitted layers.

[0368] Example: layer_list = [9, 10], meaning the layer numbers of the overfitted layers are 9 and 10 respectively.

[0369] Method 2: With the synthesis transform network fixed, a layer number list layer_list is determined offline. The determination method can refer to the relevant description in the overfitting scheme 1 of the encoder. It can be arranged in order or according to energy. In this case, the overfitting layer can be determined by passing the layer number layer_num.

[0370] Example: layer_list = [3, 5, 4, 2, 1, 0], layernum = 3, then the overfitting layer is [3, 5, 4].

[0371] 2. Overfitting parameters (choose one of the following methods)

[0372] Method 1: Pass the weights W and biases B of each overfitting layer, and assign values ​​directly based on the overfitting parameters during decoding.

[0373] Method 2: Pass the difference in correlation coefficients of each overfitting layer (i.e., the above-mentioned bias term offset and / or weight offset) W_offset, or B_offset, or W_offset and B_offset. During decoding, use the original parameters of the synthetic transformation network (if offline training is successful) plus the difference obtained from decoding to determine the final network parameters.

[0374] 3. Other optional parameters: If the rules are confirmed in advance by both the encoder and decoder, these parameters do not need to be passed.

[0375] Parameter 1: If entropy coding is used to encode the overfitting parameters of each layer, then a list of distribution coefficients of each layer parameter sigma_list or a distribution coefficient sigma of all parameters needs to be passed.

[0376] Parameter 2: A list of the number of parameters passed to each layer, num_list;

[0377] Parameter 3: Identifiers for the weighting coefficients and bias term coefficients.

[0378] Example 16: Decoding-side overfitting scheme 2 (corresponding to encoding-side overfitting scheme 2)

[0379] Depending on the method used, overfitting layers in the original network are selected, and the parameters of the low-rank matrix are passed. The required information is then encoded into the bitstream; the relevant syntax includes:

[0380] 1. Identification of overfitted layers (same as in Example 15).

[0381] 2. Parameters related to low-rank matrices. For example, passing B and A of low-rank matrices.

[0382] 3. Other optional parameters: If the rules are confirmed in advance by both the encoder and decoder, these parameters do not need to be passed:

[0383] Parameter 1: rank_list of the ranks of the low-rank matrix. If all layers use the same rank, only one rank needs to be passed.

[0384] Parameter 2: If entropy coding is used to encode the low-rank matrices of each layer, then a list of distribution coefficients of each layer parameter sigma_list or a distribution coefficient sigma of all parameters needs to be passed.

[0385] Parameter 3: A list of the number of parameters for each layer, num_list.

[0386] Example 17: Decoding-side overfitting scheme 3 (corresponding to encoding-side overfitting scheme 3)

[0387] Pass the input parameters of the additional module (such as the target input mentioned above). Encode the required information into the bitstream; the relevant syntax includes:

[0388] 1. Input parameters: pass y2.

[0389] 2. Other optional parameters: If the rules are confirmed in advance by both the encoder and decoder, these parameters do not need to be passed:

[0390] Parameter 1: If entropy coding is used to encode y2, then the distribution coefficient sigma needs to be passed.

[0391] Parameter 2: The number of parameters passed to y2, num.

[0392] Example 18: Decoding-side overfitting scheme 4 (corresponding to encoding-side overfitting scheme 4)

[0393] The input parameters of the additional network (such as the target input mentioned above) are passed. The required information is then encoded into the bitstream; the relevant syntax includes:

[0394] 1. Input parameters: pass y2.

[0395] 2. Other optional parameters: If the rules are confirmed in advance by both the encoder and decoder, these parameters do not need to be passed:

[0396] Parameter 1: If entropy coding is used to encode y2, then the distribution coefficient sigma needs to be passed.

[0397] Parameter 2: The number of parameters passed to y2, num.

[0398] Example 19: Decoding-side overfitting scheme 5 (corresponding to encoding-side overfitting scheme 5)

[0399] Depending on the method used, overfitting layers in the original network are selected, and their relevant parameters and low-rank matrices are passed. The required information is then encoded into the bitstream; the relevant syntax includes:

[0400] 1. Identification of overfitted layers (same as in Example 15).

[0401] 2. The identifier type of the network layer and low-rank matrix in the overfitted original network (i.e., the overfitting layer type identifier mentioned above, used to indicate whether the overfitting layer is the first type of overfitting layer or the second type of overfitting layer).

[0402] For example: type=0, overfitting the original network layer (i.e., the overfitting layer is a first-type overfitting layer); type=1, overfitting the low-rank matrix (i.e., the overfitting layer is a second-type overfitting layer).

[0403] 3. Overfit the original network parameters and low-rank matrix parameters (see Examples 15 and 16).

[0404] 4. Other optional transmission parameters: If the rules are confirmed in advance by both the encoding and decoding ends, these parameters may not be transmitted (see Examples 15 and 16).

[0405] Example 20: Decoding-side overfitting scheme 6 (corresponding to encoding-side overfitting scheme 6)

[0406] If one of Examples 1 to 4 is selected, then according to the syntax for passing relevant information in the examples above, only an additional scheme identifier index needs to be passed.

[0407] For example, if index=1, it means that scheme 1 is used, and the parameters passed are the relevant parameters of the overfitting layer in the original synthetic transformation network.

[0408] It should be noted that, in the embodiments of this application, the above embodiments can also be combined with each other to obtain new embodiments, and these new embodiments should also fall within the protection scope of this application.

[0409] Furthermore, in the embodiments of this application, the specific implementations of the decoding end embodiment and the specific implementations of the encoding end embodiment can refer to each other.

[0410] For example, since the encoder network structure includes the decoder network structure, the encoder end will also perform the above-mentioned operation of "obtaining the reconstructed image block corresponding to the current image block using the decoder network based on the compressed feature information and the overfitting parameters" during the image encoding process. The specific implementation process can be referred to the relevant description in the above-mentioned method embodiment on the decoder end side.

[0411] Based on the same concept as the above method, the decoding device (also known as a video decoder) provided in this application embodiment, from a hardware perspective, its hardware architecture diagram can be found in [reference needed]. Figure 15As shown, it includes: a processor 1501 and a machine-readable storage medium 1502, the machine-readable storage medium 1502 storing machine-executable instructions that can be executed by the processor 1501; the processor 1501 is used to execute the machine-executable instructions to implement the decoding method described in the above embodiments of this application.

[0412] Based on the same concept as the above method, the encoding end device (also known as a video encoder) provided in this application embodiment, from a hardware perspective, its hardware architecture diagram can be found in [reference needed]. Figure 16 As shown, it includes: a processor 1601 and a machine-readable storage medium 1602, the machine-readable storage medium 1602 storing machine-executable instructions that can be executed by the processor 1601; the processor 1601 is used to execute the machine-executable instructions to implement the encoding method described in the above embodiments of this application.

[0413] Based on the same concept as the methods described above, this application provides an electronic device. It includes a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor executes the machine-executable instructions to implement the decoding or encoding methods disclosed in the examples above.

[0414] Based on the same application concept as the above methods, embodiments of this application also provide a machine-readable storage medium storing a plurality of computer instructions. When the computer instructions are executed by a processor, they can implement the methods disclosed in the above examples of this application, such as the decoding method or encoding method disclosed in the above examples.

[0415] Based on the same application concept as the above method, this application embodiment also provides a computer application that, when executed by a processor, can implement the decoding method or encoding method disclosed in the above examples of this application.

[0416] Based on the same concept as the above method, this application also proposes a decoding device, which can be applied to a decoding end, and the decoding device may include:

[0417] The acquisition unit is used to acquire the compressed feature information of the current image block; and to acquire the overfitting parameters corresponding to the current image block.

[0418] The decoding unit is used to obtain the reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameters.

[0419] In one possible implementation, the decoder network includes a synthesis transform network;

[0420] The acquisition unit is specifically used to acquire the prior information corresponding to the current image patch, and acquire the residual probability distribution parameter corresponding to the current image patch based on the prior information; and acquire the quantization residual corresponding to the current image patch based on the residual probability distribution parameter.

[0421] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; and to obtain the reconstructed image patch corresponding to the current image patch using the synthetic transformation network based on the overfitting parameters and the reconstruction features.

[0422] In one possible implementation, the overfitting parameter includes the target weight values ​​of the overfitting layer in the decoder network;

[0423] or,

[0424] The overfitting parameters include the bias target values ​​of the overfitting layers in the decoder network;

[0425] or,

[0426] The overfitting parameters include the target weight values ​​and the target bias values ​​of the overfitting layers in the decoder network.

[0427] The decoding unit is specifically used to assign values ​​to the weights and / or biases of the overfitting layer in the decoder network based on the target weight value and / or the target bias value.

[0428] The compressed feature information is input into the assigned decoder network to determine the reconstructed image block corresponding to the current image block.

[0429] In one possible implementation, the target weight value includes: the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch; or, a weight offset determined based on the weights of each overfitted layer determined by online training of the overfitted layers in the decoder network using the current image patch and the weights of the overfitted layers determined by offline training of the decoder network.

[0430] The bias term values ​​include: bias terms of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image patch; or, bias term offset determined by comparing the bias terms of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image patch with the bias terms of the overfitting layer determined by offline training of the decoder network.

[0431] In one possible implementation, the overfitting parameters include target parameters of a first type of additional module corresponding to each overfitting layer in the decoder network. The target parameters of the first type of additional module are determined by the encoder embedding the first type of additional module in parallel for each overfitting layer in the decoder network, and by online training of the relevant parameters of the first type of additional module using the current image patch.

[0432] The decoding unit is specifically used to embed the first type of additional module in parallel for each overfitting layer in the decoder network, and to assign values ​​to the relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; to input the compressed feature information into the decoder network embedded in the first type of additional module and assigned values, and to determine the reconstructed image block corresponding to the current image block.

[0433] In one possible implementation, the overfitting parameters include target weight values ​​for the first type of overfitting layer in the decoder network, and target parameters for the first type of additional modules corresponding to each of the second type of overfitting layers in the decoder network.

[0434] or,

[0435] The overfitting parameters include the bias target values ​​of the first type of overfitting layer in the decoder network, and the target parameters of the first type of additional modules corresponding to each of the second type of overfitting layers in the decoder network.

[0436] or,

[0437] The overfitting parameters include the target weight values ​​and the target bias values ​​of the first type of overfitting layer in the decoder network, as well as the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network.

[0438] The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and by using the current image patch to train the relevant parameters of the first type of additional modules online.

[0439] The decoding unit is specifically configured to assign values ​​to the weights and / or bias terms of the first type of overfitting layer in the decoder network based on the target weight value and / or the target bias value; and to embed the first type of additional module in parallel for each second type of overfitting layer in the decoder network, and assign values ​​to the relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; input the compressed feature information into the decoder network embedded in the first type of additional module and the assigned values, and determine the reconstructed image block corresponding to the current image block.

[0440] In one possible implementation, the target weight value includes: the weights of each first type of overfitting layer determined by online training of the decoder network embedded with the first type of additional module using the current image patch; or, a weight offset determined based on the weights of each first type of overfitting layer determined by online training of the decoder network embedded with the first type of additional module using the current image patch and the weights of the first type of overfitting layer determined by offline training of the decoder network.

[0441] The bias term values ​​include: bias terms for each type of overfitting layer determined by online training of the decoder network embedded with the first type of additional modules using the current image patch; or, bias term offset determined by comparing the bias terms for each type of overfitting layer determined by online training of the decoder network embedded with the first type of additional modules using the current image patch with the bias terms for the first type of overfitting layers determined by offline training of the decoder network.

[0442] In one possible implementation, the overfitting parameters include target inputs of second-type additional modules corresponding to each overfitting layer in the decoder network. The target inputs of the second-type additional modules are determined by the encoder embedding the second-type additional modules in parallel for each overfitting layer in the decoder network and by online training on the inputs of the second-type additional modules. The relevant parameters of the second-type additional modules are determined by offline training.

[0443] The decoding unit is specifically used to embed the second type of additional modules in parallel for each overfitting layer in the decoder network; using the compressed feature information as the input of the overfitting layer in the decoder network and the target input as the input of the second type of additional modules, the decoder network with the embedded second type of additional modules is used to determine the reconstructed image block corresponding to the current image block.

[0444] In one possible implementation, the overfitting parameters include the target input of an additional network, which is determined by the encoder through online training based on the output of the decoder network and the output of the additional network, and the relevant parameters of the additional network are determined through offline training.

[0445] The decoding unit is specifically used to use the compressed feature information as the input to the overfitting layer in the decoder network and the target input as the input to the additional network, and to determine the reconstructed image block corresponding to the current image block using the output of the decoder network and the output of the additional network.

[0446] In one possible implementation, the acquisition unit is specifically used to parse the prior information corresponding to the current image block from the first bitstream corresponding to the current image block; and to decode the prior information using a superscale decoding network to obtain the residual probability distribution parameters corresponding to the current image block.

[0447] The acquisition unit is further specifically used to decode the second bitstream corresponding to the current image block based on the residual probability distribution parameter to obtain the quantization residual corresponding to the current image block;

[0448] The decoding unit is further specifically used to determine the reconstructed features corresponding to the current image patch based on the prior information and the quantization residual, using a context-based prediction model.

[0449] In one possible implementation, the acquisition unit is specifically used to parse the overfitting parameters corresponding to the current image block from the target bitstream corresponding to the current image block; wherein the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

[0450] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network.

[0451] The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block;

[0452] The decoding unit is specifically configured to obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; and to assign values ​​to the weights and / or bias terms of the overfitting layer in the synthetic transformation network based on the weight target value and / or bias target value; and to input the reconstruction features into the assigned synthetic transformation network to determine the reconstructed image patch corresponding to the current image patch.

[0453] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network.

[0454] The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block;

[0455] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; and to embed the first type of additional module in parallel for each overfitting layer in the synthetic transformation network, and to assign values ​​to the relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; to input the reconstruction features into the synthetic transformation network after embedding the first type of additional module and assigning values, and to determine the reconstructed image patch corresponding to the current image patch.

[0456] In one possible implementation, the decoder network includes a synthetic transform network; a first type of overfitting layer in the decoder network includes a first type of overfitting layer in the synthetic transform network, and a second type of overfitting layer in the decoder network includes a second type of overfitting layer in the synthetic transform network.

[0457] The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block;

[0458] The decoding unit is specifically configured to: obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; assign values ​​to the weights and / or bias terms of the first type of overfitting layer in the synthetic transformation network based on the target weight value and / or the target bias value; embed the first type of additional module in parallel for each second type of overfitting layer in the synthetic transformation network, and assign values ​​to the relevant parameters of the first type of additional module based on the target parameters of the first type of additional module; input the reconstruction features into the first type of additional module and the synthetic transformation network after assignment, and determine the reconstructed image patch corresponding to the current image patch.

[0459] In one possible implementation, the first type of additional module is a low-rank matrix.

[0460] In one possible implementation, the acquisition unit is further configured to acquire an overfitting layer type identifier corresponding to the current image patch; wherein the overfitting layer type identifier is used to indicate whether the overfitting layer is a first type of overfitting layer or a second type of overfitting layer.

[0461] In one possible implementation, the decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network.

[0462] The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block;

[0463] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; and to embed the second type of additional modules in parallel for each overfitting layer in the synthetic transformation network; using the reconstruction features as the input of the overfitting layer in the synthetic transformation network and the target input as the input of the second type of additional modules, and using the synthetic transformation network with the second type of additional modules embedded, to determine the reconstructed image patch corresponding to the current image patch.

[0464] In one possible implementation, the overfitting layer includes some or all of the network layers.

[0465] In one possible implementation, the acquisition unit is further configured to acquire an overfitting layer identifier corresponding to the current image patch when the overfitting layer includes a portion of the network layers in the network; wherein the overfitting layer identifier is used to indicate an overfitting layer in the network.

[0466] The overfitting layer identifier includes the layer number of the overfitting layer, or the number of overfitting layers N; wherein the number of overfitting layers N is used to indicate that the first N layer numbers in the offline determined layer number list are overfitting layer numbers.

[0467] In one possible implementation, the decoder network includes a synthesis transform network;

[0468] The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block;

[0469] The decoding unit is specifically used to obtain the reconstruction features corresponding to the current image patch based on the prior information and the quantization residual; use the reconstruction features as the input of the overfitting layer in the synthetic transform network and the target input as the input of the additional network; and use the output of the synthetic transform network and the output of the additional network to determine the reconstructed image patch corresponding to the current image patch.

[0470] In one possible implementation, the acquisition unit is further configured to acquire the overfitting scheme identifier corresponding to the current image block; wherein, different overfitting scheme identifiers correspond to different overfitting schemes;

[0471] The acquisition unit is further specifically used to parse the overfitting parameters corresponding to the current image block from the target bitstream corresponding to the current image block based on the overfitting scheme identifier; wherein the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

[0472] In one possible implementation, the acquisition unit is further configured to acquire a list of distribution coefficients of each overfitting layer parameter or the distribution coefficients of all parameters corresponding to the current image block when the overfitting parameters are encoded using entropy coding at the encoding end.

[0473] And / or,

[0474] When the overfitting parameters include network parameters, the number of parameters corresponding to each overfitting layer corresponding to the current image patch is obtained;

[0475] And / or,

[0476] When the overfitting parameters include parameters of a low-rank matrix, the rank of the low-rank matrix corresponding to the current image patch is obtained;

[0477] And / or,

[0478] If the overfitting parameters include input parameters, obtain the number of input parameters corresponding to the current image patch.

[0479] Based on the same concept as the above method, this application also proposes an encoding device, which can be applied to an encoding end, and the encoding device may include:

[0480] Obtain the compressed feature information of the current image block, and write the compressed feature information into the bitstream corresponding to the current image block; and,

[0481] Obtain the overfitting parameters corresponding to the current image block, and write the overfitting parameters into the bitstream corresponding to the current image block.

[0482] It should be noted that since the encoder network structure includes the decoder network structure, the encoder end will also perform the above-mentioned operation of "obtaining the reconstructed image block corresponding to the current image block using the decoder network based on the compressed feature information and the overfitting parameters" during the image encoding process. The specific implementation process can be referred to the relevant description in the above-mentioned embodiment on the decoder end side.

[0483] Accordingly, the encoding unit may also have the same functions as the decoding unit.

[0484] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. This application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The above descriptions are merely embodiments of this application and are not intended to limit this application.

[0485] Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A decoding method, comprising: The method includes: Obtain the compressed feature information of the current image patch; and, Obtain the overfitting parameters corresponding to the current image patch; Based on the compressed feature information and the overfitting parameters, the reconstructed image block corresponding to the current image block is obtained using a decoder network. The method further includes: Obtain the overfitting scheme identifier corresponding to the current image block; wherein, different overfitting scheme identifiers correspond to different overfitting schemes; The step of obtaining the overfitting parameters corresponding to the current image patch includes: Based on the overfitting scheme identifier, the overfitting parameters corresponding to the current image block are parsed from the target bitstream corresponding to the current image block; wherein, the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

2. The method of claim 1, wherein, The decoder network includes a synthesis-transformation network; The step of obtaining the compressed feature information of the current image block includes: Obtain the prior information corresponding to the current image patch, and obtain the residual probability distribution parameters corresponding to the current image patch based on the prior information; Based on the residual probability distribution parameters, the quantization residual corresponding to the current image block is obtained; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; Based on the overfitting parameters and the reconstruction features, the reconstructed image block corresponding to the current image block is obtained using the synthetic transformation network.

3. The method according to claim 1 or 2, characterized in that, The overfitting parameters include the target weight values ​​of the overfitting layers in the decoder network; or, The overfitting parameters include the bias target values ​​of the overfitting layers in the decoder network; or, The overfitting parameters include the target weight values ​​and the target bias values ​​of the overfitting layers in the decoder network. The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the target weight value and / or target bias value, assign values ​​to the weights and / or bias terms of the overfitting layer in the decoder network; The compressed feature information is input into the assigned decoder network to determine the reconstructed image block corresponding to the current image block.

4. The method according to claim 3, characterized in that, The target weight values ​​include: the weights of each overfitted layer determined by online training of the overfitted layer in the decoder network using the current image patch; or, a weight offset determined based on the weights of each overfitted layer determined by online training of the overfitted layer in the decoder network using the current image patch and the weights of the overfitted layer determined by offline training of the decoder network. The bias term values ​​include: bias terms of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image patch; or, bias term offset determined by comparing the bias terms of each overfitting layer determined by online training of the overfitting layer in the decoder network using the current image patch with the bias terms of the overfitting layer determined by offline training of the decoder network.

5. The method according to claim 1 or 2, characterized in that, The overfitting parameters include target parameters of a first type of additional module corresponding to each overfitting layer in the decoder network. The target parameters of the first type of additional module are determined by the encoder embedding the first type of additional module in parallel for each overfitting layer in the decoder network, and by online training of the relevant parameters of the first type of additional module using the current image patch. The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: In the decoder network, the first type of additional modules are embedded in parallel for each overfitting layer, and the relevant parameters of the first type of additional modules are assigned values ​​based on the target parameters of the first type of additional modules. The compressed feature information is input into the decoder network embedded in the first type of additional module and assigned a value to determine the reconstructed image block corresponding to the current image block.

6. The method according to claim 1 or 2, characterized in that, The overfitting parameters include the target weight values ​​of the first type of overfitting layer in the decoder network, and the target parameters of the first type of additional modules corresponding to each of the second type of overfitting layers in the decoder network. or, The overfitting parameters include the bias target values ​​of the first type of overfitting layer in the decoder network, and the target parameters of the first type of additional modules corresponding to each of the second type of overfitting layers in the decoder network. or, The overfitting parameters include the target weight values ​​and the target bias values ​​of the first type of overfitting layer in the decoder network, as well as the target parameters of the first type of additional modules corresponding to each second type of overfitting layer in the decoder network. The target parameters are determined by the encoder embedding the first type of additional modules in parallel for each second type of overfitting layer in the decoder network, and by using the current image patch to train the relevant parameters of the first type of additional modules online. The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the target weight value and / or the target bias value, the weights and / or bias terms of the first type of overfitting layer in the decoder network are assigned values; and, for each second type of overfitting layer in the decoder network, the first type of additional module is embedded in parallel, and the relevant parameters of the first type of additional module are assigned values ​​based on the target parameters of the first type of additional module. The compressed feature information is input into the decoder network embedded in the first type of additional module and assigned a value to determine the reconstructed image block corresponding to the current image block.

7. The method according to claim 6, characterized in that, The target weight values ​​include: the weights of each first-type overfitting layer determined by online training of the decoder network embedded with the first-type additional module using the current image patch; or, a weight offset determined based on the weights of each first-type overfitting layer determined by online training of the decoder network embedded with the first-type additional module using the current image patch and the weights of the first-type overfitting layers determined by offline training of the decoder network. The bias term values ​​include: bias terms for each type of overfitting layer determined by online training of the decoder network embedded with the first type of additional modules using the current image patch; or, bias term offset determined by comparing the bias terms for each type of overfitting layer determined by online training of the decoder network embedded with the first type of additional modules using the current image patch with the bias terms for the first type of overfitting layers determined by offline training of the decoder network.

8. The method according to claim 1 or 2, characterized in that, The overfitting parameters include the target inputs of the second type of additional modules corresponding to each overfitting layer in the decoder network. The target inputs of the second type of additional modules are determined by the encoder embedding the second type of additional modules in parallel for each overfitting layer in the decoder network and by online training on the inputs of the second type of additional modules. The relevant parameters of the second type of additional modules are determined by offline training. The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: The second type of additional modules are embedded in parallel for each overfitting layer in the decoder network; Using the compressed feature information as the input to the overfitting layer in the decoder network and the target input as the input to the second type of additional module, the decoder network with the second type of additional module embedded is used to determine the reconstructed image block corresponding to the current image block.

9. The method according to claim 1 or 2, characterized in that, The overfitting parameters include the target input of the additional network, which is determined by the encoder through online training based on the output of the decoder network and the output of the additional network. The relevant parameters of the additional network are determined through offline training. The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Using the compressed feature information as the input to the overfitting layer in the decoder network and the target input as the input to the additional network, the reconstructed image block corresponding to the current image block is determined using the output of the decoder network and the output of the additional network.

10. The method according to claim 2, characterized in that, The step of obtaining the prior information corresponding to the current image patch and obtaining the residual probability distribution parameters corresponding to the current image patch based on the prior information includes: The prior information corresponding to the current image block is parsed from the first bitstream corresponding to the current image block; The super-prior information is decoded using a super-scale decoding network to obtain the residual probability distribution parameters corresponding to the current image patch; The step of obtaining the quantization residual corresponding to the current image patch based on the residual probability distribution parameters includes: Based on the residual probability distribution parameters, the second bitstream corresponding to the current image block is decoded to obtain the quantization residual corresponding to the current image block; The step of obtaining the reconstructed features corresponding to the current image patch based on the prior information and the quantization residual includes: Based on the prior information and the quantization residual, the reconstructed features corresponding to the current image patch are determined using a context-based prediction model.

11. The method according to claim 1 or 2, characterized in that, The step of obtaining the overfitting parameters corresponding to the current image patch includes: The overfitting parameters corresponding to the current image block are parsed from the target bitstream corresponding to the current image block; wherein, the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

12. The method according to claim 3, characterized in that, The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the aforementioned prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; and, Based on the target weight value and / or target bias value, assign values ​​to the weights and / or bias terms of the overfitting layer in the synthetic transformation network; The reconstructed features are input into the synthetic transformation network after assignment to determine the reconstructed image block corresponding to the current image block.

13. The method according to claim 5, characterized in that, The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the aforementioned prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; and, In the synthetic transformation network, the first type of additional modules are embedded in parallel for each overfitting layer, and the relevant parameters of the first type of additional modules are assigned values ​​based on the target parameters of the first type of additional modules. The reconstructed features are embedded into the first type of additional module, and the synthetic transformation network after being assigned values ​​is used to determine the reconstructed image block corresponding to the current image block.

14. The method according to claim 6, characterized in that, The decoder network includes a synthetic transform network; the first type of overfitting layer in the decoder network includes the first type of overfitting layer in the synthetic transform network, and the second type of overfitting layer in the decoder network includes the second type of overfitting layer in the synthetic transform network; The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; as well as, Based on the target weight value and / or the target bias value, the weights and / or bias terms of the first type of overfitting layer in the synthetic transformation network are assigned values; and, for each second type of overfitting layer in the synthetic transformation network, the first type of additional module is embedded in parallel, and the relevant parameters of the first type of additional module are assigned values ​​based on the target parameters of the first type of additional module. The reconstructed features are embedded into the first type of additional module, and the synthetic transformation network after being assigned values ​​is used to determine the reconstructed image block corresponding to the current image block.

15. The method according to claim 5, characterized in that, The first type of additional module is a low-rank matrix.

16. The method according to claim 6, characterized in that, The method further includes: Obtain the overfitting layer type identifier corresponding to the current image patch; wherein, the overfitting layer type identifier is used to indicate whether the overfitting layer is the first type of overfitting layer or the second type of overfitting layer.

17. The method according to claim 8, characterized in that, The decoder network includes a synthetic transform network; the overfitting layer in the decoder network includes the overfitting layer in the synthetic transform network; The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the aforementioned prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; and, The second type of additional module is embedded in parallel for each overfitting layer in the synthetic transformation network; Using the reconstructed features as the input to the overfitting layer in the synthetic transformation network and the target input as the input to the second type of additional module, the synthetic transformation network with embedded second type of additional module is used to determine the reconstructed image block corresponding to the current image block.

18. The method according to claim 3, characterized in that, The overfitting layer includes some or all of the network layers.

19. The method according to claim 18, characterized in that, In the case where the overfitting layer includes a portion of a network layer, the method further includes: Obtain the overfitting layer identifier corresponding to the current image patch; wherein, the overfitting layer identifier is used to indicate the overfitting layer in the network; The overfitting layer identifier includes the layer number of the overfitting layer, or the number of overfitting layers N; wherein the number of overfitting layers N is used to indicate that the first N layer numbers in the offline determined layer number list are overfitting layer numbers.

20. The method according to claim 9, characterized in that, The decoder network includes a synthesis-transformation network; The compressed feature information includes the prior information corresponding to the current image block and the quantization residual corresponding to the current image block; The step of obtaining the reconstructed image patch corresponding to the current image patch using a decoder network based on the compressed feature information and the overfitting parameters includes: Based on the prior information and the quantization residual, the reconstructed features corresponding to the current image patch are obtained; Using the reconstructed features as input to the overfitting layer in the synthetic transformation network and the target input as input to the additional network, the reconstructed image block corresponding to the current image block is determined using the output of the synthetic transformation network and the output of the additional network.

21. The method according to claim 1 or 2, characterized in that, The method further includes: When entropy coding is used at the encoding end to encode the overfitting parameters, obtain a list of distribution coefficients of each overfitting layer parameter corresponding to the current image block or the distribution coefficients of all parameters; And / or, When the overfitting parameters include network parameters, the number of parameters corresponding to each overfitting layer corresponding to the current image patch is obtained; And / or, When the overfitting parameters include parameters of a low-rank matrix, the rank of the low-rank matrix corresponding to the current image patch is obtained; And / or, If the overfitting parameters include input parameters, obtain the number of input parameters corresponding to the current image patch.

22. An encoding method, characterized in that, Applied to the encoding end, the method includes: Obtain the compressed feature information of the current image block, and write the compressed feature information into the bitstream corresponding to the current image block; and, Obtain the overfitting parameters corresponding to the current image block, and write the overfitting parameters into the bitstream corresponding to the current image block; The method further includes: Obtain the overfitting scheme identifier corresponding to the current image block and send the overfitting scheme identifier to the decoding end; wherein, different overfitting scheme identifiers correspond to different overfitting schemes.

23. A decoding device, characterized in that, include: The acquisition unit is used to acquire the compressed feature information of the current image block; And, obtain the overfitting parameters corresponding to the current image patch; The decoding unit is used to obtain the reconstructed image block corresponding to the current image block using a decoder network based on the compressed feature information and the overfitting parameters. The acquisition unit is further configured to acquire the overfitting scheme identifier corresponding to the current image block; wherein different overfitting scheme identifiers correspond to different overfitting schemes. The acquisition unit is further specifically used to parse the overfitting parameters corresponding to the current image block from the target bitstream corresponding to the current image block based on the overfitting scheme identifier; wherein the target bitstream includes a first bitstream, a second bitstream, or a third bitstream.

24. An encoding device, characterized in that, include: The acquisition unit is used to acquire the compressed feature information of the current image block; The encoding unit is used to write the compressed feature information into the bitstream corresponding to the current image block; The acquisition unit is further configured to acquire the overfitting parameters corresponding to the current image block; The encoding unit is also used to write the overfitting parameters into the bitstream corresponding to the current image block; The acquisition unit is further configured to acquire the overfitting scheme identifier corresponding to the current image block; The encoding unit is further configured to send the overfitting scheme identifier to the decoding end; wherein, different overfitting scheme identifiers correspond to different overfitting schemes.

25. A decoding device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method according to any one of claims 1-21.

26. An encoding terminal device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of claim 22.

27. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-21, or to implement the method of claim 22.

28. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores a plurality of computer instructions, which, when executed by a processor, implement the method of any one of claims 1-21, or, when executed by a processor, implement the method of claim 22.

Images