Decoding and coding method, device and equipment

By using probability distribution parameters on the decoding end and the encoding end for feature enhancement, combined with the neural network structure, the problem of insufficient encoding performance and decoding performance in the neural network encoding and decoding method is solved, and efficient compression and reconstruction of video images are achieved.

CN119011850AActive Publication Date: 2024-11-22HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411375202.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-04-20
Publication Date
2024-11-22
Estimated Expiration
2043-04-20

AI Technical Summary

Technical Problem

The neural network-based encoding and decoding method has shortcomings in encoding and decoding performance, and is highly complex, making it difficult to effectively improve the reconstruction quality of video images.

Method used

An end-to-end video image compression method is proposed. The decoding end uses probability distribution parameters to enhance the quality of the features, and the encoding end encodes the parameters into the header information code stream. Combined with the neural network structure design, it improves the encoding and decoding performance and reduces complexity .

Benefits of technology

It improves encoding efficiency and decoding efficiency, maintains low complexity, effectively improves the reconstruction quality of video images, and enhances encoding and decoding performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011850A_ABST
    Figure CN119011850A_ABST
Patent Text Reader

Abstract

The invention provides a decoding method, a coding method, a decoding device, a coding device and equipment. The decoding method comprises the following steps: decoding a code stream corresponding to a current image block to obtain a coefficient hyper-parameter feature corresponding to the current image block; determining probability distribution parameters based on the coefficient hyper-parameter features; decoding a code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; and determining a target reconstruction image block corresponding to the current image block based on the initial reconstruction feature. Through the technical scheme of the invention, the coding performance and the decoding performance can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to a decoding and coding method, device and equipment thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. A complete video encoding method may include prediction, transformation, quantization, entropy coding, filtering and other processes. Among them, the prediction process may include intra-frame prediction and inter-frame prediction. Inter-frame prediction uses the correlation of the video time domain to predict the current pixel of the current frame image using the pixels of the adjacent encoded images to effectively remove the temporal redundancy of the video. Intra-frame prediction uses the correlation of the video space domain to predict the current pixel using the pixels of the encoded blocks of the current frame image to effectively remove the spatial redundancy of the video.

[0003] With the rapid development of deep learning, deep learning has achieved success in many high-level computer vision problems, such as image classification and target detection. Deep learning has also gradually begun to be applied in the field of encoding and decoding, that is, neural networks can be used to encode and decode images. Although the encoding and decoding methods based on neural networks have shown great performance potential, they still have problems such as poor encoding performance, poor decoding performance and high complexity. Summary of the invention

[0004] The present application provides a decoding and encoding method, apparatus and device thereof.

[0005] The present application provides a decoding method, which is applied to a decoding end, and the method includes: decoding a code stream corresponding to a current image block to obtain coefficient hyper-parameter features corresponding to the current image block; determining probability distribution parameters based on the coefficient hyper-parameter features; decoding the code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; and determining a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0006] The present application provides a coding method, which is applied to a coding end, and the method includes: encoding coefficient hyperparameter features corresponding to a current image block to obtain a first code stream corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features; encoding initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; encoding important channel identifiers to obtain a third code stream corresponding to the current image block.

[0007] The present application provides a decoding device, which is applied to a decoding end, and the device includes: a decoding module, which is used to decode a code stream corresponding to a current image block to obtain coefficient hyper-parameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyper-parameter features, and decode the code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; and a determination module, which is used to determine a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0008] The present application provides an encoding device, which is applied to an encoding end, and the device includes: an encoding module, which is used to encode coefficient hyperparameter features corresponding to a current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features; encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; encode important channel identifiers to obtain a third code stream corresponding to the current image block.

[0009] The present application provides a decoding end device, which includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned decoding method.

[0010] The present application provides an encoding end device, which includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned encoding method.

[0011] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned decoding method; or, the processor is used to execute the machine-executable instructions to implement the above-mentioned encoding method.

[0012] The present application provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the above-mentioned decoding method is implemented; or, the above-mentioned encoding method is implemented.

[0013] The present application provides a computer application, which, when executed by a processor, can implement the above-mentioned decoding method; or, implement the above-mentioned encoding method. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1It is a schematic diagram of a three-dimensional feature matrix in one embodiment of the present application.

[0015] Figure 2 It is a flowchart of a decoding method in one implementation of the present application.

[0016] Figure 3 It is a flowchart of an encoding method in one embodiment of the present application.

[0017] Figure 4 It is a schematic diagram of the processing process of the encoding end in one implementation manner of the present application.

[0018] Figure 5 It is a schematic diagram of the processing process of the decoding end in one implementation manner of the present application.

[0019] Fig. 6A It is a schematic diagram of the position of the feature domain enhancement module in one embodiment of the present application.

[0020] Figure 6B It is a schematic diagram of the positions of a feature domain enhancement module and an image domain enhancement module in one embodiment of the present application.

[0021] Figure 6C It is a schematic diagram of the position of the image domain enhancement module in one implementation manner of the present application.

[0022] Fig. 7A It is a hardware structure diagram of a decoding end device in one implementation manner of the present application.

[0023] Figure 7B It is a hardware structure diagram of an encoding end device in one implementation of the present application. DETAILED DESCRIPTION

[0024] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. In addition, depending on the context, the word "if" used can be interpreted as "at ... ", or "when ... ", or "in response to determination".

[0025] A decoding method and an encoding method are proposed in the embodiments of the present application, which may involve the following concepts: entropy coding (Entropy Encoding), neural network (Neural Network, NN), convolutional neural network (Convolutional Neural Network, CNN), deconvolution layer (Deconvolution), generalization ability (Generalization Ability), feature (feature), rate-distortion optimization (Rate-Distortion Optimized, RDO) principle.

[0026] Entropy coding is a coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information of the source (a measure of uncertainty). Entropy coding methods may include but are not limited to: Shannon coding, Huffman coding, and arithmetic coding.

[0027] Neural network refers to artificial neural network. Neural network is a computing model composed of a large number of nodes (or neurons) connected to each other. In neural network, neuron processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in neural network can be divided into three categories: input unit, output unit and hidden unit. Input unit receives signals and data from the outside world; output unit realizes the output of processing results; hidden unit is located between input unit and output unit and cannot be observed from outside the system. The connection weight between neurons reflects the connection strength between units, and the representation and processing of information are reflected in the connection relationship of processing units. Neural network is a non-programmed, brain-like information processing method. The essence of neural network is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of neural network, and imitate the information processing function of human brain nervous system to different degrees and levels. In the field of video processing, commonly used neural networks can include but are not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0028] Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. The artificial neurons of convolutional neural network can respond to surrounding units within a part of the coverage range, and have excellent performance in large image processing. The basic structure of convolutional neural network includes two layers: feature extraction layer (also called convolution layer) and feature mapping layer (also called activation layer). Among them, for the feature extraction layer, the input of each neuron is connected to the local receptive field of the previous layer, and the local features are extracted. Once the local feature is extracted, its positional relationship with other features is also determined. For the feature mapping layer, each calculation layer of the neural network consists of multiple feature maps, each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional neural network. In addition, since the neurons on a mapping surface share weights, the number of free parameters of the network is reduced.

[0029] For example, one of the advantages of convolutional neural networks over image processing algorithms is that they avoid the complex pre-processing of images (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of convolutional neural networks over ordinary neural networks is that ordinary neural networks use a fully connected approach, that is, all neurons from the input layer to the hidden layer are connected, which will result in a huge number of parameters, making network training time-consuming or even difficult to train, while convolutional neural networks avoid this difficulty through local connections, weight sharing and other methods.

[0030] The deconvolution layer is also called the transposed convolution layer. The working process of the deconvolution layer is very similar to that of the convolution layer. The main difference is that the deconvolution layer will make the output larger than the input (of course, it can also remain the same) through padding. If the stride is 1, it means that the output size is equal to the input size; if the stride is N, it means that the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0031] Generalization ability can refer to the adaptability of machine learning algorithms to fresh samples. The purpose of learning is to learn the rules hidden behind the data pairs. For data outside the learning set with the same rules, the trained network can also give appropriate outputs. This ability can be called generalization ability.

[0032] The features involved in this application are three-dimensional feature matrices or tensors of C*W*H. Figure 1 As shown, Figure 1It is a schematic diagram of a three-dimensional feature matrix. In the three-dimensional feature matrix, C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be the input of a neural network, or the output of a neural network.

[0033] There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit rate, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode is:

[0034] J(mode)=D+λ*R.

[0035] Among them, D represents Distortion, which can usually be measured using the SSE (Sum of Squared Errors) indicator. SSE refers to the mean square sum of the differences between the reconstructed image block and the source image. In order to achieve cost considerations, the SAD (Sum of Absolute Differences) indicator can also be used. SAD refers to the sum of the absolute values ​​of the differences between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc. When selecting a mode, if the rate-distortion optimization principle is used to make a comparative decision on the encoding mode, the best encoding performance can usually be guaranteed.

[0036] A large number of coding tools have been proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the coding tools that can achieve the best coding performance are often different. Therefore, in the encoding process, the RDO principle is usually used to compare the coding performance of different tools or modes to select the optimal tool or mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the flag information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to obtain the best coding performance. The decoding end can obtain relevant mode information by directly parsing the flag information, which has little effect on the decoding complexity.

[0037] In the general framework of end-to-end image coding, it mainly includes the feature main information part and the super prior side information part. The feature main information part includes analysis network, quantization, normal entropy coding, normal entropy decoding and synthesis network, and the super prior side information part includes super prior analysis network, quantization, factored entropy coding, factored entropy decoding and super prior synthesis network. The image components are compressed, encoded and reconstructed and restored in the analysis network and synthesis network of the feature main information part respectively; the super prior side information part is mainly used to model the probability of the feature main information and guide the entropy encoding and decoding of the feature main information. In the general framework of end-to-end image coding, there is a problem that the super prior side information is not fully utilized, and the image reconstruction quality is not fully improved.

[0038] In view of this, in the embodiment of the present application, the characteristics of the end-to-end image coding framework are utilized, and the decoding end uses probability distribution parameters to enhance the quality of the features to improve the quality of the reconstructed image. The encoding end does not directly change the main information, but encodes the enhancement parameters into the header information bitstream (third bitstream), and the decoding end enhances the main information features through the enhancement parameters.

[0039] The decoding method and encoding method in the embodiments of the present application are described in detail below in combination with several specific embodiments.

[0040] Example 1: A decoding method is proposed in the embodiment of the present application, see Figure 2 FIG. 2 is a flow chart of the decoding method. The decoding method can be applied to a decoding end (also called a video decoder), and the method can include steps 201-204.

[0041] In step 201, a first bitstream corresponding to a current image block is decoded to obtain coefficient hyperparameter features corresponding to the current image block.

[0042] In step 202, probability distribution parameters are determined based on the coefficient hyperparameter feature, and the second code stream corresponding to the current image block is decoded based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block.

[0043] In step 203, the third bitstream corresponding to the current image block is decoded to obtain enhancement parameters corresponding to the current image block, wherein the third bitstream may also be referred to as a header information bitstream corresponding to the current image block.

[0044] In step 204, the initial reconstruction feature is enhanced based on the enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, and a target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction feature.

[0045] Exemplarily, the enhancement parameters include important channel identifiers. The initial reconstruction features include C feature channel maps. The probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps and the C feature channels Figure 1 One-to-one correspondence. From the C feature channel maps, the feature channel map corresponding to the important channel identifier is selected as the important feature channel map, and the remaining feature channel maps are selected as the unimportant feature channel maps; the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0046] Exemplarily, the initial reconstruction feature may include C feature channel maps, the probability distribution parameter may include C probability distribution channel maps, and the C probability distribution channel maps and the C feature channel maps may be Figure 1 One-to-one correspondence. Based on this, for each feature channel map, the number of consumed bits corresponding to the feature channel map can be determined based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, an important feature channel map is selected from the C feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps; then, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0047] Exemplarily, the initial reconstruction features may include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters may include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. The initial reconstruction features are enhanced based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, which may include but is not limited to: performing feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature adaptive edge enhancement; performing feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching; generating enhanced reconstruction features based on the first reconstruction feature and the second reconstruction feature.

[0048] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segmentation strength values ​​and multiple edge enhancement segmentation thresholds, and the multiple edge enhancement segmentation thresholds may form multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. On this basis, based on the feature domain enhancement parameters and the important probability distribution channel map, feature adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature adaptive edge enhancement, which may include but is not limited to: if the important probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, the edge enhancement segmentation strength value corresponding to the probability distribution value may be determined based on the edge enhancement threshold interval corresponding to the probability distribution value; then, based on the edge enhancement segmentation strength value corresponding to each probability distribution value, feature adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0049] Exemplarily, based on the edge enhancement segmented strength value corresponding to each probability distribution value, feature-adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature-adaptive edge enhancement, which may include but is not limited to: normalizing the important feature channel map to obtain a normalized feature map; generating a high-frequency detail image based on the important feature channel map and the normalized feature map; for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segmented strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue; based on the edge enhancement eigenvalue corresponding to each eigenvalue, an edge enhancement feature map is determined; and the edge enhancement feature map is denormalized to obtain the first reconstructed feature.

[0050] Exemplarily, the feature domain enhancement parameters may include stretching parameter values, and feature adaptive stretching is performed on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching, which may include but is not limited to: if the unimportant feature channel map includes multiple eigenvalues, and the unimportant probability distribution channel map includes multiple probability distribution values, then, for each eigenvalue in the unimportant feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue; then, the second reconstructed feature is determined based on the stretched eigenvalue corresponding to each eigenvalue in the unimportant feature channel map.

[0051] Exemplarily, determining a target reconstructed image block corresponding to a current image block based on enhanced reconstruction features may include, but is not limited to: inputting the enhanced reconstruction features into a synthesis transformation network to obtain a target reconstructed image block corresponding to the current image block; or, inputting the enhanced reconstruction features into a synthesis transformation network to obtain an initial reconstructed image block corresponding to the current image block; performing image adaptive edge enhancement on the initial reconstructed image block based on image domain enhancement parameters and probability distribution parameters corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block; wherein the image domain enhancement parameters are obtained by decoding a third code stream corresponding to the current image block, i.e., decoding the third code stream to obtain image domain enhancement parameters corresponding to the current image block.

[0052] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, and the multiple image enhancement segmentation thresholds may form multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values. Based on this, based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, the initial reconstructed image block is image-adaptively edge enhanced to obtain the target reconstructed image block corresponding to the current image block, which may include but is not limited to: obtaining the target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, the image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value; based on the image enhancement segmentation intensity value corresponding to each probability distribution value, the initial reconstructed image block is image-adaptively edge enhanced to obtain the target reconstructed image block corresponding to the current image block.

[0053] Exemplarily, obtaining a target probability distribution channel map based on probability distribution parameters may include, but is not limited to: if the probability distribution parameters include important probability distribution channel maps and unimportant probability distribution channel maps, then upsampling the important probability distribution channel maps to obtain a target probability distribution channel map; the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0054] Exemplarily, performing image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segment strength value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block may include but is not limited to: generating a high-frequency detail image based on the initial reconstructed image block; for each eigenvalue in the high-frequency detail image, performing edge enhancement on the eigenvalue based on the image enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determining the target reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image.

[0055] Exemplarily, enhancing the initial reconstructed features based on the enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed features may include but is not limited to: performing feature domain enhancement on the initial reconstructed features corresponding to the brightness component of the current image block based on the feature domain enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed features corresponding to the brightness component. Performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain the target reconstructed image block corresponding to the current image block may include but is not limited to: performing image adaptive edge enhancement on the initial reconstructed image block corresponding to the brightness component of the current image block based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block corresponding to the brightness component; performing image adaptive edge enhancement on the initial reconstructed image block corresponding to the chrominance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block corresponding to the chrominance component.

[0056] Exemplarily, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameters include a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps are Figure 1 Correspondingly, the method also includes: decoding the code stream corresponding to the current image block to obtain an important channel identifier; according to the important channel identifier, selecting the feature channel map corresponding to the important channel identifier from the multiple feature channel maps as the important feature channel map, and selecting the remaining feature channel maps as unimportant feature channel maps; selecting the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0057] Exemplarily, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameters include a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps are Figure 1 Correspondingly, the method also includes: for each feature channel map, based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map, determining the number of consumed bits corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, selecting important feature channel maps from the multiple feature channel maps, and selecting the remaining feature channel maps as unimportant feature channel maps; selecting the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0058] Exemplarily, by decoding a first code stream related to the current image block, the coefficient hyper-parameter features and the probability distribution parameters corresponding to the current image block are obtained; by decoding a second code stream related to the current image block, the initial reconstruction features corresponding to the current image block are obtained; by decoding a third code stream related to the current image block, the important channel identifier is obtained; wherein the first code stream, the second code stream and the third code stream are code streams that encode different information.

[0059] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0060] It can be seen from the above technical scheme that in the embodiment of the present application, after obtaining the initial reconstruction feature corresponding to the current image block, the initial reconstruction feature can be enhanced based on the enhancement parameter and the probability distribution parameter to obtain the enhanced reconstruction feature, and the target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction feature, thereby proposing an end-to-end video image compression method, which can realize the encoding and decoding of video images based on a neural network, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining the enhancement parameter and the probability distribution parameter. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network effectively guarantees the quality of the reconstructed image block while maintaining low complexity, thereby achieving the purpose of improving the encoding performance and decoding performance, and reducing the complexity. The enhancement parameter and the probability distribution parameter are used to enhance the quality of the feature. The encoding end does not directly change the feature information, but encodes the enhancement parameter (such as the important channel identifier) ​​into the header information code stream. The decoding end enhances the feature through the enhancement parameter, improves the encoding performance, and improves the quality of the reconstructed image.

[0061] Example 2: This application proposes an encoding method, see Figure 3 FIG. 2 is a flow chart of the encoding method. The encoding method may be applied to an encoding end (also called a video encoder), and the method may include steps 301-304.

[0062] In step 301, the coefficient hyper-parameter features corresponding to the current image block are encoded to obtain a first code stream corresponding to the current image block.

[0063] In step 302, probability distribution parameters are determined based on the coefficient hyperparameter features, and initial image features corresponding to the current image block are encoded based on the probability distribution parameters to obtain a second code stream corresponding to the current image block.

[0064] In step 303, for each candidate enhancement parameter, the initial reconstruction feature is enhanced based on the candidate enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, a target reconstructed image block is determined based on the enhanced reconstruction feature, and a cost value corresponding to the candidate enhancement parameter is determined based on the target reconstructed image block.

[0065] In step 304, based on the cost value corresponding to each candidate enhancement parameter, an enhancement parameter corresponding to the current image block is selected from all candidate enhancement parameters, and the enhancement parameter is encoded to obtain a third code stream corresponding to the current image block. The third code stream may also be referred to as a header information code stream corresponding to the current image block.

[0066] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. The initial reconstruction features are enhanced based on the candidate enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, which may include but is not limited to: performing feature adaptive edge enhancement on the important feature channel map based on the candidate feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature adaptive edge enhancement; performing feature adaptive stretching on the unimportant feature channel map based on the candidate feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching; generating enhanced reconstruction features based on the first reconstruction feature and the second reconstruction feature.

[0067] Exemplarily, determining the target reconstructed image block based on the enhanced reconstruction feature may include, but is not limited to: inputting the enhanced reconstruction feature into the synthesis transformation network to obtain an initial reconstructed image block corresponding to the current image block; for each candidate image domain enhancement parameter, performing image adaptive edge enhancement on the initial reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block. On this basis, for each candidate image domain enhancement parameter, a cost value corresponding to the candidate image domain enhancement parameter may also be determined based on the target reconstructed image block; based on the cost value corresponding to each candidate image domain enhancement parameter, an image domain enhancement parameter corresponding to the current image block is selected from all candidate image domain enhancement parameters, and the image domain enhancement parameter is encoded to obtain a third code stream corresponding to the current image block.

[0068] Exemplarily, the processing process of the encoding end is similar to the processing process of the decoding end, and the similarities are not repeated here. The processing process of the decoding end can be applied to the encoding end, that is, the encoding end adopts the same processing method.

[0069] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0070] It can be seen from the above technical scheme that in the embodiment of the present application, after obtaining the initial reconstruction feature corresponding to the current image block, the initial reconstruction feature can be enhanced based on the enhancement parameter and the probability distribution parameter to obtain the enhanced reconstruction feature, and the target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction feature, thereby proposing an end-to-end video image compression method, which can realize the encoding and decoding of video images based on a neural network, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining the enhancement parameter and the probability distribution parameter. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network effectively guarantees the quality of the reconstructed image block while maintaining low complexity, thereby achieving the purpose of improving the encoding performance and decoding performance, and reducing the complexity. The enhancement parameter and the probability distribution parameter are used to enhance the quality of the feature. The encoding end does not directly change the feature information, but encodes the enhancement parameter (such as the important channel identifier) ​​into the header information code stream. The decoding end enhances the feature through the enhancement parameter, improves the encoding performance, and improves the quality of the reconstructed image.

[0071] Embodiment 3: For Embodiment 1 and Embodiment 2, the processing process of the encoding end can be seen in Figure 4 Of course, Figure 4 This is just an example of the processing process at the encoding end, and there is no limitation on the processing process at the encoding end.

[0072] After obtaining the current image block x (the current image block x may be the original image block x, i.e., the input image block), the encoder may analyze and transform the current image block x through an analysis transformation network (i.e., a neural network) to obtain an image feature y corresponding to the current image block x. Among them, analyzing and transforming the current image block x through an analysis transformation network means: transforming the current image block x to an image feature y in a latent domain, thereby facilitating all subsequent processes to operate on the latent domain.

[0073] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the encoding and decoding process for the image block can also be directly used for the image.

[0074] After obtaining the image feature y, the encoding end performs coefficient hyperparameter feature transformation on the image feature y to obtain coefficient hyperparameter feature z. For example, the image feature y can be input into a hyperparameter encoding network (i.e., a neural network), and the hyperparameter encoding network performs coefficient hyperparameter feature transformation on the image feature y to obtain coefficient hyperparameter feature z. The hyperparameter encoding network can be a trained neural network, and there is no restriction on the training process of the hyperparameter encoding network, as long as the coefficient hyperparameter feature transformation can be performed on the image feature y. The image feature y in the latent domain obtains the super-prior latent information z after passing through the hyperparameter encoding network.

[0075] After obtaining the coefficient hyperparameter feature z, the encoder can quantize the coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, that is, Figure 4 Q in represents the quantization process. After obtaining the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain Bitstream#1 (i.e., the first bitstream) corresponding to the current image block, i.e. Figure 4 AE in represents the encoding process, such as the entropy encoding process. Alternatively, the encoding end can also directly encode the coefficient hyperparameter feature z to obtain the Bitstream#1 corresponding to the current image block. Among them, the hyperparameter quantization feature or coefficient hyperparameter feature z carried in Bitstream#1 is mainly used to obtain the parameters of the mean and probability distribution model.

[0076] After obtaining the Bitstream#1 corresponding to the current image block, the encoder may send the Bitstream#1 corresponding to the current image block to the decoder. For the processing process of the decoder for the Bitstream#1 corresponding to the current image block, refer to the subsequent embodiments.

[0077] After obtaining the Bitstream#1 corresponding to the current image block, the encoder can also decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 4 AD in the figure represents the decoding process. Then, the encoder can dequantize the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat can be the same as or different from the coefficient hyperparameter feature z. Figure 4The IQ in the code represents the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current image block, the encoder can also decode the Bitstream#1 to obtain the coefficient hyperparameter feature z_hat without involving the inverse quantization process of the coefficient hyperparameter feature z_hat.

[0078] For the encoding process of Bitstream#1, the encoding method of the fixed probability density model can be adopted, and for the decoding process of Bitstream#1, the decoding method of the fixed probability density model can be adopted. There is no restriction on this encoding and decoding process.

[0079] After obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (the determination process of the reconstruction feature y_hat is described in the subsequent embodiments) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat, and there is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu. The prediction value mu is used to obtain the residual feature r by subtracting from the original feature and is added to the decoded residual feature r_hat to obtain the reconstructed feature y_hat.

[0080] It should be noted that the mean prediction network is an optional neural network, that is, there is no need to determine the predicted value mu through the mean prediction network. Figure 4 The dashed box in indicates that the mean prediction network is optional.

[0081] After obtaining the image feature y, the encoder can determine the residual feature r based on the image feature y and the prediction value mu, such as taking the difference between the image feature y and the prediction value mu as the residual feature r. Then, the residual feature r is feature processed to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the prediction value mu. Alternatively, after obtaining the image feature y, the encoder can perform feature processing on the image feature y to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, there is no need to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0082] After obtaining the image feature s, the encoder can quantize the image feature s to obtain the image quantization feature corresponding to the image feature s, that is, Figure 4 Q in represents the quantization process. After obtaining the image quantization feature corresponding to the image feature s, the encoding end can encode the image quantization feature to obtain Bitstream#2 (i.e., the second bitstream) corresponding to the current image block, i.e. Figure 4 AE in the above represents an encoding process, such as an entropy encoding process. Alternatively, the encoding end may directly encode the image feature s to obtain Bitstream#2 corresponding to the current image block without involving the quantization process of the image feature s.

[0083] After obtaining Bitstream#2 corresponding to the current image block, the encoder may send Bitstream#2 corresponding to the current image block to the decoder. For the processing of Bitstream#2 corresponding to the current image block by the decoder, refer to the subsequent embodiments.

[0084] After obtaining the Bitstream#2 corresponding to the current image block, the encoder can also decode Bitstream#2 to obtain the image quantization feature, that is, Figure 4 AD in the figure represents the decoding process. Then, the encoder can dequantize the image quantization feature to obtain the image feature s'. The image feature s' can be the same as or different from the image feature s. Figure 4 The IQ in the code represents the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the encoder can also decode the Bitstream#2 to obtain the image feature s' without involving the inverse quantization process of the image quantization feature.

[0085] After obtaining the image feature s', the encoder can perform feature recovery (i.e., the inverse process of feature processing) on ​​the image feature s' to obtain the residual feature r_hat. There is no restriction on this feature recovery process, and it can be any feature recovery method. The residual feature r_hat and the residual feature r can be the same or different. After the encoder obtains the residual feature r_hat, the image feature y_hat (i.e., the reconstructed feature) is determined based on the residual feature r_hat and the prediction value mu. For example, the sum of the residual feature r_hat and the prediction value mu is used as the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the prediction value mu. Alternatively, after obtaining the image feature s', the encoder performs feature recovery (i.e., the inverse process of feature processing) on ​​the image feature s' to obtain the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, it is not necessary to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0086] After obtaining the image feature y_hat, the encoder can perform a synthetic transformation on the image feature y_hat to obtain a reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input to the synthetic transformation network, and the synthetic transformation network performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat. At this point, the image reconstruction process is completed.

[0087] In a possible implementation, when the encoding end encodes the image quantization feature or the image feature s to obtain the Bitstream#2 corresponding to the current image block, the encoding end needs to first determine the probability distribution model, and then encode the image quantization feature or the image feature s based on the probability distribution model. In addition, when the encoding end decodes Bitstream#2, it also needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model.

[0088] To obtain the probability distribution model, continue to refer to Figure 4 As shown, after obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters. For example, the coefficient hyperparameter feature z_hat is input into the probability hyperparameter decoding network, and the probability hyperparameter decoding network performs a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. After obtaining the probability distribution parameters, a probability distribution model can be generated based on the probability distribution parameters. Among them, the probability hyperparameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability hyperparameter decoding network. It can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters.

[0089] In a possible implementation, the processing process at the encoding end may be performed by a deep learning model or a neural network model, thereby realizing an end-to-end image compression and encoding process, without limitation.

[0090] Embodiment 4: For Embodiment 1 and Embodiment 2, the processing process of the decoding end can be seen in Figure 5 Of course, Figure 5 This is just an example of the processing process at the decoding end, and there is no limitation on the processing process at the decoding end.

[0091] After obtaining the Bitstream#1 corresponding to the current image block, the decoding end can also decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 5AD in the figure represents the decoding process. Then, the decoding end can dequantize the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat can be the same as or different from the coefficient hyperparameter feature z. Figure 5 The IQ in the above expression represents the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current image block, the decoder can also decode the Bitstream#1 to obtain the coefficient hyperparameter feature z_hat without involving the inverse quantization process of the coefficient hyperparameter feature z_hat.

[0092] For the decoding process of Bitstream#1, a decoding method of a fixed probability density model may be used, and there is no restriction on this.

[0093] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the decoding process for the image block can also be directly used for the image.

[0094] After obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (the determination process of the reconstruction feature y_hat is described in the subsequent embodiments) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat. There is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are combined to obtain a more accurate prediction value mu.

[0095] It should be noted that the mean prediction network is an optional neural network, that is, there is no need to determine the predicted value mu through the mean prediction network. Figure 5 The dashed box in indicates that the mean prediction network is optional.

[0096] After obtaining the Bitstream#2 corresponding to the current image block, the decoding end can decode Bitstream#2 to obtain the image quantization feature, that is, Figure 5 AD in the figure represents the decoding process. Then, the decoding end can dequantize the image quantization feature to obtain the image feature s'. The image feature s' can be the same as or different from the image feature s. Figure 5The IQ in the figure represents the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the decoder can decode the Bitstream#2 to obtain the image feature s' without involving the inverse quantization process of the image quantization feature.

[0097] After obtaining the image feature s', the decoder can perform feature recovery on the image feature s' (i.e., the inverse process of feature processing) to obtain the residual feature r_hat, which may be the same or different from the residual feature r. After obtaining the residual feature r_hat, the decoder determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the prediction value mu, such as taking the sum of the residual feature r_hat and the prediction value mu as the image feature y_hat, which may be the same or different from the image feature y. In this case, it is necessary to deploy a mean prediction network, which provides the prediction value mu. Alternatively, after obtaining the image feature s', the decoder performs feature recovery on the image feature s' to obtain the image feature y_hat, which may be the same or different from the image feature y. In this case, it is not necessary to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0098] After obtaining the image feature y_hat, the decoder can perform a synthetic transformation on the image feature y_hat to obtain a reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input to the synthetic transformation network, and the synthetic transformation network performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat. At this point, the image reconstruction process is completed.

[0099] In a possible implementation, when decoding Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model. To obtain the probability distribution model, continue to refer to Figure 5 As shown, after obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. For example, the coefficient hyperparameter feature z_hat is input into the probability hyperparameter decoding network, and the probability hyperparameter decoding network performs a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters. After obtaining the probability distribution parameters, a probability distribution model can be generated based on the probability distribution parameters. Among them, the probability hyperparameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability hyperparameter decoding network. It can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters.

[0100] In a possible implementation, the processing process at the above decoding end can be performed by a deep learning model or a neural network model, so as to realize an end-to-end image decoding process, which is not limited.

[0101] Example 5: Based on Example 3 and Example 4, a feature domain enhancement module can be added before the synthesis transformation network, see Fig. 6A As shown. The input feature of the feature domain enhancement module may be the image feature y_hat (hereinafter referred to as the initial reconstruction feature y_hat), and the output feature of the feature domain enhancement module may be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input to the synthesis transformation network, and the synthesis transformation network performs a synthesis transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0102] For the encoding end: After obtaining the current image block x, the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block (which can be the coefficient hyperparameter feature z itself or the hyperparameter quantization feature of the coefficient hyperparameter feature z) is encoded to obtain the first code stream corresponding to the current image block.

[0103] The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (e.g., the coefficient hyperparameter feature z_hat itself is decoded from the first bitstream, or the hyperparameter quantization feature is decoded from the first bitstream, and the hyperparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat). Then, the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0104] The initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma to obtain a second code stream corresponding to the current image block. The initial image feature may be an image feature y, or a residual feature r corresponding to the image feature y, or an image feature s after feature processing of the image feature y or the residual feature r, or an image quantization feature corresponding to the image feature y, or the residual feature r, or the image feature s, and there is no restriction on the initial image feature.

[0105] The second bitstream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, if the initial image feature is the image feature y, the initial reconstruction feature y_hat is decoded from the second bitstream. If the initial image feature is the image quantization feature corresponding to the image feature y, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the initial reconstruction feature y_hat. For another example, if the initial image feature is the residual feature r corresponding to the image feature y, the residual feature r_hat is decoded from the second bitstream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the residual feature r, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the residual feature r_hat, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image feature y or the image feature s corresponding to the residual feature r, the image feature s' is decoded from the second bitstream, and the image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the image feature s, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the image feature s'. The image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0106] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0107] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (such as decoding the coefficient hyperparameter feature z_hat itself from the first bitstream, or decoding the hyperparameter quantization feature from the first bitstream, and dequantizing the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat). Then, perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0108] Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, the initial reconstruction feature y_hat is decoded from the second code stream. Alternatively, the image quantization feature is decoded from the second code stream, and the image quantization feature is dequantized to obtain the initial reconstruction feature y_hat. For another example, the residual feature r_hat is decoded from the second code stream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second code stream, and the image quantization feature is dequantized to obtain the residual feature r_hat, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. For another example, the image feature s' is decoded from the second code stream, and the image feature s' is feature restored to obtain the initial reconstruction feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstruction feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second bit stream, and the image quantization feature is dequantized to obtain the image feature s', and the image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0109] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0110] Example 6: Based on Example 3 and Example 4, a feature domain enhancement module can be added before the synthetic transformation network, and an image domain enhancement module can be added after the synthetic transformation network. Figure 6BAs shown. The input feature of the feature domain enhancement module is the initial reconstructed feature y_hat, and the output feature of the feature domain enhancement module can be the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced can be input to the synthesis transformation network, and the synthesis transformation network performs a synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x. The input feature of the image domain enhancement module can be the initial reconstructed image block x_hat, and the output feature of the image domain enhancement module can be the target reconstructed image block x_hat_enhanced. In other words, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat, so as to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0111] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second code stream corresponding to the current image block. Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. Among them, the above process can be referred to Example 5, which will not be repeated here.

[0112] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0113] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0114] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, decode the second bitstream corresponding to the current image block to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5, which will not be repeated here.

[0115] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0116] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0117] Example 7: Based on Example 3 and Example 4, an image domain enhancement module can be added after the synthesis transformation network, see Figure 6C As shown. The input feature of the image domain enhancement module is the initial reconstructed image block x_hat, and the output feature of the image domain enhancement module is the target reconstructed image block x_hat_enhanced. In other words, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0118] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second code stream corresponding to the current image block. Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. Among them, the above process can be referred to Example 5, which will not be repeated here.

[0119] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input to the synthesis transformation network, which performs synthesis transformation on the initial reconstructed feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x. After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0120] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, decode the second bitstream corresponding to the current image block to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5, which will not be repeated here.

[0121] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input to the synthesis transformation network, which performs synthesis transformation on the initial reconstructed feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x. After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0122] Example 8: In Example 5 and Example 6, the initial reconstructed feature y_hat is input to the feature domain enhancement module, and the feature domain enhancement module performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. For example, the initial reconstructed feature y_hat can be enhanced in the feature domain based on the feature domain enhancement parameters and probability distribution parameters sigma corresponding to the current image block to obtain the enhanced reconstructed feature y_hat_enhanced. Feature domain enhancement can include feature adaptive edge enhancement and feature adaptive stretching. The probability distribution parameter sigma is used to assist in calculation, determine the channels that require feature adaptive edge enhancement and the channels that require feature adaptive stretching, and determine the strength of the stretching. The process is described below.

[0123] The initial reconstruction feature y_hat is C L xH L xW L The three-dimensional tensor, C L is the number of channels of the initial reconstructed feature y_hat (i.e., the number of channels in the feature domain or the number of channels in the Latent domain), H L is the feature height of the initial reconstructed feature y_hat (i.e., the feature height of the feature domain or the feature height of the Latent domain), W L is the feature width of the initial reconstructed feature y_hat (i.e., the feature width of the feature domain or the feature width of the Latent domain). For each channel of the initial reconstructed feature y_hat, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of channel ch is H L xW L The two-dimensional tensor y_hat_ch of is called the feature channel map. Therefore, the initial reconstruction feature y_hat can include C L Feature channel map.

[0124] The probability distribution parameter sigma is also C L xH L xW L The three-dimensional tensor, C L is the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. For each channel of the probability distribution parameter sigma, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of channel ch is H L xW LThe two-dimensional tensor sigma_ch is called the probability distribution channel map. Therefore, the probability distribution parameter sigma can include C L probability distribution channel graph, and C L The probability distribution channel diagram and C L Feature Channels Figure 1 One-to-one correspondence, such as the first probability distribution channel map corresponds to the first feature channel map, the second probability distribution channel map corresponds to the second feature channel map, and so on. L The probability distribution channel diagram and the C L The feature channel map corresponds to

[0125] For C L feature channel map, C L The feature channel graphs are divided into important feature channel graphs and unimportant feature channel graphs. There can be at least one important feature channel graph and multiple unimportant feature channel graphs. For example, C L At least one feature channel map is selected from the feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps. L The probability distribution channel diagram and C L Feature Channels Figure 1 One-to-one correspondence, therefore, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map. For example, if the first feature channel map is an important feature channel map, the first probability distribution channel map corresponding to the first feature channel map is selected as the important probability distribution channel map.

[0126] In summary, the initial reconstructed feature y_hat may include an important feature channel map and an unimportant feature channel map, and the probability distribution parameter sigma may include an important probability distribution channel map corresponding to the important feature channel map and an unimportant probability distribution channel map corresponding to the unimportant feature channel map. For the important feature channel map, the important feature channel map may be feature-adaptively edge enhanced based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstructed feature after feature-adaptive edge enhancement. For the unimportant feature channel map, the unimportant feature channel map may be feature-adaptively stretched based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature-adaptive stretching. On this basis, an enhanced reconstructed feature y_hat_enhanced may be generated based on the first reconstructed feature and the second reconstructed feature.

[0127] For example, assuming that the feature channel Figure 1 is an important feature channel map, feature channel Figure 2 To Feature Channel Figure 4is a non-important feature channel map, probability distribution channel Figure 1 It is an important probability distribution channel diagram, probability distribution channel Figure 2 To probability distribution channel Figure 4 For non-important probability distribution channel graphs, then: the parameters and probability distribution channels can be enhanced based on the feature domain Figure 1 For feature channels Figure 1 Perform feature adaptive edge enhancement to obtain the first reconstructed feature 1 (i.e., the reconstructed feature of the first channel) after feature adaptive edge enhancement. Figure 2 For feature channels Figure 2 The feature is adaptively stretched to obtain the second reconstructed feature 2 (i.e., the reconstructed feature of the second channel) after the feature adaptive stretching. The feature domain enhancement parameters and probability distribution channels can be Figure 3 For feature channels Figure 3 The feature adaptive stretching is performed to obtain the second reconstructed feature 3 (i.e., the reconstructed feature of the third channel) after the feature adaptive stretching. Figure 4 For feature channels Figure 4 The feature is adaptively stretched to obtain the second reconstruction feature 4 (i.e., the reconstruction feature of the fourth channel) after feature adaptive stretching. Then, the first reconstruction feature 1, the second reconstruction feature 2, the second reconstruction feature 3, and the second reconstruction feature 4 can be combined according to the channel (such as splicing according to the channel), that is, the reconstruction features of the four channels are combined to obtain the enhanced reconstruction feature y_hat_enhanced.

[0128] Embodiment 9: In embodiment 8, it is necessary to replace C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps. For example, the following method is used to divide C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps.

[0129] Mode 1: The encoder encodes the important channel identifier in the third bitstream corresponding to the current image block, and the decoder decodes the important channel identifier from the third bitstream corresponding to the current image block.

[0130] For example, the encoding end will C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps. There is no restriction on this process, and the important channel identifier (also called important channel number, recorded as important_channel) is encoded in the third bitstream corresponding to the current image block. After receiving the third bitstream, the decoding end decodes the important channel identifier from the third bitstream and obtains the important channel identifier from C LThe feature channel map corresponding to the important channel identifier is selected from the feature channel maps as the important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0131] Among them, the encoding end is C L When the feature channel graphs are divided into important feature channel graphs and unimportant feature channel graphs, the feature values ​​in each feature channel graph and the probability distribution values ​​in each probability distribution channel graph can be used to obtain the feature channel graphs from C. L The important feature channel maps are selected from the feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps.

[0132] Method 2: For both the encoding end and the decoding end, based on the feature values ​​in each feature channel map and the probability distribution values ​​in each probability distribution channel map, L Select the important feature channel maps from the feature channel maps.

[0133] For example, for each feature channel map, the number of consumed bits corresponding to the feature channel map can be determined based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map. For example, the number of consumed bits corresponding to the feature channel map can be determined using the following expression, which is, of course, only an example.

[0134]

[0135] In the above expression, bits_per_ch is used to represent the number of consumed bits corresponding to the feature channel graph ch, y_hat_ch(i,j) is used to represent the eigenvalue of the feature point (i,j) in the feature channel graph ch, sigma_ch(i,j) is used to represent the probability distribution value of the feature point (i,j) in the probability distribution channel graph ch, and Φ(.) is the standard normal distribution function.

[0136] After performing the above processing on each feature channel map, the number of consumed bits corresponding to each feature channel map can be obtained. Based on the number of consumed bits corresponding to each feature channel map, it can be obtained from C L For example, the feature channel map with the largest bit consumption is selected as the important feature channel map, or K feature channel maps with large bit consumption are selected as the important feature channel maps, where K can be a positive integer greater than 1.

[0137] In from C LAfter selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map.

[0138] Method 3: For both the encoding and decoding ends, the bit rate consumption corresponding to each feature channel map is calculated from C L For example, based on the consumed bit rate corresponding to each feature channel map, C L The feature channel maps are sorted, and the top K feature channel maps are selected as important feature channel maps. Alternatively, based on the consumed bit rate corresponding to each feature channel map, the C feature channel maps can be sorted in order of the consumed bit rate from small to large. L The feature channel maps are sorted, and the K feature channel maps with the lowest sorting order are selected as important feature channel maps.

[0139] In from C L After selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map.

[0140] Mode 4: For both the encoding end and the decoding end, the default feature channel map (i.e., the fixed feature channel map) is used as the important feature channel map. For example, the first feature channel map is pre-agreed as the important feature channel map, or the sixth feature channel map is pre-agreed as the important feature channel map, or the tenth feature channel map is pre-agreed as the important feature channel map. Of course, any feature channel map can be used as an important feature channel map, and there is no restriction on this.

[0141] Example 10: In Example 8, feature adaptive edge enhancement is performed on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain the first reconstructed feature. The feature adaptive edge enhancement process is described below.

[0142] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. For example, the encoder encodes multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block, and the decoder decodes multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds from the third bitstream corresponding to the current image block, and the multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds are used as feature domain enhancement parameters.

[0143] The number of edge enhancement segment strength values ​​and the number of edge enhancement segment thresholds may be the same or different. Taking the case where the two are the same as an example, the encoding end may also encode the number of edge enhancement segment strength values ​​(or edge enhancement segment thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of edge enhancement segment strength values ​​from the third bitstream corresponding to the current image block. Taking the case where the number of edge enhancement segment strength values ​​is n as an example, the n edge enhancement segment strength values ​​may be recorded as magl-1, magl-2, ..., magl-n, and the n edge enhancement segment thresholds may be recorded as thrl-1, thrl-2, ..., thrl-n.

[0144] Exemplarily, multiple edge enhancement segmentation thresholds may form multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. See Table 1 for an example of the correspondence.

[0145] Table 1

[0146] Edge enhancement threshold range Edge enhancement segment strength value Less than thrl-1 1 (no edge enhancement) [thrl-1, thrl-2) magl-1 [thrl-2, thrl-3) magl-2 [thrl-3, thrl-4) magl-3 … … [thrl-(n-1),thrl-n) magl-(n-1) Greater than or equal to thrl-n magl-n

[0147] Exemplarily, the important probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the edge enhancement threshold interval corresponding to the probability distribution value is first determined, and the edge enhancement segmentation strength value corresponding to the probability distribution value is determined based on the edge enhancement threshold interval. For example, if the probability distribution value is within [thrl-2, thrl-3), the edge enhancement segmentation strength value corresponding to the probability distribution value is magl-2. If the probability distribution value is within [thrl-3, thrl-4), the edge enhancement segmentation strength value corresponding to the probability distribution value is magl-3, and so on. After obtaining the edge enhancement segmentation strength value corresponding to each probability distribution value, feature adaptive edge enhancement can be performed on the important feature channel map based on the edge enhancement segmentation strength value corresponding to each probability distribution value to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0148] For example, the following steps S11-S15 may be used to perform feature adaptive edge enhancement on the important feature channel map.

[0149] In step S11, the important feature channel map is normalized to obtain a normalized feature map.

[0150] Exemplarily, normalization can be performed based on the important feature channel map, the mean feature corresponding to the important feature channel map, and the variance feature corresponding to the important feature channel map to obtain a normalized feature map. For example, the important feature channel map can be subtracted from the mean feature and then divided by the variance feature to obtain a normalized feature map. For another example, the important feature channel map can be subtracted from the mean feature and then divided by the variance feature to obtain an intermediate feature, and then the intermediate feature is converted to obtain a normalized feature map, such as multiplying the intermediate feature by 0.1 and adding 0.5 (0.1 and 0.5 are both examples), and then limiting the feature value to between 0 and 1. Of course, the above are just a few examples of normalizing the important feature channel map, and there is no limitation on this normalization method.

[0151] In step S12, a high-frequency detail image is generated based on the important feature channel map and the normalized feature map.

[0152] Exemplarily, a convolution operation (such as a two-dimensional convolution operation) can be performed on the normalized feature map and a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no limitation on this convolution kernel) to obtain a Gaussian blur image, and then the Gaussian blur image is subtracted from the important feature channel map to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image, and there is no limitation on this as long as the high-frequency details of the important feature channel map can be obtained.

[0153] In step S13, for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue.

[0154] Exemplarily, the important feature channel map includes multiple eigenvalues, the important probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Since the high-frequency detail image includes multiple eigenvalues ​​(corresponding one-to-one to the multiple eigenvalues ​​of the important feature channel map), the multiple probability distribution values ​​of the important probability distribution channel map correspond to the multiple eigenvalues ​​of the high-frequency detail image. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the important probability distribution channel map, and the edge enhancement threshold interval corresponding to the probability distribution value can be determined, and the edge enhancement segment strength value corresponding to the probability distribution value can be determined based on the edge enhancement threshold interval.

[0155] After obtaining the edge enhancement segment strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the edge enhancement segment strength value to obtain an edge enhancement eigenvalue. For example, the edge enhancement eigenvalue is multiplied by the edge enhancement segment strength value (such as magl-1, magl-2, etc.) to obtain an edge enhancement eigenvalue. Obviously, if the edge enhancement threshold interval is "less than thrl-1", the eigenvalue is multiplied by 1, that is, the eigenvalue remains unchanged, and the eigenvalue is not edge enhanced. If the edge enhancement threshold interval is [thrl-1, thrl-2), the eigenvalue is multiplied by magl-1, magl-1 is a value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other edge enhancement segment strength values ​​such as magl-2 are greater than 1, which can achieve edge enhancement.

[0156] For each eigenvalue in the high-frequency detail image, the edge enhancement segment strength value corresponding to the eigenvalue can be obtained in the above manner, and then the edge enhancement is performed on the eigenvalue based on the edge enhancement segment strength value to obtain the edge enhancement eigenvalue.

[0157] In step S14, an edge enhancement feature map is determined based on the edge enhancement feature value corresponding to each feature value. For example, the edge enhancement feature values ​​corresponding to all feature values ​​may be combined to obtain an edge enhancement feature map.

[0158] In step S15, the edge enhancement feature map is denormalized to obtain a first reconstructed feature (ie, a first reconstructed feature map). The first reconstructed feature (map) is a reconstructed feature (map) after feature adaptive edge enhancement is performed on the important feature channel map.

[0159] After obtaining the edge enhancement feature map, the edge enhancement feature map can be directly denormalized to obtain the first reconstructed feature map. Alternatively, after obtaining the edge enhancement feature map, the edge enhancement feature map can be added to the normalized feature map to obtain a modified edge enhancement feature map, and the modified edge enhancement feature map can be denormalized to obtain the first reconstructed feature map.

[0160] Exemplarily, the edge-enhanced normalized feature map can be determined based on the edge-enhanced feature map, and the first reconstructed feature map can be determined based on the edge-enhanced normalized feature map, the mean feature corresponding to the edge-enhanced normalized feature map, and the variance feature corresponding to the edge-enhanced normalized feature map. The first reconstructed feature can also be called y_hat_sharp.

[0161] For example, the edge enhancement feature map can be transformed to obtain a normalized feature map after edge enhancement. For example, the feature values ​​in the edge enhancement feature map are first limited to between 0 and 1, and then the feature values ​​in the edge enhancement feature map are subtracted by 0.5 and divided by 0.1 (0.5 and 0.1 are both examples) to obtain the normalized feature map after edge enhancement.

[0162] The normalized feature map after edge enhancement is multiplied by the variance feature, and then the mean feature is added to obtain the edge-enhanced feature map corresponding to the important feature channel map. The edge-enhanced feature map is recorded as the first reconstructed feature y_hat_sharp.

[0163] Of course, the above is only an example of denormalizing the edge enhancement feature map, and there is no limitation on the denormalization method.

[0164] In a possible implementation, the encoder needs to encode multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder may adopt the following method.

[0165] The encoding end can configure multiple candidate feature domain enhancement parameters. For each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. The cost value corresponding to each candidate feature domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block can be selected from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the smallest cost value. The encoding end can encode the feature domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0166] For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature can be enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature. The feature domain enhancement process is described in the above embodiment. The target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed feature, and the cost value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced. There is no limitation on the method for determining the cost value.

[0167] For example, for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes magl-1, magl-2, ..., magl-n, thrl-1, thrl-2, ..., thrl-n. Based on the candidate feature domain enhancement parameter, the enhanced reconstruction feature can be obtained, and then the target reconstructed image block x_hat_enhanced is obtained. The distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter corresponding to the minimum distortion index value can be selected as the feature domain enhancement parameter corresponding to the current image block, that is, the optimal feature domain enhancement parameter. In this way, the encoding end encodes the feature domain enhancement parameter in the third bit stream.

[0168] Example 11: In Example 8, it is necessary to perform feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map (i.e., the unimportant probability distribution channel map corresponding to the unimportant feature channel map) to obtain the second reconstructed feature after feature adaptive stretching. The feature adaptive stretching process is described below.

[0169] Exemplarily, the feature domain enhancement parameter may include a stretching parameter value. For example, the encoder may encode the stretching parameter value in the third bitstream corresponding to the current image block, and the decoder may decode the stretching parameter value from the third bitstream corresponding to the current image block, and the stretching parameter value is used as the feature domain enhancement parameter, and the stretching parameter value may be recorded as ρ.

[0170] Exemplarily, the non-important feature channel map may include multiple eigenvalues, and the non-important probability distribution channel map corresponding to the non-important feature channel map may include multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Based on this, for each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue. For example, the stretched eigenvalue corresponding to the eigenvalue can be determined by using the following expression. Of course, the following expression is only an example and is not limited to this.

[0171] y_hat_scale=y_hat+ρ*clip3(sigma*y_hat,-0.5,0.5)

[0172] In the above expression, y_hat_scale represents the stretched eigenvalue, y_hat represents the eigenvalue in the unimportant feature channel map, ρ represents the stretching parameter value, sigma represents the probability distribution value in the unimportant probability distribution channel map, and the probability distribution value sigma corresponds to the eigenvalue y_hat, clip3 is a limit operation, which is used to limit sigma*y_hat between -0.5 and 0.5. -0.5 and 0.5 are configurable values ​​and are not restricted. For example, when sigma*y_hat is less than -0.5, sigma*y_hat is limited to -0.5; when sigma*y_hat is greater than 0.5, sigma*y_hat is limited to 0.5; when sigma*y_hat is greater than or equal to -0.5 and less than or equal to 0.5, the value of sigma*y_hat remains unchanged.

[0173] After performing the above processing on each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to each eigenvalue can be obtained, and then the second reconstruction feature (i.e., the second reconstruction feature map) is determined based on the stretched eigenvalue corresponding to each eigenvalue. The second reconstruction feature (map) is a reconstruction feature (map) after feature adaptive stretching of the non-important feature channel map, such as combining the stretched eigenvalues ​​corresponding to all eigenvalues ​​to obtain the second reconstruction feature map. The second reconstruction feature is also called y_hat_scale.

[0174] The second reconstructed feature y_hat_scale corresponds to the non-important feature channel map of the initial reconstructed feature y_hat, that is, the non-important feature channel map is stretched to obtain the second reconstructed feature y_hat_scale. The first reconstructed feature y_hat_sharp corresponds to the important feature channel map of the initial reconstructed feature y_hat, that is, the important feature channel map is enhanced to obtain the first reconstructed feature y_hat_sharp. The second reconstructed feature y_hat_scale and the first reconstructed feature y_hat_sharp are combined to obtain the enhanced reconstructed feature y_hat_enhanced.

[0175] In a possible implementation, the encoding end needs to encode the stretching parameter value in the third code stream corresponding to the current image block. For this process, the encoding end can adopt the following method: the encoding end can configure multiple candidate stretching parameter values, and can determine the cost value corresponding to each candidate stretching parameter value. Based on the cost value corresponding to each candidate stretching parameter value, the stretching parameter value corresponding to the current image block can be selected from all candidate stretching parameter values, that is, the candidate stretching parameter value with the smallest cost value. The encoding end can encode the stretching parameter value to obtain the third code stream corresponding to the current image block.

[0176] For example, for each candidate stretching parameter value, feature adaptive stretching can be performed on the non-important feature channel map based on the candidate stretching parameter value and the non-important probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching, and then an enhanced reconstruction feature is obtained based on the second reconstruction feature, and a target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstruction feature, and a cost value corresponding to the candidate stretching parameter value is determined based on the target reconstructed image block x_hat_enhanced.

[0177] For example, for each candidate stretch parameter value, an enhanced reconstruction feature can be obtained based on the candidate stretch parameter value, and then the target reconstructed image block x_hat_enhanced is obtained. The distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate stretch parameter value, the encoder can select the candidate stretch parameter value corresponding to the minimum distortion index value as the stretch parameter value corresponding to the current image block, that is, the optimal stretch parameter value, and the encoder can encode the stretch parameter value in the third bitstream.

[0178] In another possible implementation, the encoding end configures multiple candidate feature domain enhancement parameters, and for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds and a stretching parameter value, determines the cost value corresponding to each candidate feature domain enhancement parameter, and based on the cost value corresponding to each candidate feature domain enhancement parameter, selects the feature domain enhancement parameter corresponding to the current image block from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the smallest cost value, and the encoding end encodes the feature domain enhancement parameter (that is, multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds and a stretching parameter value) to obtain a third bitstream corresponding to the current image block. For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature is enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature, determines the target reconstructed image block x_hat_enhanced based on the enhanced reconstructed feature, and determines the cost value corresponding to the candidate feature domain enhancement parameter based on the target reconstructed image block x_hat_enhanced.

[0179] Embodiment 12: In Embodiment 6 and Embodiment 7, the initial reconstructed image block x_hat is input to the image domain enhancement module, and the image domain enhancement module performs image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. For example, image adaptive edge enhancement can be performed on the initial reconstructed image block x_hat based on the image domain enhancement parameter and probability distribution parameter sigma corresponding to the current image block to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block. The image adaptive edge enhancement process is described below.

[0180] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds. For example, the encoder encodes multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds in the third bitstream corresponding to the current image block, and the decoder decodes multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds from the third bitstream corresponding to the current image block, and the multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds are used as image domain enhancement parameters.

[0181] The number of image enhancement segmentation strength values ​​and the number of image enhancement segmentation thresholds may be the same or different. Taking the case where the two are the same as an example, the encoding end may also encode the number of image enhancement segmentation strength values ​​(or image enhancement segmentation thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of image enhancement segmentation strength values ​​from the third bitstream corresponding to the current image block. Taking the case where the number of image enhancement segmentation strength values ​​is m as an example, the m image enhancement segmentation strength values ​​are recorded as magy-1, magy-2, ..., magy-m, and the m image enhancement segmentation thresholds are recorded as thry-1, thry-2, ..., thry-m.

[0182] Exemplarily, multiple image enhancement segmentation thresholds may form multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation strength values. See Table 2 for an example of the correspondence.

[0183] Table 2

[0184] Image enhancement threshold range Image enhancement segment intensity value Less than thry-1 1 (no edge enhancement) [thry-1, thry-2) magy-1 [thry-2, thry-3) magy-2 [thry-3, thry-4) magy-3 … … [thry-(m-1), thry-m) magy-(m-1) Greater than or equal to thry-m magy-m

[0185] Exemplarily, the following steps S21-S23 may be used to perform image adaptive edge enhancement on the initial reconstructed image block x_hat.

[0186] In step S21, a target probability distribution channel map is obtained based on the probability distribution parameters.

[0187] The initial reconstructed image block x_hat is a two-dimensional tensor of H×W, where H is the image height of the initial reconstructed image block x_hat and W is the image width of the initial reconstructed image block x_hat. The probability distribution parameter sigma is C L xH L xW L The three-dimensional tensor, C L is the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. The image height H of the initial reconstructed image block x_hat can be greater than the characteristic height H of the probability distribution parameter sigma. L , such as the image height H is the feature height H L 4 times, 8 times, 16 times, etc. The image width W of the initial reconstructed image block x_hat may be greater than the characteristic width W of the probability distribution parameter sigma L , such as the image width W is the feature width W L 4 times, 8 times, 16 times, etc.

[0188] In order to perform image adaptive edge enhancement on the initial reconstructed image block x_hat using probability distribution parameters, it is necessary to obtain a target probability distribution channel map based on the probability distribution parameters. The target probability distribution channel map is a two-dimensional tensor of H×W.

[0189] For example, from C L Select one probability distribution channel graph from the probability distribution channel graphs. Since the probability distribution parameter sigma includes important probability distribution channel graphs (at least one) and unimportant probability distribution channel graphs (multiple), it can be selected from C L Select an important probability distribution channel diagram from the probability distribution channel diagram, or select an important probability distribution channel diagram from C L A non-important probability distribution channel graph is selected from the probability distribution channel graphs. The following description will be given by taking the selection of an important probability distribution channel graph as an example.

[0190] Then, the important probability distribution channel map is upsampled to obtain a target probability distribution channel map. There is no restriction on the upsampling method, and the size of the target probability distribution channel map can be the same as the size of the initial reconstructed image block x_hat.

[0191] For example, the nearest neighbor upsampling method can be used to upsample the important probability distribution channel map to obtain the target probability distribution channel map. For the nearest neighbor upsampling method, a pixel can be selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding area on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is used as the value of the corresponding pixel on the target high-resolution image. Repeat the above process until all pixels of the target high-resolution image are assigned values.

[0192] In summary, the important probability distribution channel map can be upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_ch_upscale.

[0193] In step S22, if the target probability distribution channel map includes multiple probability distribution values, for each probability distribution value, an image enhancement segment strength value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value.

[0194] Exemplarily, the target probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the image enhancement threshold interval corresponding to the probability distribution value may be determined first, and the image enhancement segmentation intensity value corresponding to the probability distribution value may be determined based on the image enhancement threshold interval. For example, if the probability distribution value is within [thry-2, thry-3), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-2; if the probability distribution value is within [thry-3, thry-4), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-3, and so on.

[0195] In step S23, image adaptive edge enhancement is performed on the initial reconstructed image block x_hat based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block.

[0196] First, a high-frequency detail image can be generated based on the initial reconstructed image block x_hat. For example, a convolution operation (such as a two-dimensional convolution operation) can be performed on the initial reconstructed image block x_hat and a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no restriction on this convolution kernel) to obtain a Gaussian blur image, and then the Gaussian blur image is subtracted from the initial reconstructed image block x_hat to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image, and there is no restriction as long as the high-frequency details of the initial reconstructed image block x_hat can be obtained.

[0197] Then, for each eigenvalue in the high-frequency detail image, the eigenvalue is edge enhanced based on the image enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain the image enhancement eigenvalue. For example, the initial reconstructed image block x_hat includes multiple eigenvalues, the target probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Since the high-frequency detail image includes multiple eigenvalues ​​(one-to-one corresponding to the multiple eigenvalues ​​of the initial reconstructed image block x_hat), the multiple probability distribution values ​​of the target probability distribution channel map correspond to the multiple eigenvalues ​​of the high-frequency detail image. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the target probability distribution channel map, and the image enhancement threshold interval corresponding to the probability distribution value can be determined, and then the image enhancement segmentation intensity value corresponding to the probability distribution value can be determined based on the image enhancement threshold interval.

[0198] After obtaining the image enhancement segmentation strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the image enhancement segmentation strength value to obtain the image enhancement eigenvalue. For example, the image enhancement eigenvalue is obtained by multiplying the eigenvalue by the image enhancement segmentation strength value (such as magy-1, magy-2, etc.). Obviously, if the image enhancement threshold interval is "less than thry-1", the eigenvalue is multiplied by 1, that is, the eigenvalue remains unchanged, and the eigenvalue is not edge enhanced. If the image enhancement threshold interval is [thry-1, thry-2), the eigenvalue is multiplied by magy-1, magy-1 is a value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other image enhancement segmentation strength values ​​such as magy-2 are greater than 1, which can achieve edge enhancement.

[0199] Then, the target reconstructed image block x_hat_enhanced is determined based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image. For example, for each eigenvalue in the high-frequency detail image, the image enhancement eigenvalue corresponding to the eigenvalue can be obtained in the above manner, and then the image enhancement feature map can be determined based on the image enhancement eigenvalue corresponding to each eigenvalue. For example, the image enhancement eigenvalues ​​corresponding to all eigenvalues ​​can be combined to obtain an image enhancement feature map. Then, the image enhancement feature map is added to the initial reconstructed image block x_hat to obtain the final edge enhanced image, and then the edge enhanced image is limited to the value range of the image to obtain the target reconstructed image block x_hat_enhanced.

[0200] In a possible implementation, the encoder needs to encode multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder can adopt the following method.

[0201] The encoding end can configure multiple candidate image domain enhancement parameters. For each candidate image domain enhancement parameter, the candidate image domain enhancement parameter may include multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds. The cost value corresponding to each candidate image domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate image domain enhancement parameter, the image domain enhancement parameter corresponding to the current image block can be selected from all candidate image domain enhancement parameters, that is, the candidate image domain enhancement parameter with the smallest cost value. The encoding end can encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0202] For example, for each candidate image domain enhancement parameter, the initial reconstructed image block x_hat is subjected to image adaptive edge enhancement based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block. The image adaptive edge enhancement process is described in the above embodiment and will not be described again here. The cost value corresponding to the candidate image domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced, and the method for determining the cost value is not limited.

[0203] For example, for each candidate image domain enhancement parameter, the candidate image domain enhancement parameter includes magy-1, magy-2, ..., magy-m, thry-1, thry-2, ..., thry-m. Based on the candidate image domain enhancement parameter, the target reconstructed image block x_hat_enhanced can be obtained. The distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate image domain enhancement parameter, the candidate image domain enhancement parameter corresponding to the minimum distortion index value can be selected as the image domain enhancement parameter corresponding to the current image block, that is, the optimal image domain enhancement parameter. In this way, the encoding end can encode the image domain enhancement parameter in the third bitstream.

[0204] Example 13: For Example 5-Example 12, if a feature domain enhancement module is added before the synthetic transformation network, then the feature domain enhancement module can perform feature domain enhancement on the brightness component, can also perform feature domain enhancement on the chrominance component, and can also perform feature domain enhancement on the brightness component and the chrominance component at the same time. In this embodiment, feature domain enhancement of the brightness component is taken as an example. In order to perform feature domain enhancement on the brightness component, feature domain enhancement can be performed on the initial reconstructed feature y_hat corresponding to the brightness component of the current image block based on the feature domain enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed feature y_hat_enhanced corresponding to the brightness component. The feature domain enhancement process can be referred to Example 5-Example 12, which will not be repeated here.

[0205] For Example 5-Example 12, if an image domain enhancement module is added after the synthetic transformation network, then the image domain enhancement module can perform image domain enhancement on the brightness component, can also perform image domain enhancement on the chrominance component, and can also perform image domain enhancement on the brightness component and the chrominance component at the same time. In this embodiment, the image domain enhancement of the brightness component and the chrominance component is taken as an example. For example, in order to perform image domain enhancement on the brightness component, the initial reconstructed image block x_hat corresponding to the brightness component of the current image block can be image-adaptively edge enhanced based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block x_hat_enhanced corresponding to the brightness component. The image domain enhancement process can refer to Example 5-Example 12, which will not be repeated here. In addition, in order to perform image domain enhancement on the chrominance component, the initial reconstructed image block x_hat corresponding to the chrominance component of the current image block can be image-adaptively edge enhanced based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block x_hat_enhanced corresponding to the chrominance component.

[0206] When image adaptive edge enhancement is performed on the initial reconstructed image block x_hat corresponding to the chrominance component, the target probability distribution channel map of the chrominance component can be obtained based on the probability distribution parameters of the luminance component. For example, an important probability distribution channel map is selected from all probability distribution channel maps of the probability distribution parameters of the luminance component, and the important probability distribution channel map is upsampled to obtain the target probability distribution channel map of the chrominance component, and the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block x_hat corresponding to the chrominance component. For example, the important probability distribution channel map is upsampled using the nearest neighbor upsampling method to obtain the target probability distribution channel map of the chrominance component. For the nearest neighbor upsampling method, a pixel is selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding area on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is used as the value of the corresponding pixel on the target high-resolution image. Repeat the above process until all pixels of the target high-resolution image are assigned values.

[0207] After obtaining the target probability distribution channel map of the chroma component, for each probability distribution value in the target probability distribution channel map, the image enhancement segment intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value, and the initial reconstructed image block x_hat corresponding to the chroma component is image-adaptively enhanced based on the image enhancement segment intensity value corresponding to each probability distribution value, to obtain the target reconstructed image block x_hat_enhanced corresponding to the chroma component. The image adaptive edge enhancement process can refer to Example 12, and the relevant operations can be performed on the chroma component.

[0208] Embodiment 14: This embodiment proposes a decoding method, which may include the following steps S31 to S38.

[0209] In step S31, decode the feature domain enhancement parameters and image domain enhancement parameters from Bitstream#3 (i.e., the third bitstream, which can also be called the feature enhancement header information bitstream) corresponding to the current image block. Among them, the feature domain enhancement parameters may include an important channel identifier (important channel number important_channel), multiple edge enhancement segmentation strength values ​​(recorded as magl-1, magl-2, ..., magl-n), multiple edge enhancement segmentation thresholds (recorded as thrl-1, thrl-2, ..., thrl-n), and a stretching parameter value ρ, and the image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​(recorded as magy-1, magy-2, ..., magy-m), and multiple image enhancement segmentation thresholds (recorded as thry-1, thry-2, ..., thry-m). The feature domain enhancement parameters may also include the number n of edge enhancement segmentation strength values, and the image domain enhancement parameters may also include the number m of image enhancement segmentation strength values.

[0210] In step S32, the input features of the feature domain enhancement module are the initial reconstruction feature y_hat and the probability distribution parameter sigma. The initial reconstruction feature y_hat is a feature with a size of C L xH L xW L The three-dimensional tensor, C L is the number of channels in the feature domain, i.e., the Latent domain, H L is the characteristic height of the Latent domain, W L is the characteristic width of the Latent domain. The probability distribution parameter sigma is the size of C L xH L xW L When the channel of the initial reconstruction feature y_hat is [1, 2, …, C L ], this shape is H L The two-dimensional tensor y_hat_ch of xW is called the feature channel map. When the channel of the probability distribution parameter sigma is [1, 2, ..., C L ], this shape is H L xW L The two-dimensional tensor sigma_ch is called the probability distribution channel map.

[0211] In step S33, the initial reconstruction feature y_hat has C L feature channel graphs, and the probability distribution parameter sigma has C Lprobability distribution channel diagram, C L The probability distribution channel diagram and C L Feature Channels Figure 1 One-to-one correspondence, the probability distribution channel map is represented as sigma_ch. The feature channel map where the channel ch is an important channel identifier important_channel is subjected to feature adaptive edge enhancement, and the feature channel map where the channel ch is a non-important channel identifier is subjected to feature adaptive stretching.

[0212] In step S34, for the feature adaptive edge enhancement process, the input data are the important feature channel map y_hat_ch, the important probability distribution channel map sigma_ch, multiple edge enhancement segment strength values ​​(magl-1, magl-2, ..., magl-n), and multiple edge enhancement segment thresholds (thrl-1, thrl-2, ..., thrl-n). Based on the above input data, the important feature channel map y_hat_ch can be feature-adaptively enhanced to obtain a reconstructed feature map after feature adaptive edge enhancement. The reconstructed feature map is called the first reconstructed feature y_hat_sharp. The feature adaptive edge enhancement process refers to Example 10 and will not be repeated here.

[0213] In step S35, for the feature adaptive stretching process, the input data is the non-important feature channel map (the remaining (C L -1) non-important feature channel maps), non-important probability distribution channel maps (the remaining (C L -1) non-important probability distribution channel map), stretching parameter value ρ, based on the above input data, feature adaptive stretching can be performed on each non-important feature channel map to obtain a reconstructed feature map after feature adaptive stretching, and the reconstructed feature map is called the second reconstructed feature y_hat_scale. The feature adaptive stretching process is shown in Example 11 and will not be repeated here. Among them, the size of the non-important feature channel map subjected to feature adaptive stretching is (C L -1)xH L xW L , and the size of the corresponding probability distribution parameter sigma is also (C L -1)xH L xW L Each element of this three-dimensional tensor can be stretched through the following adaptive stretching algorithm to obtain the second reconstruction feature:

[0214] y_hat_scale=y_hat+ρ*clip3(sigma*y_hat,-0.5,0.5).

[0215] Among them, clip3 is a limit operation.

[0216] In step S36, the second reconstructed feature y_hat_scale corresponds to the enhancement of the unimportant channels of the initial reconstructed feature y_hat, and the first reconstructed feature y_hat_sharp corresponds to the enhancement of the important channels of the initial reconstructed feature y_hat. After the first reconstructed feature y_hat_sharp and the second reconstructed feature y_hat_scale are merged, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is obtained.

[0217] In step S37, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is input to the synthesis transformation network to obtain an initial reconstructed image block x_hat. The initial reconstructed image block x_hat is a two-dimensional tensor with a size of H×W. The important probability distribution channel map sigma_ch of the probability distribution parameter sigma is upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_ch_upscale.

[0218] In step S38, for the image domain enhancement process, the input data of the image domain enhancement module are the initial reconstructed image block x_hat, the target probability distribution channel map sigma_ch_upscale, multiple image enhancement segmentation intensity values ​​(magy-1, magy-2, ..., magy-m), and multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). Based on the above input data, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. The image domain enhancement process refers to Example 12 and will not be repeated here.

[0219] Embodiment 15: This embodiment proposes a coding method, which may include the following steps S41 to S45.

[0220] In step S41, based on C L The feature channel map y_hat and C L A probability distribution channel graph sigma is used to determine the important channel identifier corresponding to the important feature channel graph (the important channel identifier is also called the important channel number important_channel).

[0221] For example, slice y_hat and sigma along the channel dimension to get the current y_hat_ch and sigma_ch, and use these two tensors to calculate the bits_per_ch of each feature channel map, as shown in the following expression. LThe bits_per_ch of each feature channel map is obtained, and the channel number corresponding to the maximum bits_per_ch is selected as the important channel identifier.

[0222]

[0223] In step S42, the stretching parameter value ρ corresponding to the feature adaptive stretching process is determined. For example, N1 candidate stretching parameter values ​​ρ are selected, and the enhanced reconstruction features and the target reconstructed image block x_hat_enhanced corresponding to each candidate stretching parameter value ρ are obtained, and the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate stretching parameter value ρ corresponding to the minimum distortion index value is selected as the optimal stretching parameter value ρ corresponding to the feature adaptive stretching process.

[0224] In step S43, the feature domain enhancement parameters corresponding to the feature adaptive edge enhancement process are determined, and the feature domain enhancement parameters include multiple edge enhancement segment strength values ​​(magl-1, magl-2, ..., magl-n) and multiple edge enhancement segment thresholds (thrl-1, thrl-2, ..., thrl-n). For example, N2 candidate feature domain enhancement parameters are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate feature domain enhancement parameter are obtained, and the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate feature domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal feature domain enhancement parameter corresponding to the feature adaptive edge enhancement process.

[0225] In step S44, the image domain enhancement parameters corresponding to the image domain enhancement process are determined, and the image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​(magy-1, magy-2, ..., magy-m) and multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). For example, N3 candidate image domain enhancement parameters are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate image domain enhancement parameter are obtained, and the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate image domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal image domain enhancement parameter corresponding to the image domain enhancement process.

[0226] In step S45, the important channel identifier corresponding to the important feature channel map, the optimal stretch parameter value ρ, the optimal feature domain enhancement parameter, and the optimal image domain enhancement parameter are encoded into the header information bitstream (the third bitstream Bitstream#3 corresponding to the current image block). It should be noted that the feature domain enhancement module and the image domain enhancement module will not change Bitstream#1 and Bitstream#2, but the encoding end and the decoding end will pass through the enhancement module when reconstructing the image, so the reconstructed image will be consistent at the encoding and decoding ends.

[0227] Embodiment 16: A method for image adaptive edge enhancement, i.e., a process in which an image domain enhancement module performs image domain enhancement on an initial reconstructed image block x_hat to obtain a target reconstructed image block x_hat_enhanced, may be a non-edge enhancement mask edge enhancement algorithm, i.e., a USM edge enhancement algorithm (Unsharp Masking edge enhancement), and the process includes steps S51 to S54.

[0228] In step S51, a two-dimensional convolution operation is performed on the original reconstructed image and the Gaussian blur convolution kernel to obtain a Gaussian blur image.

[0229] In step S52, the Gaussian blurred image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0230] In step S53, the high-frequency detail image is multiplied by an edge enhancement coefficient (ie, the image enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue), and added to the original reconstructed image to obtain a final edge enhanced image.

[0231] In step S54, the edge enhanced image is limited to the value range of the image.

[0232] Exemplarily, the original reconstructed image is the initial reconstructed image block x_hat. After limiting the edge enhanced image to the value range of the image, the target reconstructed image block x_hat_enhanced can be obtained. The process can be referred to in Example 12.

[0233] Example 17: A method for feature adaptive edge enhancement, that is, a feature domain enhancement module performs feature adaptive edge enhancement on an important feature channel map based on feature domain enhancement parameters and an important probability distribution channel map to obtain a first reconstructed feature. The process may be a non-edge enhancement mask edge enhancement algorithm, that is, a USM edge enhancement algorithm, and the process includes steps S61 to S68.

[0234] In step S61, the important feature channel map y_hat_ch is subtracted from the mean and divided by the variance to obtain a normalized feature map.

[0235] In step S62, the normalized feature map is multiplied by 0.1 and added with 0.5, and the feature value is limited to between 0 and 1.

[0236] In step S63, a two-dimensional convolution operation is performed on the normalized feature map and the Gaussian blur convolution kernel to obtain a Gaussian blurred image.

[0237] In step S64, the Gaussian blurred image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0238] In step S65, the high-frequency detail image is multiplied by an edge enhancement coefficient (ie, the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue), and is added to the normalized feature map to obtain an edge enhancement feature map.

[0239] In step S66, the feature value of the edge enhancement feature map is limited to between 0 and 1.

[0240] In step S67, the edge enhanced feature map is subtracted by 0.5 and divided by 0.1 to obtain a normalized feature map after edge enhancement.

[0241] In step S68, the normalized feature map after edge enhancement is multiplied by the variance and added with the mean to obtain the important feature channel edge enhanced feature map y_hat_sharp corresponding to the important feature channel map y_hat_ch, that is, the first reconstructed feature.

[0242] Exemplarily, the original reconstructed image is the important feature channel map y_hat_ch. The process can be referred to in Example 10.

[0243] For example, the above embodiments can be implemented individually or in combination. For example, each embodiment in Embodiment 1 to Embodiment 17 can be implemented individually, and at least two embodiments in Embodiment 1 to Embodiment 17 can be implemented in combination.

[0244] Exemplarily, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way as the encoding end, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way as the decoding end.

[0245] Based on the same application concept as the above method, a decoding device is also proposed in the embodiment of the present application. The device is applied to the decoding end, and the device includes: a memory, which is configured to store video data; a decoder, which is configured to implement the decoding method in the above embodiments 1 to 17, that is, the processing flow of the decoding end.

[0246] For example, in one possible implementation, a decoder is configured to: decode a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode a second bitstream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decode a third bitstream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; and enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determine a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0247] Based on the same application concept as the above method, an encoding device is also proposed in the embodiment of the present application. The device is applied to the encoding end, and the device includes: a memory, which is configured to store video data; an encoder, which is configured to implement the encoding method in the above embodiments 1 to 17, that is, the processing flow of the encoding end.

[0248] For example, in one possible implementation, the encoder is configured to: encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; for each candidate enhancement parameter, enhance the initial reconstruction features based on the candidate enhancement parameter and the probability distribution parameters to obtain enhanced reconstruction features, determine a target reconstructed image block based on the enhanced reconstruction features, and determine a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encode the enhancement parameter, and obtain a third code stream corresponding to the current image block.

[0249] Based on the same application concept as the above method, the embodiment of the present application provides a decoding end device (also referred to as a video decoder). From the hardware level, the hardware architecture diagram of the decoding end device can be seen in Fig. 7A The decoding end device includes: a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 stores machine-executable instructions that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the decoding method of the above-mentioned embodiments 1-17 of the present application.

[0250] For example, in one possible implementation, the processor 711 executes machine-executable instructions to implement: decoding a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features, decoding a second bitstream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decoding a third bitstream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; and enhancing the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determining a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0251] Based on the same application concept as the above method, the embodiment of the present application provides an encoding end device (also referred to as a video encoder). From the hardware level, the hardware architecture diagram of the encoding end device can be seen in Figure 7B The encoding end device includes: a processor 721 and a machine-readable storage medium 722, wherein the machine-readable storage medium 722 stores machine-executable instructions that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the encoding method of the above-mentioned embodiments 1-17 of the present application.

[0252] For example, in one possible implementation, the processor 721 executes machine executable instructions to implement: encoding coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features, encoding initial image features corresponding to the current image block based on the probability distribution parameters, to obtain a second code stream corresponding to the current image block; for each candidate enhancement parameter, enhancing the initial reconstruction features based on the candidate enhancement parameter and the probability distribution parameters to obtain enhanced reconstruction features, determining a target reconstructed image block based on the enhanced reconstruction features, and determining a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; selecting an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encoding the enhancement parameter, and obtaining a third code stream corresponding to the current image block.

[0253] Based on the same application concept as the above method, an embodiment of the present application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the decoding method or encoding method of the above embodiments 1-17 of the present application.

[0254] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above embodiment of the present application can be implemented, such as the decoding method or encoding method in the above embodiments.

[0255] Based on the same application concept as the above method, the embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the decoding method or encoding method disclosed in the above embodiment of the present application.

[0256] Based on the same application concept as the above method, a decoding device is also proposed in an embodiment of the present application. The decoding device can be applied to a decoding end (also referred to as a video decoder). The decoding device may include: a decoding module, used to decode a first code stream corresponding to a current image block to obtain coefficient hyper-parameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyper-parameter features, and decode the second code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decode the third code stream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; an enhancement module, used to enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features; and a determination module, used to determine the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0257] Exemplarily, the enhancement parameter includes an important channel identifier, the initial reconstruction feature includes C feature channel maps, the probability distribution parameter includes C probability distribution channel maps, and the C probability distribution channel maps are related to the C feature channels. Figure 1 Correspondingly, the enhancement module is also used to select the feature channel map corresponding to the important channel identifier from the C feature channel maps as the important feature channel map, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0258] Exemplarily, the initial reconstruction features include C feature channel maps, the probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps and the C feature channel maps are Figure 1Correspondingly, the enhancement module is further used to determine, for each feature channel map, the number of consumed bits corresponding to the feature channel map based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, select important feature channel maps from the C feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0259] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps, the enhancement module enhances the initial reconstruction features based on the enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used for: performing feature adaptive edge enhancement on the important feature channel map based on feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature adaptive edge enhancement; performing feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching; and generating an enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature.

[0260] Exemplarily, the feature domain enhancement parameters include multiple edge enhancement segmentation strength values ​​and multiple edge enhancement segmentation thresholds, the multiple edge enhancement segmentation thresholds constitute multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values; the enhancement module performs feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map, and obtains the first reconstructed feature after feature-adaptive edge enhancement, which is specifically used for: if the important probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, determine the edge enhancement segmentation strength value corresponding to the probability distribution value based on the edge enhancement threshold interval corresponding to the probability distribution value; perform feature-adaptive edge enhancement on the important feature channel map based on the edge enhancement segmentation strength value corresponding to each probability distribution value, and obtain the first reconstructed feature after feature-adaptive edge enhancement.

[0261] Exemplarily, the enhancement module performs feature-adaptive edge enhancement on the important feature channel map based on the edge enhancement segmented strength value corresponding to each probability distribution value, and obtains the first reconstructed feature after feature-adaptive edge enhancement, which is specifically used for: normalizing the important feature channel map to obtain a normalized feature map; generating a high-frequency detail image based on the important feature channel map and the normalized feature map; for each eigenvalue in the high-frequency detail image, performing edge enhancement on the eigenvalue based on the edge enhancement segmented strength value corresponding to the probability distribution value corresponding to the eigenvalue, to obtain an edge enhanced eigenvalue; determining an edge enhancement feature map based on the edge enhancement eigenvalue corresponding to each eigenvalue; and denormalizing the edge enhancement feature map to obtain the first reconstructed feature.

[0262] Exemplarily, the feature domain enhancement parameters include stretching parameter values, and the enhancement module performs feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameters and the non-important probability distribution channel map to obtain the second reconstructed feature after feature adaptive stretching, which is specifically used for: if the non-important feature channel map includes multiple eigenvalues, and the non-important probability distribution channel map includes multiple probability distribution values, then, for each eigenvalue in the non-important feature channel map, based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue, determine the stretched eigenvalue corresponding to the eigenvalue; and determine the second reconstructed feature based on the stretched eigenvalue corresponding to each eigenvalue in the non-important feature channel map.

[0263] Exemplarily, when the determination module determines the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction feature, it is specifically used to: input the enhanced reconstruction feature to a synthetic transformation network to obtain a target reconstructed image block corresponding to the current image block; or, input the enhanced reconstruction feature to a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; perform image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters to obtain the target reconstructed image block corresponding to the current image block; wherein the image domain enhancement parameters are obtained by decoding a third code stream corresponding to the current image block.

[0264] Exemplarily, the image domain enhancement parameters include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, the multiple image enhancement segmentation thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values; the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters to obtain the target reconstructed image block corresponding to the current image block, and is specifically used to: obtain a target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, determine the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value; perform image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain the target reconstructed image block corresponding to the current image block.

[0265] Exemplarily, when the determination module obtains the target probability distribution channel map based on the probability distribution parameters, it is specifically used to: if the probability distribution parameters include important probability distribution channel maps and unimportant probability distribution channel maps, upsample the important probability distribution channel maps to obtain the target probability distribution channel map; wherein the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0266] Exemplarily, the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segment strength value corresponding to each probability distribution value to obtain the target reconstructed image block corresponding to the current image block, and is specifically used to: generate a high-frequency detail image based on the initial reconstructed image block; for each eigenvalue in the high-frequency detail image, perform edge enhancement on the eigenvalue based on the image enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determine the target reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image.

[0267] Exemplarily, the enhancement module enhances the initial reconstructed features based on the enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used for: performing feature domain enhancement on the initial reconstructed features corresponding to the brightness component of the current image block based on the feature domain enhancement parameters and the probability distribution parameters, to obtain the enhanced reconstruction features corresponding to the brightness component; the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters, and when obtaining the target reconstructed image block corresponding to the current image block, is specifically used for: performing image adaptive edge enhancement on the initial reconstructed image block corresponding to the brightness component of the current image block based on the image domain enhancement parameters and the probability distribution parameters, to obtain the target reconstructed image block corresponding to the brightness component; performing image adaptive edge enhancement on the initial reconstructed image block corresponding to the chrominance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters, to obtain the target reconstructed image block corresponding to the chrominance component.

[0268] Exemplarily, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameters include a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps are Figure 1 Correspondingly, the decoding module is further used to: decode the code stream corresponding to the current image block to obtain an important channel identifier; the determination module is further used to: select the feature channel map corresponding to the important channel identifier from the multiple feature channel maps as the important feature channel map, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0269] Exemplarily, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameters include a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps are Figure 1 One by one, the determination module is further used to: for each feature channel map, determine the number of consumed bits corresponding to the feature channel map based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, select important feature channel maps from the multiple feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0270] Exemplarily, by decoding a first code stream related to the current image block, the coefficient hyper-parameter features and the probability distribution parameters corresponding to the current image block are obtained; by decoding a second code stream related to the current image block, the initial reconstruction features corresponding to the current image block are obtained; by decoding a third code stream related to the current image block, the important channel identifier is obtained; wherein the first code stream, the second code stream and the third code stream are code streams that encode different information.

[0271] Based on the same application concept as the above method, an encoding device is also proposed in an embodiment of the present application. The device is applied to an encoding end (also referred to as a video encoder), and the device may include: an encoding module, used to encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; an enhancement module, used to enhance the initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters for each candidate enhancement parameter to obtain an enhanced reconstruction feature; a determination module, used to determine a target reconstructed image block based on the enhanced reconstruction feature, and determine a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter; the encoding module is also used to encode the enhancement parameter to obtain a third code stream corresponding to the current image block.

[0272] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. The enhancement module enhances the initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used for: performing feature-adaptive edge enhancement on the important feature channel map based on the candidate feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature-adaptive edge enhancement; performing feature-adaptive stretching on the unimportant feature channel map based on the candidate feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature-adaptive stretching; and generating an enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature.

[0273] Exemplarily, when the determination module determines the target reconstructed image block based on the enhanced reconstruction feature, it is specifically used to: input the enhanced reconstruction feature to the synthesis transformation network to obtain an initial reconstructed image block corresponding to the current image block; for each candidate image domain enhancement parameter, perform image adaptive edge enhancement on the initial reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block;

[0274] The encoding module is further used to determine, for each candidate image domain enhancement parameter, a cost value corresponding to the candidate image domain enhancement parameter based on the target reconstructed image block; based on the cost value corresponding to each candidate image domain enhancement parameter, select the image domain enhancement parameter corresponding to the current image block from all the candidate image domain enhancement parameters, encode the image domain enhancement parameter, and obtain a third code stream corresponding to the current image block.

[0275] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The above is only an embodiment of the present application and is not intended to limit the present application.

[0276] For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image decoding method, characterized in that: The method comprises: Inputting the initial reconstruction features corresponding to the current image block into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block; Obtaining image domain enhancement parameters corresponding to the current image block; Image adaptive edge enhancement is performed on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block.

2. The method according to claim 1, characterized in that: Before inputting the initial reconstruction features into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block, the method further includes: Decoding a bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; Probability distribution parameters are determined based on the coefficient hyperparameter features, and a code stream corresponding to the current image block is decoded based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block.

3. The method according to claim 1, characterized in that The image domain enhancement parameters include multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds, the multiple image enhancement segment thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segment strength values.

4. The method according to claim 3, characterized in that The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block includes: Determine a target probability distribution channel map based on the probability distribution parameters; If the target probability distribution channel map includes multiple probability distribution values, for each probability distribution value, determining the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value; Image adaptive edge enhancement is performed on the initial reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block.

5. The method according to claim 4, characterized in that The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block includes: generating a high-frequency detail image based on the initial reconstructed image block; For each eigenvalue in the high-frequency detail image, an image enhancement segmentation strength value is determined based on a probability distribution value corresponding to the eigenvalue; the image enhancement segmentation strength value is used to perform edge enhancement on the eigenvalue to obtain an image enhancement eigenvalue; A target reconstructed image block is determined based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image.

6. The method according to claim 2, characterized in that By decoding a first bitstream corresponding to the current image block, a coefficient hyperparameter feature corresponding to the current image block is obtained; by decoding a second bitstream corresponding to the current image block, an initial reconstruction feature corresponding to the current image block is obtained; The first code stream and the second code stream are code streams encoding different information.

7. An image encoding method, characterized in that: The method comprises: Inputting the initial reconstruction features corresponding to the current image block into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block; Obtaining image domain enhancement parameters corresponding to the current image block; Image adaptive edge enhancement is performed on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block.

8. An image decoding device, characterized in that: The device comprises: A determination module, used for inputting the initial reconstruction features corresponding to the current image block into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block; An acquisition module, used to acquire image domain enhancement parameters corresponding to the current image block; The enhancement module is used to perform image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block.

9. An image encoding device, characterized in that: The device comprises: A determination module, used for inputting the initial reconstruction features corresponding to the current image block into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block; An acquisition module, used to acquire image domain enhancement parameters corresponding to the current image block; The enhancement module is used to perform image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain a target reconstructed image block corresponding to the current image block.

10. A decoding end device, characterized in that: The decoding end device comprises: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement the method according to any one of claims 1 to 6.

11. A coding terminal device, characterized in that: The encoding end device includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine executable instructions to implement the method of claim 7.

12. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by a processor, the method described in any one of claims 1 to 6 is implemented; or, when the computer instructions are executed by a processor, the method described in claim 7 is implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented; or, when the computer program is executed by a processor, the method according to claim 7 is implemented.

Citation Information

Patent Citations

  • Decoding method, coding method, decoding device and coding device

    CN106998470A

  • Coarse-grained context entropy coding method

    CN113347422A

  • Enhancement layer coding and decoding method and device

    CN114615500A

  • Image data expansion method and device, electronic equipment and storage medium

    CN115346089A

  • Method and apparatus for reconstruct image from bitstreams and encoding image into bitstreams, and computer program product

    TW202228081A