Decoding method and apparatus, coding method and apparatus, and devices

MYPI2025006458A0Undetermined Publication Date: 2024-10-20HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025006458
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-20
Filing Date
2024-04-17
Publication Date
2024-10-20

AI Technical Summary

Technical Problem

Existing video encoding and decoding methods based on neural networks have problems such as poor coding performance, poor decoding performance, and high complexity. They fail to make full use of super-priori side information, resulting in insufficient improvement in image reconstruction quality.

Method used

Using an end-to-end image coding framework, the decoder uses probability distribution parameters to enhance the quality of features to improve the quality of the reconstructed image. The enhancement parameters are encoded at the encoding end into the header information stream, and the decoding end uses these parameters to perform the main information features. Enhancement, combining feature domain and image domain enhancement technology to improve encoding and decoding performance.

Benefits of technology

It improves the encoding and decoding efficiency of video images, improves the encoding performance and decoding performance, reduces the complexity, and improves the quality of image reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025006458_DRAWING_001
    Figure PI2025006458_DRAWING_001
  • Figure PI2025006458_DRAWING_002
    Figure PI2025006458_DRAWING_002
  • Figure PI2025006458_DRAWING_003
    Figure PI2025006458_DRAWING_003
Patent Text Reader

Abstract

Provided in the present application are a decoding method and apparatus, a coding method and apparatus, and devices. The decoding method comprises: decoding a code stream corresponding to the current image block, so as to obtain a coefficient hyper-parameter feature corresponding to the current image block; determining a probability distribution parameter on the basis of the coefficient hyper-parameter feature; on the basis of the probability distribution parameter, decoding a code stream corresponding to the current image block, so as to obtain an initial reconstruction feature corresponding to the current image block; and on the basis of the initial reconstruction feature, determining a target reconstructed image block corresponding to the current image block. By means of the technical solution of the present application, the coding performance and the decoding performance can be improved. Fig 2.
Need to check novelty before this filing date? Find Prior Art

Description

A decoding and encoding method, device and equipment thereof Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to a decoding and encoding method, device and equipment thereof. Background Art

[0002] To save space, video images are encoded before transmission. A complete video encoding method can include prediction, transformation, quantization, entropy coding, filtering, and other processes. The prediction process can include intra-frame prediction and inter-frame prediction. Inter-frame prediction uses the correlation in the temporal domain of the video to predict the current pixel of the current frame image using the pixels of adjacent encoded images to effectively remove temporal redundancy in the video. Intra-frame prediction uses the correlation in the spatial domain of the video to predict the current pixel using the pixels of the encoded blocks of the current frame image to effectively remove spatial redundancy in the video.

[0003] With the rapid development of deep learning, it has achieved success in many high-level computer vision problems, such as image classification and object detection. Deep learning is also gradually being applied in the field of codecs, where neural networks can be used to encode and decode images. Although neural network-based codecs show great performance potential, they still suffer from poor encoding and decoding performance and high complexity.

[0004] Summary of the Invention

[0005] The present application provides a decoding and encoding method, apparatus and device thereof.

[0006] The present application provides a decoding method, which is applied to a decoding end. The method includes: decoding a code stream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features; decoding the code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; and determining a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0007] The present application provides an encoding method, which is applied to an encoding end. The method includes: encoding coefficient hyperparameter features corresponding to a current image block to obtain a first code stream corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features; encoding initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; and encoding important channel identifiers to obtain a third code stream corresponding to the current image block.

[0008] The present application provides a decoding device, which is applied to a decoding end. The device includes: a decoding module, which is used to decode a code stream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode the code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; and a determination module, which is used to determine a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0009] The present application provides an encoding device, which is applied to an encoding end. The device includes: an encoding module, which is used to encode coefficient hyperparameter features corresponding to a current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features; encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; and encode important channel identifiers to obtain a third code stream corresponding to the current image block.

[0010] The present application provides a decoding end device, which includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned decoding method.

[0011] The present application provides an encoding end device, which includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned encoding method.

[0012] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned decoding method; or, the processor is used to execute the machine-executable instructions to implement the above-mentioned encoding method.

[0013] The present application provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the above-mentioned decoding method is implemented; or, the above-mentioned encoding method is implemented.

[0014] The present application provides a computer application, which, when executed by a processor, can implement the above-mentioned decoding method; or, implement the above-mentioned encoding method. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG1 is a schematic diagram of a three-dimensional feature matrix in one embodiment of the present application.

[0016] FIG2 is a flowchart of a decoding method in one embodiment of the present application.

[0017] FIG3 is a flow chart of an encoding method in one embodiment of the present application.

[0018] FIG4 is a schematic diagram of the processing process of the encoding end in one embodiment of the present application.

[0019] FIG5 is a schematic diagram of a processing process at a decoding end in one embodiment of the present application.

[0020] FIG6A is a schematic diagram showing the position of a feature domain enhancement module in one embodiment of the present application.

[0021] FIG6B is a schematic diagram showing the positions of a feature domain enhancement module and an image domain enhancement module in one embodiment of the present application.

[0022] FIG6C is a schematic diagram showing the position of an image domain enhancement module in one embodiment of the present application.

[0023] FIG7A is a hardware structure diagram of a decoding end device in one embodiment of the present application.

[0024] FIG7B is a hardware structure diagram of an encoding terminal device in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. In addition, depending on the context, the word "if" used can be interpreted as "at the time of...", or "when...", or "in response to determination".

[0026] In the embodiments of the present application, a decoding method and an encoding method are proposed, which may involve the following concepts: entropy encoding, neural network (NN), convolutional neural network (CNN), deconvolution layer (Deconvolution), generalization ability (Generalization Ability), feature (feature), and rate-distortion optimization (RDO) principle.

[0027] Entropy coding is a method of encoding data in which no information is lost according to the entropy principle. Entropy is the average amount of information in a source (a measure of uncertainty). Entropy coding methods include, but are not limited to, Shannon coding, Huffman coding, and arithmetic coding.

[0028] A neural network refers to an artificial neural network. A neural network is a computational model composed of a large number of interconnected nodes (or neurons). In a neural network, neuronal processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. Processing units in a neural network can be divided into three categories: input units, output units, and hidden units. Input units receive signals and data from the external world; output units output the processed results; and hidden units are located between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the strength of the connections between units, and the representation and processing of information is reflected in the connections between processing units. A neural network is a non-programmed, brain-like information processing method. Its essence is to achieve a parallel and distributed information processing function through the transformation and dynamic behavior of the neural network, mimicking the information processing functions of the human brain to varying degrees and levels. In the field of video processing, commonly used neural networks include, but are not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected networks.

[0029] A convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. The artificial neurons in a convolutional neural network can respond to surrounding units within a certain coverage area, making it an excellent choice for processing large images. The basic structure of a convolutional neural network consists of two layers: a feature extraction layer (also called a convolution layer) and a feature mapping layer (also called an activation layer). In the feature extraction layer, the input of each neuron is connected to the local receptive field of the previous layer, extracting the local features. Once the local feature is extracted, its positional relationship with other features is also determined. In the feature mapping layer, each computational layer of the neural network consists of multiple feature maps. Each feature map is a plane, and all neurons on the plane have equal weights. The feature map structure can use Sigmoid functions, ReLU functions, Leaky-ReLU functions, PReLU functions, GDN functions, and other activation functions as the activation function of the convolutional neural network. In addition, because neurons on a mapping surface share weights, the number of free parameters in the network is reduced.

[0030] For example, one of the advantages of convolutional neural networks over image processing algorithms is that they avoid complex pre-processing of images (such as extracting artificial features) and can directly input the original image for end-to-end learning. One of the advantages of convolutional neural networks over ordinary neural networks is that ordinary neural networks use a fully connected approach, that is, all neurons from the input layer to the hidden layer are connected. This will result in a huge number of parameters, making network training time-consuming or even difficult to train. Convolutional neural networks avoid this difficulty through methods such as local connections and weight sharing.

[0031] The deconvolution layer, also known as the transposed convolution layer, works very similarly to the convolution layer. The main difference is that the deconvolution layer uses padding to make the output larger than the input (of course, it can remain the same). If the stride is 1, the output size is equal to the input size; if the stride is N, the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0032] Generalization ability can refer to the adaptability of a machine learning algorithm to fresh samples. The purpose of learning is to learn the rules hidden behind the data pairs. For data outside the learning set with the same rules, the trained network can also give appropriate output. This ability can be called generalization ability.

[0033] The features involved in this application are three-dimensional feature matrices or tensors of C*W*H. Referring to Figure 1, Figure 1 is a schematic diagram of a three-dimensional feature matrix. In the three-dimensional feature matrix, C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be the input or output of a neural network.

[0034] Coding efficiency is evaluated using two main metrics: bit rate and PSNR (Peak Signal to Noise Ratio). A smaller bitrate results in a higher compression ratio, while a higher PSNR indicates better reconstructed image quality. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of these two factors. For example, the cost associated with a mode is: J(mode) = D + λ*R.

[0035] Here, D represents distortion, which is typically measured using the SSE (Sum of Squared Errors) metric. SSE is the mean squared sum of the differences between the reconstructed image block and the source image. For cost considerations, the SAD (Sum of Absolute Differences) metric can also be used. SAD is the sum of the absolute differences between the reconstructed image block and the source image. λ is the Lagrange multiplier, and R is the actual number of bits required to encode the image block in that mode, including the sum of bits required for coding mode information, motion information, residuals, and so on. When selecting a mode, using the rate-distortion optimization principle to compare coding modes generally ensures optimal coding performance.

[0036] Numerous coding tools have been proposed for various modules on the encoding side, and each tool often has multiple modes. Different coding tools often achieve optimal coding performance for different video sequences. Therefore, during the encoding process, the RDO principle is often used to compare the coding performance of different tools or modes to select the optimal one. After determining the optimal tool or mode, the decision-making information for the tool or mode is conveyed by encoding flag information in the bitstream. While this approach introduces higher coding complexity, it can adaptively select the optimal mode combination for different content, achieving optimal coding performance. The decoding side can obtain relevant mode information by directly parsing the flag information, which has a minimal impact on decoding complexity.

[0037] The general end-to-end image coding framework mainly includes the feature main information part and the super-prior side information part. The feature main information part includes the analysis network, quantization, normal entropy coding, normal entropy decoding, and synthesis network, while the super-prior side information part includes the super-prior analysis network, quantization, factored entropy coding, factored entropy decoding, and super-prior synthesis network. Image components are compressed, encoded, and reconstructed and restored in the analysis network and synthesis network of the feature main information part respectively; the super-prior side information part is mainly used to model the probability of the feature main information and guide the entropy encoding and decoding of the feature main information. In the general end-to-end image coding framework, the super-prior side information is not fully utilized, and the image reconstruction quality is not fully improved.

[0038] In view of this, in the embodiments of the present application, the characteristics of the end-to-end image coding framework are utilized, and the decoding end uses probability distribution parameters to enhance the quality of the features, thereby improving the quality of the reconstructed image. The encoding end does not directly change the main information, but instead encodes the enhancement parameters into the header information bitstream (the third bitstream). The decoding end then enhances the main information features using the enhancement parameters.

[0039] The following describes in detail the decoding method and encoding method in the embodiments of the present application in combination with several specific embodiments.

[0040] Embodiment 1: In the embodiment of the present application, a decoding method is proposed, as shown in Figure 2, which is a flowchart of the decoding method. The decoding method can be applied to a decoding end (also called a video decoder), and the method may include steps 201-204.

[0041] In step 201, a first code stream corresponding to a current image block is decoded to obtain coefficient hyperparameter features corresponding to the current image block.

[0042] In step 202, probability distribution parameters are determined based on the coefficient hyperparameter features, and the second code stream corresponding to the current image block is decoded based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block.

[0043] In step 203, the third code stream corresponding to the current image block is decoded to obtain enhancement parameters corresponding to the current image block, wherein the third code stream may also be referred to as a header information code stream corresponding to the current image block.

[0044] In step 204, the initial reconstruction feature is enhanced based on the enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, and a target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction feature.

[0045] Exemplarily, the enhancement parameters include important channel identifiers. The initial reconstruction features include C feature channel maps. The probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps correspond one-to-one to the C feature channel maps. From the C feature channel maps, the feature channel map corresponding to the important channel identifier is selected as the important feature channel map, and the remaining feature channel maps are selected as non-important feature channel maps; the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the non-important feature channel map is selected as the non-important probability distribution channel map.

[0046] Exemplarily, the initial reconstructed features may include C feature channel maps, and the probability distribution parameters may include C probability distribution channel maps, and the C probability distribution channel maps correspond one to one to the C feature channel maps. Based on this, for each feature channel map, the number of consumed bits corresponding to the feature channel map can be determined based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, important feature channel maps are selected from the C feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps; then, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0047] Exemplarily, the initial reconstruction features may include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters may include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. Enhancing the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features may include, but is not limited to: performing feature-adaptive edge enhancement on the important feature channel maps based on the feature domain enhancement parameters and the important probability distribution channel maps to obtain a first reconstruction feature after feature-adaptive edge enhancement; performing feature-adaptive stretching on the unimportant feature channel maps based on the feature domain enhancement parameters and the unimportant probability distribution channel maps to obtain a second reconstruction feature after feature-adaptive stretching; and generating enhanced reconstruction features based on the first reconstruction features and the second reconstruction features.

[0048] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segmentation strength values ​​and multiple edge enhancement segmentation thresholds, the multiple edge enhancement segmentation thresholds may form multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. On this basis, based on the feature domain enhancement parameters and the important probability distribution channel map, feature adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature adaptive edge enhancement, which may include but is not limited to: if the important probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, the edge enhancement segmentation strength value corresponding to the probability distribution value can be determined based on the edge enhancement threshold interval corresponding to the probability distribution value; then, based on the edge enhancement segmentation strength value corresponding to each probability distribution value, feature adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0049] Exemplarily, feature-adaptive edge enhancement is performed on the important feature channel map based on the edge enhancement segmentation intensity value corresponding to each probability distribution value to obtain the first reconstructed feature after feature-adaptive edge enhancement, which may include but is not limited to: normalizing the important feature channel map to obtain a normalized feature map; generating a high-frequency detail image based on the important feature channel map and the normalized feature map; for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue; determining an edge enhancement feature map based on the edge enhancement eigenvalue corresponding to each eigenvalue; and denormalizing the edge enhancement feature map to obtain the first reconstructed feature.

[0050] Exemplarily, the feature domain enhancement parameter may include a stretching parameter value, and the non-important feature channel map is feature-adaptively stretched based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching, which may include but is not limited to: if the non-important feature channel map includes multiple eigenvalues, and the non-important probability distribution channel map includes multiple probability distribution values, then, for each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue; then, the second reconstructed feature is determined based on the stretched eigenvalue corresponding to each eigenvalue in the non-important feature channel map.

[0051] Exemplarily, determining the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features may include but is not limited to: inputting the enhanced reconstruction features into a synthesis transformation network to obtain the target reconstructed image block corresponding to the current image block; or, inputting the enhanced reconstruction features into a synthesis transformation network to obtain an initial reconstructed image block corresponding to the current image block; performing image adaptive edge enhancement on the initial reconstructed image block based on image domain enhancement parameters and probability distribution parameters corresponding to the current image block to obtain the target reconstructed image block corresponding to the current image block; wherein the image domain enhancement parameters are obtained by decoding the third code stream corresponding to the current image block, that is, decoding the third code stream to obtain the image domain enhancement parameters corresponding to the current image block.

[0052] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, the multiple image enhancement segmentation thresholds may form multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values. Based on this, image-adaptive edge enhancement is performed on the initial reconstructed image block based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block to obtain the target reconstructed image block corresponding to the current image block, which may include but is not limited to: obtaining a target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, determining the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value; performing image-adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain the target reconstructed image block corresponding to the current image block.

[0053] Exemplarily, obtaining a target probability distribution channel map based on probability distribution parameters may include but is not limited to: if the probability distribution parameters include important probability distribution channel maps and unimportant probability distribution channel maps, then upsampling the important probability distribution channel map to obtain a target probability distribution channel map; the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0054] Exemplarily, performing image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block may include but is not limited to: generating a high-frequency detail image based on the initial reconstructed image block; for each eigenvalue in the high-frequency detail image, performing edge enhancement on the eigenvalue based on the image enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determining the target reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image.

[0055] Exemplarily, enhancing the initial reconstructed features based on the enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed features may include but is not limited to: performing feature domain enhancement on the initial reconstructed features corresponding to the luminance component of the current image block based on the feature domain enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed features corresponding to the luminance component. Performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block to obtain the target reconstructed image block corresponding to the current image block may include but is not limited to: performing image-adaptive edge enhancement on the initial reconstructed image block corresponding to the luminance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block corresponding to the luminance component; performing image-adaptive edge enhancement on the initial reconstructed image block corresponding to the chrominance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block corresponding to the chrominance component.

[0056] Exemplarily, the initial reconstructed features include multiple feature channel maps, the probability distribution parameters include multiple probability distribution channel maps, and the multiple probability distribution channel maps correspond one-to-one to the multiple feature channel maps. The method also includes: decoding the code stream corresponding to the current image block to obtain an important channel identifier; based on the important channel identifier, selecting the feature channel map corresponding to the important channel identifier from the multiple feature channel maps as the important feature channel map, and selecting the remaining feature channel maps as non-important feature channel maps; selecting the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting the probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map.

[0057] Exemplarily, the initial reconstructed features include multiple feature channel maps, the probability distribution parameters include multiple probability distribution channel maps, and the multiple probability distribution channel maps correspond one-to-one to the multiple feature channel maps. The method also includes: for each feature channel map, based on the characteristic value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map, determining the number of consumed bits corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, selecting important feature channel maps from the multiple feature channel maps, and selecting the remaining feature channel maps as unimportant feature channel maps; selecting the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0058] Exemplarily, by decoding a first code stream related to the current image block, the coefficient hyperparameter features and the probability distribution parameters corresponding to the current image block are obtained; by decoding a second code stream related to the current image block, the initial reconstruction features corresponding to the current image block are obtained; by decoding a third code stream related to the current image block, the important channel identifier is obtained; wherein, the first code stream, the second code stream, and the third code stream are code streams that encode different information.

[0059] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0060] It can be seen from the above technical solutions that in the embodiment of the present application, after obtaining the initial reconstruction features corresponding to the current image block, the initial reconstruction features can be enhanced based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and the target reconstructed image block corresponding to the current image block can be determined based on the enhanced reconstruction features, thereby proposing an end-to-end video image compression method, which can realize the encoding and decoding of video images based on a neural network, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining the enhancement parameters and the probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network can effectively ensure the quality of the reconstructed image block while maintaining low complexity, thereby achieving the purpose of improving the encoding performance and decoding performance, and reducing the complexity. The enhancement parameters and the probability distribution parameters are used to enhance the quality of the features. The encoding end does not directly change the feature information, but encodes the enhancement parameters (such as the important channel identifier) ​​into the header information code stream. The decoding end enhances the features through the enhancement parameters, improves the encoding performance, and improves the quality of the reconstructed image.

[0061] Embodiment 2: In the embodiment of the present application, an encoding method is proposed, as shown in Figure 3, which is a flowchart of the encoding method. The encoding method can be applied to an encoding end (also called a video encoder), and the method may include steps 301-304.

[0062] In step 301, the coefficient hyperparameter features corresponding to the current image block are encoded to obtain a first code stream corresponding to the current image block.

[0063] In step 302, probability distribution parameters are determined based on the coefficient hyperparameter features, and initial image features corresponding to the current image block are encoded based on the probability distribution parameters to obtain a second code stream corresponding to the current image block.

[0064] In step 303, for each candidate enhancement parameter, the initial reconstruction feature is enhanced based on the candidate enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, a target reconstructed image block is determined based on the enhanced reconstruction feature, and a cost value corresponding to the candidate enhancement parameter is determined based on the target reconstructed image block.

[0065] In step 304, based on the cost value corresponding to each candidate enhancement parameter, an enhancement parameter corresponding to the current image block is selected from all candidate enhancement parameters, and the enhancement parameter is encoded to obtain a third code stream corresponding to the current image block. The third code stream may also be referred to as a header information code stream corresponding to the current image block.

[0066] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. Enhancing the initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features may include but is not limited to: performing feature-adaptive edge enhancement on the important feature channel maps based on the candidate feature domain enhancement parameters and the important probability distribution channel maps to obtain a first reconstruction feature after feature-adaptive edge enhancement; performing feature-adaptive stretching on the unimportant feature channel maps based on the candidate feature domain enhancement parameters and the unimportant probability distribution channel maps to obtain a second reconstruction feature after feature-adaptive stretching; and generating enhanced reconstruction features based on the first reconstruction features and the second reconstruction features.

[0067] Exemplarily, determining a target reconstructed image block based on enhanced reconstruction features may include, but is not limited to: inputting the enhanced reconstruction features into a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; performing image-adaptive edge enhancement on the initial reconstructed image block based on each candidate image domain enhancement parameter and a probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block. Furthermore, for each candidate image domain enhancement parameter, a cost value corresponding to the candidate image domain enhancement parameter may be determined based on the target reconstructed image block; and based on the cost value corresponding to each candidate image domain enhancement parameter, an image domain enhancement parameter corresponding to the current image block is selected from all candidate image domain enhancement parameters, and the image domain enhancement parameter is encoded to obtain a third bitstream corresponding to the current image block.

[0068] Exemplarily, the processing process of the encoding end is similar to the processing process of the decoding end, and the similarities are not repeated here. The processing process of the decoding end can be applied to the encoding end, that is, the encoding end adopts the same processing method.

[0069] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0070] It can be seen from the above technical solutions that in the embodiment of the present application, after obtaining the initial reconstruction features corresponding to the current image block, the initial reconstruction features can be enhanced based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and the target reconstructed image block corresponding to the current image block can be determined based on the enhanced reconstruction features, thereby proposing an end-to-end video image compression method, which can realize the encoding and decoding of video images based on a neural network, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining the enhancement parameters and the probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network can effectively ensure the quality of the reconstructed image block while maintaining low complexity, thereby achieving the purpose of improving the encoding performance and decoding performance, and reducing the complexity. The enhancement parameters and the probability distribution parameters are used to enhance the quality of the features. The encoding end does not directly change the feature information, but encodes the enhancement parameters (such as the important channel identifier) ​​into the header information code stream. The decoding end enhances the features through the enhancement parameters, improves the encoding performance, and improves the quality of the reconstructed image.

[0071] Embodiment 3: With respect to Embodiment 1 and Embodiment 2, the processing process of the encoding end can be referred to as shown in FIG4. Of course, FIG4 is only an example of the processing process of the encoding end and does not limit the processing process of the encoding end.

[0072] After obtaining the current image block x (which can be the original image block x, i.e., the input image block), the encoder can perform an analytical transformation on the current image block x using an analytical transformation network (i.e., a neural network) to obtain the image features y corresponding to the current image block x. Performing an analytical transformation on the current image block x using the analytical transformation network means transforming the current image block x into image features y in the latent domain, thereby facilitating all subsequent processes to operate in the latent domain.

[0073] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the encoding and decoding process for the image block can also be directly applied to the image.

[0074] After obtaining the image feature y, the encoding end performs a coefficient hyperparameter feature transformation on the image feature y to obtain a coefficient hyperparameter feature z. For example, the image feature y can be input into a hyperparameter encoding network (i.e., a neural network), and the hyperparameter encoding network performs a coefficient hyperparameter feature transformation on the image feature y to obtain a coefficient hyperparameter feature z. The hyperparameter encoding network can be a trained neural network, and there is no restriction on the training process of the hyperparameter encoding network, as long as it can perform a coefficient hyperparameter feature transformation on the image feature y. The image feature y in the latent domain obtains the super-prior latent information z after passing through the hyperparameter encoding network.

[0075] After obtaining the coefficient hyperparameter feature z, the encoding end can quantize the coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, that is, Q in Figure 4 represents the quantization process. After obtaining the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain Bitstream#1 (that is, the first code stream) corresponding to the current image block, that is, AE in Figure 4 represents the encoding process, such as the entropy encoding process. Alternatively, the encoding end can also directly encode the coefficient hyperparameter feature z to obtain Bitstream#1 corresponding to the current image block. Among them, the hyperparameter quantization feature or coefficient hyperparameter feature z carried in Bitstream#1 is mainly used to obtain the parameters of the mean and probability distribution model.

[0076] After obtaining the Bitstream#1 corresponding to the current image block, the encoder can send the Bitstream#1 corresponding to the current image block to the decoder. For the processing process of the Bitstream#1 corresponding to the current image block by the decoder, please refer to the subsequent embodiments.

[0077] After obtaining Bitstream#1 corresponding to the current image block, the encoder can also decode Bitstream#1 to obtain the super-parameter quantization feature. That is, AD in Figure 4 represents the decoding process. Then, the encoder can dequantize the super-parameter quantization feature to obtain the coefficient super-parameter feature z_hat. The coefficient super-parameter feature z_hat can be the same as or different from the coefficient super-parameter feature z. IQ in Figure 4 represents the dequantization process. Alternatively, after obtaining Bitstream#1 corresponding to the current image block, the encoder can also decode Bitstream#1 to obtain the coefficient super-parameter feature z_hat without involving the dequantization process of the coefficient super-parameter feature z_hat.

[0078] For the encoding process of Bitstream#1, an encoding method of a fixed probability density model can be used. For the decoding process of Bitstream#1, a decoding method of a fixed probability density model can be used. There is no restriction on this encoding and decoding process.

[0079] After obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (see the subsequent embodiments for the determination process of the reconstruction feature y_hat) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat. There is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu. The predicted value mu is used to obtain the residual feature r by subtracting it from the original feature and is added to the decoded residual feature r_hat to obtain the reconstructed feature y_hat.

[0080] It should be noted that the mean prediction network is an optional neural network, that is, it is also possible to have no mean prediction network. In other words, it is not necessary to use the mean prediction network to determine the predicted value mu. The dotted box in Figure 4 indicates that the mean prediction network is optional.

[0081] After obtaining the image feature y, the encoding end can determine the residual feature r based on the image feature y and the predicted value mu, such as taking the difference between the image feature y and the predicted value mu as the residual feature r. Then, the residual feature r is feature processed to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the predicted value mu. Alternatively, after obtaining the image feature y, the encoding end can perform feature processing on the image feature y to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, there is no need to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0082] After obtaining image feature s, the encoder can quantize image feature s to obtain the image quantization feature corresponding to image feature s. In other words, Q in Figure 4 represents the quantization process. After obtaining the image quantization feature corresponding to image feature s, the encoder can encode the image quantization feature to obtain Bitstream #2 (i.e., the second bitstream) corresponding to the current image block. In other words, AE in Figure 4 represents the encoding process, such as the entropy encoding process. Alternatively, the encoder can directly encode image feature s to obtain Bitstream #2 corresponding to the current image block without involving the quantization process of image feature s.

[0083] After obtaining Bitstream#2 corresponding to the current image block, the encoder can send Bitstream#2 corresponding to the current image block to the decoder. For details about the processing of Bitstream#2 corresponding to the current image block by the decoder, see the subsequent embodiments.

[0084] After obtaining Bitstream#2 corresponding to the current image block, the encoder can also decode Bitstream#2 to obtain the image quantization features. AD in Figure 4 represents the decoding process. The encoder can then dequantize the image quantization features to obtain image features s'. Image features s' can be the same as or different from image features s. IQ in Figure 4 represents the dequantization process. Alternatively, after obtaining Bitstream#2 corresponding to the current image block, the encoder can also decode Bitstream#2 to obtain image features s' without involving the dequantization process of the image quantization features.

[0085] After obtaining the image feature s', the encoder can perform feature recovery on the image feature s' (i.e., the inverse process of feature processing) to obtain the residual feature r_hat. There is no restriction on this feature recovery process and it can be any feature recovery method. The residual feature r_hat and the residual feature r can be the same or different. After obtaining the residual feature r_hat, the encoder determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the prediction value mu. For example, the sum of the residual feature r_hat and the prediction value mu is used as the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, it is necessary to deploy a mean prediction network, which provides the prediction value mu. Alternatively, after obtaining the image feature s', the encoder performs feature recovery on the image feature s' (i.e., the inverse process of feature processing) to obtain the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, there is no need to deploy a mean prediction network. The dotted box indicates that the residual process is an optional process.

[0086] After obtaining the image feature y_hat, the encoder can perform a composition transformation on it to obtain the reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input to a composition transformation network, which then performs a composition transformation on it to obtain the reconstructed image block x_hat. This completes the image reconstruction process.

[0087] In one possible implementation, when the encoder encodes the image quantization feature or image feature s to obtain Bitstream #2 corresponding to the current image block, the encoder needs to first determine a probability distribution model and then encode the image quantization feature or image feature s based on the probability distribution model. Furthermore, when decoding Bitstream #2, the encoder also needs to first determine a probability distribution model and then decode Bitstream #2 based on the probability distribution model.

[0088] In order to obtain the probability distribution model, as shown in Figure 4, after obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters. For example, the coefficient hyperparameter feature z_hat is input into the probability hyperparameter decoding network, and the probability hyperparameter decoding network performs a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. After obtaining the probability distribution parameters, a probability distribution model can be generated based on the probability distribution parameters. Among them, the probability hyperparameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability hyperparameter decoding network. It can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters.

[0089] In one possible implementation, the processing at the encoding end may be performed by a deep learning model or a neural network model, thereby achieving an end-to-end image compression and encoding process, without limitation.

[0090] Embodiment 4: With respect to Embodiment 1 and Embodiment 2, the processing process of the decoding end can be referred to as shown in FIG5. Of course, FIG5 is only an example of the processing process of the decoding end and does not limit the processing process of the decoding end.

[0091] After obtaining Bitstream#1 corresponding to the current image block, the decoder can also decode Bitstream#1 to obtain the super-parameter quantization feature. That is, AD in Figure 5 represents the decoding process. Then, the decoder can dequantize the super-parameter quantization feature to obtain the coefficient super-parameter feature z_hat. The coefficient super-parameter feature z_hat can be the same as or different from the coefficient super-parameter feature z. IQ in Figure 5 represents the dequantization process. Alternatively, after obtaining Bitstream#1 corresponding to the current image block, the decoder can also decode Bitstream#1 to obtain the coefficient super-parameter feature z_hat without involving the dequantization process of the coefficient super-parameter feature z_hat.

[0092] For the decoding process of Bitstream#1, a decoding method using a fixed probability density model may be used, and there is no restriction on this.

[0093] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, then the current image block x can also be an image, that is, the decoding process for the image block can also be directly applied to the image.

[0094] After obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform a context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (the determination process of the reconstruction feature y_hat is described in subsequent embodiments) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat. There is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu.

[0095] It should be noted that the mean prediction network is an optional neural network, that is, it is also possible to have no mean prediction network. In other words, it is not necessary to use the mean prediction network to determine the predicted value mu. The dotted box in Figure 5 indicates that the mean prediction network is optional.

[0096] After obtaining Bitstream#2 corresponding to the current image block, the decoder can decode Bitstream#2 to obtain the image quantization features. AD in Figure 5 represents the decoding process. The decoder can then dequantize the image quantization features to obtain image features s'. Image features s' can be the same as or different from image features s. IQ in Figure 5 represents the dequantization process. Alternatively, after obtaining Bitstream#2 corresponding to the current image block, the decoder can decode Bitstream#2 to obtain image features s' without dequantizing the image quantization features.

[0097] After obtaining the image feature s', the decoding end can perform feature recovery on the image feature s' (i.e., the inverse process of feature processing) to obtain the residual feature r_hat. The residual feature r_hat and the residual feature r can be the same or different. After obtaining the residual feature r_hat, the decoding end determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the prediction value mu. For example, the sum of the residual feature r_hat and the prediction value mu is used as the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the prediction value mu. Alternatively, after obtaining the image feature s', the decoding end performs feature recovery on the image feature s' to obtain the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, there is no need to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0098] After obtaining the image feature y_hat, the decoder can perform a composition transformation on it to obtain the reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input into a composition transformation network, which then performs a composition transformation on it to obtain the reconstructed image block x_hat. This completes the image reconstruction process.

[0099] In one possible implementation, when decoding Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model. In order to obtain the probability distribution model, referring to FIG5 , after obtaining the coefficient super-parameter feature z_hat, the decoding end can perform a coefficient super-parameter feature inverse transformation on the coefficient super-parameter feature z_hat to obtain the probability distribution parameter sigma. For example, the coefficient super-parameter feature z_hat is input to the probability super-parameter decoding network, and the probability super-parameter decoding network performs a coefficient super-parameter feature inverse transformation on the coefficient super-parameter feature z_hat to obtain the probability distribution parameters. After obtaining the probability distribution parameters, a probability distribution model can be generated based on the probability distribution parameters. Among them, the probability super-parameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability super-parameter decoding network. It can perform a coefficient super-parameter feature inverse transformation on the coefficient super-parameter feature z_hat to obtain the probability distribution parameters.

[0100] In a possible implementation, the processing process at the decoding end may be performed by a deep learning model or a neural network model, thereby realizing an end-to-end image decoding process, without limitation.

[0101] Example 5: Based on Examples 3 and 4, a feature domain enhancement module can be added before the synthesis transformation network, as shown in Figure 6A. The input feature of the feature domain enhancement module can be the image feature y_hat (hereinafter referred to as the initial reconstruction feature y_hat), and the output feature of the feature domain enhancement module can be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input to the synthesis transformation network, which performs a synthesis transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0102] On the encoding side: After obtaining the current image block x, the analysis and transformation network performs an analysis and transformation on the current image block x to obtain the image feature y corresponding to the current image block x. The hyperparameter coding network performs a coefficient hyperparameter feature transformation on the image feature y to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block (which can be the coefficient hyperparameter feature z itself or the hyperparameter quantization feature of the coefficient hyperparameter feature z) is encoded to obtain the first bitstream corresponding to the current image block.

[0103] The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (e.g., the coefficient hyperparameter feature z_hat itself is decoded from the first bitstream, or the superparameter quantization feature is decoded from the first bitstream and the superparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat). Then, the coefficient hyperparameter feature z_hat is inversely transformed using a probabilistic hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0104] The initial image features corresponding to the current image block are encoded based on the probability distribution parameter sigma to obtain a second code stream corresponding to the current image block. The initial image features may be image features y, residual features r corresponding to image features y, image features s obtained by feature processing of image features y or residual features r, or image quantization features corresponding to image features y, residual features r, or image features s. There is no restriction on the initial image features.

[0105] The second bitstream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstructed feature y_hat corresponding to the current image block. For example, if the initial image feature is the image feature y, the initial reconstructed feature y_hat is decoded from the second bitstream. If the initial image feature is the image quantization feature corresponding to the image feature y, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the initial reconstructed feature y_hat. For another example, if the initial image feature is the residual feature r corresponding to the image feature y, the residual feature r_hat is decoded from the second bitstream, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the residual feature r, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the residual feature r_hat, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image feature y or the image feature s corresponding to the residual feature r, then image feature s' is decoded from the second bitstream, and feature recovery is performed on image feature s' to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the image feature s, then the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain image feature s'. Feature recovery is performed on image feature s' to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0106] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthetic transformation network, which performs synthetic transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0107] On the decoding end: Decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (e.g., decode the coefficient hyperparameter feature z_hat itself from the first bitstream, or decode the hyperparameter quantization feature from the first bitstream and dequantize the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat). Then, perform an inverse transform on the coefficient hyperparameter feature z_hat through the probabilistic hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0108] The second bitstream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstructed feature y_hat corresponding to the current image block. For example, the initial reconstructed feature y_hat is decoded from the second bitstream. Alternatively, the image quantization feature is decoded from the second bitstream and dequantized to obtain the initial reconstructed feature y_hat. In another example, the residual feature r_hat is decoded from the second bitstream, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second bitstream, and dequantized to obtain the residual feature r_hat. The initial reconstructed feature y_hat is determined based on the residual feature r_hat and the predicted value mu. In another example, the image feature s' is decoded from the second bitstream, and feature recovery is performed on the image feature s' to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second code stream, and the image quantization feature is dequantized to obtain the image feature s', and the image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0109] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthetic transformation network, which performs synthetic transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0110] Example 6: Based on Example 3 and Example 4, a feature domain enhancement module can be added before the synthetic transformation network, and an image domain enhancement module can be added after the synthetic transformation network, as shown in Figure 6B. The input feature of the feature domain enhancement module is the initial reconstructed feature y_hat, and the output feature of the feature domain enhancement module can be the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced can be input to the synthetic transformation network, and the synthetic transformation network performs a synthetic transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x. The input feature of the image domain enhancement module can be the initial reconstructed image block x_hat, and the output feature of the image domain enhancement module can be the target reconstructed image block x_hat_enhanced. In other words, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat, thereby obtaining the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0111] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient superparameter feature transformation through the hyperparameter coding network to obtain the coefficient superparameter feature z. The coefficient superparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient superparameter feature z_hat corresponding to the current image block, and the coefficient superparameter feature z_hat is inversely transformed through the probability superparameter decoding network to obtain the probability distribution parameter sigma. The initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma to obtain the second code stream corresponding to the current image block. The second code stream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5 and will not be repeated here.

[0112] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0113] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0114] On the decoding end: The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block. The coefficient hyperparameter feature z_hat is then inversely transformed using a probabilistic hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the second bitstream corresponding to the current image block is decoded to obtain the initial reconstructed feature y_hat corresponding to the current image block. The above process can be found in Example 5 and will not be repeated here.

[0115] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0116] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0117] Example 7: Based on Examples 3 and 4, an image domain enhancement module can be added after the synthetic transformation network, as shown in Figure 6C. The input features of the image domain enhancement module are the initial reconstructed image block x_hat, and the output features of the image domain enhancement module are the target reconstructed image block x_hat_enhanced. In other words, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0118] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient superparameter feature transformation through the hyperparameter coding network to obtain the coefficient superparameter feature z. The coefficient superparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient superparameter feature z_hat corresponding to the current image block, and the coefficient superparameter feature z_hat is inversely transformed through the probability superparameter decoding network to obtain the probability distribution parameter sigma. The initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma to obtain the second code stream corresponding to the current image block. The second code stream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5 and will not be repeated here.

[0119] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the synthesis transformation network, which performs a synthesis transformation on the initial reconstructed feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x. After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0120] On the decoding end: Decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block. Perform an inverse transformation on the coefficient hyperparameter feature z_hat using the probabilistic hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, decode the second bitstream corresponding to the current image block to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be found in Example 5 and will not be repeated here.

[0121] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the synthesis transformation network, which performs a synthesis transformation on the initial reconstructed feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x. After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0122] Example 8: In Example 5 and Example 6, the initial reconstructed feature y_hat is input into the feature domain enhancement module, and the feature domain enhancement module performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. For example, the initial reconstructed feature y_hat can be enhanced in the feature domain based on the feature domain enhancement parameters and probability distribution parameter sigma corresponding to the current image block to obtain the enhanced reconstructed feature y_hat_enhanced. Feature domain enhancement can include feature adaptive edge enhancement and feature adaptive stretching. The probability distribution parameter sigma is used to assist in calculation, determine the channels that require feature adaptive edge enhancement and the channels that require feature adaptive stretching, and determine the strength of stretching. This process is described below.

[0123] The initial reconstruction feature y_hat is C L ×H L ×W L The three-dimensional tensor, C L is the number of channels of the initial reconstructed feature y_hat (i.e., the number of channels in the feature domain or the number of channels in the Latent domain), H L is the feature height of the initial reconstructed feature y_hat (i.e., the feature height of the feature domain or the feature height of the Latent domain), W L is the feature width of the initial reconstructed feature y_hat (i.e., the feature width of the feature domain or the feature width of the Latent domain). For each channel of the initial reconstructed feature y_hat, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of the channel ch is H L ×W L The two-dimensional tensor y_hat_ch is called the feature channel map. Therefore, the initial reconstruction feature y_hat can include C L Feature channel map.

[0124] The probability distribution parameter sigma is also C L ×H L ×W L The three-dimensional tensor, C Lis the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. For each channel of the probability distribution parameter sigma, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of the channel ch is H L ×W L The two-dimensional tensor sigma_ch is called the probability distribution channel map. Therefore, the probability distribution parameter sigma can include C L probability distribution channel graph, and C L The probability distribution channel graph and C L The feature channel maps correspond one to one, such as the first probability distribution channel map corresponds to the first feature channel map, the second probability distribution channel map corresponds to the second feature channel map, and so on. L The probability distribution channel diagram and the C L The feature channel map corresponds to

[0125] Targeting C L feature channel map, C L The feature channel map is divided into important feature channel maps and unimportant feature channel maps. There can be at least one important feature channel map and multiple unimportant feature channel maps. For example, C L At least one feature channel map is selected from the feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as non-important feature channel maps. L The probability distribution channel graph and C L The feature channel maps are in one-to-one correspondence. Therefore, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map. For example, if the first feature channel map is an important feature channel map, the first probability distribution channel map corresponding to the first feature channel map is selected as the important probability distribution channel map.

[0126] In summary, the initial reconstructed feature y_hat may include an important feature channel map and an unimportant feature channel map, and the probability distribution parameter sigma may include an important probability distribution channel map corresponding to the important feature channel map and an unimportant probability distribution channel map corresponding to the unimportant feature channel map. For the important feature channel map, feature-adaptive edge enhancement can be performed on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstructed feature after feature-adaptive edge enhancement. For the unimportant feature channel map, feature-adaptive stretching can be performed on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature-adaptive stretching. On this basis, an enhanced reconstructed feature y_hat_enhanced can be generated based on the first reconstructed feature and the second reconstructed feature.

[0127] For example, assuming that feature channel map 1 is an important feature channel map, feature channel maps 2 to feature channel maps 4 are unimportant feature channel maps, probability distribution channel map 1 is an important probability distribution channel map, and probability distribution channel maps 2 to probability distribution channel maps 4 are unimportant probability distribution channel maps, then: feature channel map 1 can be feature-adaptively edge enhanced based on the feature domain enhancement parameters and probability distribution channel map 1 to obtain the first reconstructed feature 1 after feature-adaptive edge enhancement (i.e., the reconstructed feature of the first channel). Feature channel map 2 can be feature-adaptively stretched based on the feature domain enhancement parameters and probability distribution channel map 2 to obtain the second reconstructed feature 2 after feature-adaptive stretching (i.e., the reconstructed feature of the second channel). Feature channel map 3 can be feature-adaptively stretched based on the feature domain enhancement parameters and probability distribution channel map 3 to obtain the second reconstructed feature 3 after feature-adaptive stretching (i.e., the reconstructed feature of the third channel). Feature channel map 4 can be feature-adaptively stretched based on the feature domain enhancement parameters and probability distribution channel map 4 to obtain the second reconstructed feature 4 after feature-adaptive stretching (i.e., the reconstructed feature of the fourth channel). Then, the first reconstruction feature 1, the second reconstruction feature 2, the second reconstruction feature 3 and the second reconstruction feature 4 may be combined according to channels (eg, spliced ​​according to channels), that is, the reconstruction features of the four channels may be combined to obtain the enhanced reconstruction feature y_hat_enhanced.

[0128] Example 9: In Example 8, it is necessary to L The feature channel maps are divided into important feature channel maps and non-important feature channel maps. For example, the following method is used to divide C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps.

[0129] Mode 1: The encoder encodes the important channel identifier in the third bitstream corresponding to the current image block, and the decoder decodes the important channel identifier from the third bitstream corresponding to the current image block.

[0130] For example, the encoding end will C L The feature channel maps are divided into important feature channel maps and non-important feature channel maps. There is no restriction on this process, and the important channel identifier (also called important channel number, recorded as important_channel) is encoded in the third bit stream corresponding to the current image block. After receiving the third bit stream, the decoding end decodes the important channel identifier from the third bit stream and obtains the important channel identifier from C L Among the feature channel maps, the feature channel map corresponding to the important channel identifier is selected as the important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0131] Among them, the encoding end is C L When the feature channel graphs are divided into important feature channel graphs and unimportant feature channel graphs, the feature values ​​in each feature channel graph and the probability distribution values ​​in each probability distribution channel graph can be used to calculate the feature channel graphs from C. L The important feature channel maps are selected from the feature channel maps, and the remaining feature channel maps are selected as non-important feature channel maps.

[0132] Method 2: For both the encoding and decoding ends, based on the feature values ​​in each feature channel map and the probability distribution values ​​in each probability distribution channel map, L Select the important feature channel maps from the feature channel maps.

[0133] For example, for each feature channel map, the number of consumed bits corresponding to the feature channel map can be determined based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map. For example, the number of consumed bits corresponding to the feature channel map can be determined using the following expression, which is only an example.

[0134] In the above expression, bits_per_ch is used to represent the number of consumed bits corresponding to the feature channel map ch, y_hat_ch(i,j) is used to represent the eigenvalue of the feature point (i,j) in the feature channel map ch, sigma_ch(i,j) is used to represent the probability distribution value of the feature point (i,j) in the probability distribution channel map ch, and Φ(.) is the standard normal distribution function.

[0135] After performing the above processing on each feature channel map, the number of consumed bits corresponding to each feature channel map can be obtained. Based on the number of consumed bits corresponding to each feature channel map, the number of consumed bits corresponding to each feature channel map can be obtained from C LFor example, the feature channel map with the largest bit consumption is selected as the important feature channel map, or the K feature channel maps with the largest bit consumption are selected as the important feature channel maps, where K can be a positive integer greater than 1.

[0136] In C L After selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as non-important feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the non-important feature channel map can be selected as the non-important probability distribution channel map.

[0137] Method 3: For both the encoding and decoding ends, the consumption bit rate corresponding to each feature channel map is calculated from C L For example, based on the consumption bit rate corresponding to each feature channel map, the C L The feature channel maps are sorted and the top K feature channel maps are selected as important feature channel maps. Alternatively, based on the consumption bit rate corresponding to each feature channel map, the C feature channel maps can be sorted in the order of consumption bit rate from small to large. L The feature channel maps are sorted and the K feature channel maps with the lowest sorting order are selected as important feature channel maps.

[0138] In C L After selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as non-important feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the non-important feature channel map can be selected as the non-important probability distribution channel map.

[0139] Method 4: For both the encoding and decoding ends, the default feature channel map (i.e., fixed feature channel map) is used as the important feature channel map. For example, the first feature channel map is pre-agreed as the important feature channel map, or the sixth feature channel map is pre-agreed as the important feature channel map, or the tenth feature channel map is pre-agreed as the important feature channel map. Of course, any feature channel map can be used as the important feature channel map, and there is no restriction on this.

[0140] Example 10: In Example 8, feature-adaptive edge enhancement is performed on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain the first reconstructed feature. The feature-adaptive edge enhancement process is described below.

[0141] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. For example, the encoder encodes multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block, and the decoder decodes the multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds from the third bitstream corresponding to the current image block, and the multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds are used as the feature domain enhancement parameters.

[0142] The number of edge enhancement segment strength values ​​and the number of edge enhancement segment thresholds may be the same or different. Taking the case where the two are the same as each other, the encoding end may also encode the number of edge enhancement segment strength values ​​(or edge enhancement segment thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of edge enhancement segment strength values ​​from the third bitstream corresponding to the current image block. Taking the case where the number of edge enhancement segment strength values ​​is n as an example, the n edge enhancement segment strength values ​​may be recorded as magl-1, magl-2, ..., magl-n, and the n edge enhancement segment thresholds may be recorded as thrl-1, thrl-2, ..., thrl-n.

[0143] Illustratively, multiple edge enhancement segmentation thresholds may constitute multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. See Table 1 for an example of the correspondence.

[0144] Table 1

[0145] Exemplarily, the important probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the edge enhancement threshold interval corresponding to the probability distribution value is first determined, and the edge enhancement segmentation strength value corresponding to the probability distribution value is determined based on the edge enhancement threshold interval. For example, if the probability distribution value is within [thrl-2, thrl-3), the edge enhancement segmentation strength value corresponding to the probability distribution value is magl-2; if the probability distribution value is within [thrl-3, thrl-4), the edge enhancement segmentation strength value corresponding to the probability distribution value is magl-3, and so on. After obtaining the edge enhancement segmentation strength value corresponding to each probability distribution value, feature-adaptive edge enhancement can be performed on the important feature channel map based on the edge enhancement segmentation strength value corresponding to each probability distribution value to obtain the first reconstructed feature after feature-adaptive edge enhancement.

[0146] For example, the following steps S11-S15 may be used to perform feature-adaptive edge enhancement on the important feature channel map.

[0147] In step S11, the important feature channel map is normalized to obtain a normalized feature map.

[0148] Exemplarily, normalization can be performed based on the important feature channel map, the mean feature corresponding to the important feature channel map, and the variance feature corresponding to the important feature channel map to obtain a normalized feature map. For example, the important feature channel map can be subtracted from the mean feature and then divided by the variance feature to obtain a normalized feature map. For another example, the important feature channel map can be subtracted from the mean feature and then divided by the variance feature to obtain an intermediate feature, and then the intermediate feature is converted to obtain a normalized feature map, such as multiplying the intermediate feature by 0.1 and adding 0.5 (0.1 and 0.5 are both examples), and then limiting the feature value to between 0 and 1. Of course, the above are just a few examples of normalizing the important feature channel map, and there is no limitation on this normalization method.

[0149] In step S12, a high-frequency detail image is generated based on the important feature channel map and the normalized feature map.

[0150] For example, a convolution operation (e.g., a two-dimensional convolution operation) can be performed on the normalized feature map with a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no limitation on this convolution kernel) to obtain a Gaussian blurred image. Then, the Gaussian blurred image is subtracted from the important feature channel map to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image. As long as the high-frequency details of the important feature channel map can be obtained, there is no limitation on this.

[0151] In step S13, for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue.

[0152] Exemplarily, the important feature channel map includes multiple eigenvalues, the important probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond one-to-one to the multiple eigenvalues. Since the high-frequency detail image includes multiple eigenvalues ​​(which correspond one-to-one to the multiple eigenvalues ​​of the important feature channel map), the multiple probability distribution values ​​of the important probability distribution channel map correspond one-to-one to the multiple eigenvalues ​​of the high-frequency detail image. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the important probability distribution channel map, and the edge enhancement threshold interval corresponding to the probability distribution value can be determined. Based on the edge enhancement threshold interval, the edge enhancement segment strength value corresponding to the probability distribution value is determined.

[0153] After obtaining the edge enhancement segmentation strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the edge enhancement segmentation strength value to obtain an edge enhancement eigenvalue. For example, the edge enhancement eigenvalue is multiplied by the edge enhancement segmentation strength value (such as magl-1, magl-2, etc.) to obtain an edge enhancement eigenvalue. Obviously, if the edge enhancement threshold interval is "less than thrl-1", the eigenvalue is multiplied by 1, that is, the eigenvalue remains unchanged, and the eigenvalue is not edge enhanced. If the edge enhancement threshold interval is [thrl-1, thrl-2), the eigenvalue is multiplied by magl-1, magl-1 is a numerical value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other edge enhancement segmentation strength values ​​such as magl-2 are greater than 1, which can achieve edge enhancement.

[0154] For each eigenvalue in the high-frequency detail image, the above method can be used to obtain the edge enhancement segment strength value corresponding to the eigenvalue, and then the edge enhancement is performed on the eigenvalue based on the edge enhancement segment strength value to obtain the edge enhancement eigenvalue.

[0155] In step S14, an edge enhancement feature map is determined based on the edge enhancement feature value corresponding to each feature value. For example, the edge enhancement feature values ​​corresponding to all feature values ​​can be combined to obtain the edge enhancement feature map.

[0156] In step S15, the edge enhancement feature map is denormalized to obtain a first reconstructed feature (ie, a first reconstructed feature map). The first reconstructed feature (map) is a reconstructed feature (map) after feature-adaptive edge enhancement is performed on the important feature channel map.

[0157] After obtaining the edge enhancement feature map, the edge enhancement feature map can be directly denormalized to obtain a first reconstructed feature map. Alternatively, after obtaining the edge enhancement feature map, the edge enhancement feature map can be added to the normalized feature map to obtain a modified edge enhancement feature map, and the modified edge enhancement feature map can be denormalized to obtain the first reconstructed feature map.

[0158] For example, an edge-enhanced normalized feature map can be determined based on the edge-enhanced feature map, and a first reconstructed feature map can be determined based on the edge-enhanced normalized feature map, a mean feature corresponding to the edge-enhanced normalized feature map, and a variance feature corresponding to the edge-enhanced normalized feature map. The first reconstructed feature can also be referred to as y_hat_sharp.

[0159] For example, the edge-enhanced feature map can be transformed to obtain an edge-enhanced normalized feature map. For example, the eigenvalues ​​in the edge-enhanced feature map are first limited to between 0 and 1, and then the eigenvalues ​​in the edge-enhanced feature map are subtracted by 0.5 and divided by 0.1 (0.5 and 0.1 are both examples) to obtain the edge-enhanced normalized feature map.

[0160] The normalized feature map after edge enhancement is multiplied by the variance feature, and then the mean feature is added to obtain the edge-enhanced feature map corresponding to the important feature channel map. The edge-enhanced feature map is recorded as the first reconstructed feature y_hat_sharp.

[0161] Of course, the above is only an example of denormalizing the edge enhancement feature map, and there is no limitation on the denormalization method.

[0162] In a possible implementation, the encoder needs to encode multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder may adopt the following approach.

[0163] The encoding end can configure multiple candidate feature domain enhancement parameters. For each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. The cost value corresponding to each candidate feature domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block can be selected from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the smallest cost value. The encoding end can encode the feature domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0164] For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature can be enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature. The feature domain enhancement process is described in the above embodiment. A target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed feature. A cost value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced. The method for determining this cost value is not limited.

[0165] For example, for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes magl-1, magl-2, ..., magl-n, thrl-1, thrl-2, ..., thrl-n. Based on the candidate feature domain enhancement parameter, the enhanced reconstruction feature can be obtained, and then the target reconstructed image block x_hat_enhanced is obtained. The distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter corresponding to the minimum distortion index value can be selected as the feature domain enhancement parameter corresponding to the current image block, that is, the optimal feature domain enhancement parameter. In this way, the encoding end encodes the feature domain enhancement parameter in the third code stream.

[0166] Example 11: In Example 8, it is necessary to perform feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameters and the non-important probability distribution channel map (i.e., the non-important probability distribution channel map corresponding to the non-important feature channel map) to obtain the second reconstructed feature after feature adaptive stretching. The feature adaptive stretching process is described below.

[0167] Exemplarily, the feature domain enhancement parameter may include a stretch parameter value. For example, the encoder may encode the stretch parameter value in the third bitstream corresponding to the current image block, and the decoder may decode the stretch parameter value from the third bitstream corresponding to the current image block. The stretch parameter value is used as the feature domain enhancement parameter and may be denoted as ρ.

[0168] Exemplarily, the non-important feature channel map may include multiple eigenvalues, and the non-important probability distribution channel map corresponding to the non-important feature channel map may include multiple probability distribution values, and the multiple probability distribution values ​​correspond one-to-one to the multiple eigenvalues. Based on this, for each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value, and the probability distribution value corresponding to the eigenvalue. For example, the following expression can be used to determine the stretched eigenvalue corresponding to the eigenvalue. Of course, the following expression is only an example and is not limited to this. y_hat_scale=y_hat+ρ*clip3(sigma*y_hat,-0.5,0.5)

[0169] In the above expression, y_hat_scale represents the stretched eigenvalue, y_hat represents the eigenvalue in the unimportant feature channel map, ρ represents the stretching parameter value, sigma represents the probability distribution value in the unimportant probability distribution channel map, and the probability distribution value sigma corresponds to the eigenvalue y_hat. Clip3 is a limit operation used to limit sigma*y_hat to between -0.5 and 0.5. -0.5 and 0.5 are configurable values ​​and are not restricted. For example, when sigma*y_hat is less than -0.5, sigma*y_hat is limited to -0.5. When sigma*y_hat is greater than 0.5, sigma*y_hat is limited to 0.5. When sigma*y_hat is greater than or equal to -0.5 and less than or equal to 0.5, the value of sigma*y_hat remains unchanged.

[0170] After performing the above processing on each eigenvalue in the non-significant feature channel map, a stretched eigenvalue corresponding to each eigenvalue can be obtained, and then a second reconstructed feature (i.e., a second reconstructed feature map) is determined based on the stretched eigenvalue corresponding to each eigenvalue. The second reconstructed feature (map) is the reconstructed feature (map) after feature-adaptive stretching of the non-significant feature channel map. For example, the second reconstructed feature map is obtained by combining the stretched eigenvalues ​​corresponding to all eigenvalues. The second reconstructed feature is also called y_hat_scale.

[0171] The second reconstructed feature y_hat_scale corresponds to the non-important feature channel map of the initial reconstructed feature y_hat. That is, the non-important feature channel map is stretched to obtain the second reconstructed feature y_hat_scale. The first reconstructed feature y_hat_sharp corresponds to the important feature channel map of the initial reconstructed feature y_hat. That is, the important feature channel map is enhanced to obtain the first reconstructed feature y_hat_sharp. The second reconstructed feature y_hat_scale and the first reconstructed feature y_hat_sharp are combined to form the enhanced reconstructed feature y_hat_enhanced.

[0172] In one possible implementation, the encoder needs to encode a stretching parameter value in a third codestream corresponding to the current image block. For this process, the encoder may employ the following approach: the encoder may configure multiple candidate stretching parameter values, determine a cost value corresponding to each candidate stretching parameter value, and based on the cost value corresponding to each candidate stretching parameter value, select a stretching parameter value corresponding to the current image block from all candidate stretching parameter values, i.e., a candidate stretching parameter value with the smallest cost value. The encoder may then encode the stretching parameter value to obtain a third codestream corresponding to the current image block.

[0173] For example, for each candidate stretching parameter value, feature adaptive stretching can be performed on the non-important feature channel map based on the candidate stretching parameter value and the non-important probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching, and then an enhanced reconstruction feature is obtained based on the second reconstruction feature. The target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstruction feature, and the cost value corresponding to the candidate stretching parameter value is determined based on the target reconstructed image block x_hat_enhanced.

[0174] For example, for each candidate stretch parameter value, an enhanced reconstruction feature can be obtained based on the candidate stretch parameter value, and then the target reconstructed image block x_hat_enhanced can be obtained. The distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate stretch parameter value, the encoder can select the candidate stretch parameter value corresponding to the minimum distortion index value as the stretch parameter value corresponding to the current image block, i.e., the optimal stretch parameter value. The encoder can encode this stretch parameter value in the third bitstream.

[0175] In another possible embodiment, the encoder configures multiple candidate feature domain enhancement parameters. For each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds, and a stretching parameter value. A cost value corresponding to each candidate feature domain enhancement parameter is determined. Based on the cost value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block is selected from all candidate feature domain enhancement parameters, i.e., the candidate feature domain enhancement parameter with the smallest cost value. The encoder encodes the feature domain enhancement parameter (i.e., multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds, and a stretching parameter value) to obtain a third bitstream corresponding to the current image block. For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature is enhanced based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature. A target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed feature. The cost value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced.

[0176] Example 12: In Examples 6 and 7, the initial reconstructed image block x_hat is input to an image domain enhancement module, which performs image domain enhancement on the initial reconstructed image block x_hat to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block x. For example, image-adaptive edge enhancement can be performed on the initial reconstructed image block x_hat based on the image domain enhancement parameters and probability distribution parameter sigma corresponding to the current image block to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block. This image-adaptive edge enhancement process is described below.

[0177] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds. For example, the encoder encodes multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds in the third bitstream corresponding to the current image block, and the decoder decodes the multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds from the third bitstream corresponding to the current image block, and the multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds are used as the image domain enhancement parameters.

[0178] The number of image enhancement segmentation strength values ​​and the number of image enhancement segmentation thresholds may be the same or different. Taking the case where the two are the same as one another, the encoding end may also encode the number of image enhancement segmentation strength values ​​(or image enhancement segmentation thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of image enhancement segmentation strength values ​​from the third bitstream corresponding to the current image block. Taking the case where the number of image enhancement segmentation strength values ​​is m as an example, the m image enhancement segmentation strength values ​​are recorded as magy-1, magy-2, ..., magy-m, and the m image enhancement segmentation thresholds are recorded as thry-1, thry-2, ..., thry-m.

[0179] Exemplarily, multiple image enhancement segmentation thresholds may constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation strength values. See Table 2 for an example of this correspondence.

[0180] Table 2

[0181] Exemplarily, the following steps S21-S23 may be used to perform image adaptive edge enhancement on the initial reconstructed image block x_hat.

[0182] In step S21, a target probability distribution channel map is obtained based on the probability distribution parameters.

[0183] The initial reconstructed image block x_hat is a two-dimensional tensor of H×W, where H is the image height of the initial reconstructed image block x_hat and W is the image width of the initial reconstructed image block x_hat. The probability distribution parameter sigma is C L ×H L ×W L The three-dimensional tensor, C L is the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. The image height H of the initial reconstructed image block x_hat can be greater than the characteristic height H of the probability distribution parameter sigma. L , such as the image height H is the feature height H L 4 times, 8 times, 16 times, etc. The image width W of the initial reconstructed image block x_hat can be greater than the feature width W of the probability distribution parameter sigma L , such as the image width W is the feature width W L 4 times, 8 times, 16 times, etc.

[0184] In order to perform image adaptive edge enhancement on the initial reconstructed image block x_hat using probability distribution parameters, it is necessary to obtain a target probability distribution channel map based on the probability distribution parameters. The target probability distribution channel map is a two-dimensional tensor of H×W.

[0185] For example, you can start from C L Select a probability distribution channel graph from the probability distribution channel graphs. Since the probability distribution parameter sigma includes important probability distribution channel graphs (at least one) and non-important probability distribution channel graphs (multiple), it can be obtained from C L Select an important probability distribution channel diagram from the probability distribution channel diagram, or you can select an important probability distribution channel diagram from C L Select a non-important probability distribution channel graph from the probability distribution channel graphs. The following explanation will be given using the example of selecting an important probability distribution channel graph.

[0186] Then, the important probability distribution channel map is upsampled to obtain the target probability distribution channel map. There is no restriction on the upsampling method, and the size of the target probability distribution channel map can be the same as the size of the initial reconstructed image block x_hat.

[0187] For example, the nearest neighbor upsampling method can be used to upsample the important probability distribution channel map to obtain the target probability distribution channel map. For the nearest neighbor upsampling method, a pixel can be selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding area on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is used as the value of the corresponding pixel on the target high-resolution image. Repeat the above process until all pixels of the target high-resolution image are assigned values.

[0188] In summary, the important probability distribution channel map can be upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_ch_upscale.

[0189] In step S22, if the target probability distribution channel map includes multiple probability distribution values, for each probability distribution value, the image enhancement segment intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value.

[0190] Exemplarily, the target probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the image enhancement threshold interval corresponding to the probability distribution value may be determined first, and the image enhancement segmentation intensity value corresponding to the probability distribution value may be determined based on the image enhancement threshold interval. For example, if the probability distribution value is within [thry-2, thry-3), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-2; if the probability distribution value is within [thry-3, thry-4), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-3, and so on.

[0191] In step S23 , image adaptive edge enhancement is performed on the initial reconstructed image block x_hat based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block.

[0192] First, a high-frequency detail image can be generated based on the initial reconstructed image block x_hat. For example, a convolution operation (such as a two-dimensional convolution operation) can be performed on the initial reconstructed image block x_hat with a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no restriction on this convolution kernel) to obtain a Gaussian blurred image. Then, the Gaussian blurred image is subtracted from the initial reconstructed image block x_hat to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image. As long as the high-frequency details of the initial reconstructed image block x_hat can be obtained, there is no restriction on this.

[0193] Then, for each eigenvalue in the high-frequency detail image, the edge enhancement of the eigenvalue is performed based on the image enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain the image enhancement eigenvalue. For example, the initial reconstructed image block x_hat includes multiple eigenvalues, the target probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Since the high-frequency detail image includes multiple eigenvalues ​​(one-to-one corresponding to the multiple eigenvalues ​​of the initial reconstructed image block x_hat), the multiple probability distribution values ​​of the target probability distribution channel map correspond to the multiple eigenvalues ​​of the high-frequency detail image. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the target probability distribution channel map, and the image enhancement threshold interval corresponding to the probability distribution value can be determined, and then the image enhancement segmentation intensity value corresponding to the probability distribution value can be determined based on the image enhancement threshold interval.

[0194] After obtaining the image enhancement segmentation strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the image enhancement segmentation strength value to obtain an image enhancement eigenvalue. For example, the image enhancement eigenvalue is multiplied by the image enhancement segmentation strength value (such as magy-1, magy-2, etc.) to obtain an image enhancement eigenvalue. Obviously, if the image enhancement threshold interval is "less than thry-1", the eigenvalue is multiplied by 1, that is, the eigenvalue remains unchanged, and the eigenvalue is not edge enhanced. If the image enhancement threshold interval is [thry-1, thry-2), the eigenvalue is multiplied by magy-1, magy-1 is a value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other image enhancement segmentation strength values ​​such as magy-2 are greater than 1, which can achieve edge enhancement.

[0195] Then, the target reconstructed image block x_hat_enhanced is determined based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image. For example, for each eigenvalue in the high-frequency detail image, the image enhancement eigenvalue corresponding to the eigenvalue can be obtained in the above manner, and then the image enhancement feature map can be determined based on the image enhancement eigenvalue corresponding to each eigenvalue. For example, the image enhancement eigenvalues ​​corresponding to all eigenvalues ​​can be combined to obtain an image enhancement feature map. Then, the image enhancement feature map is added to the initial reconstructed image block x_hat to obtain the final edge-enhanced image, and the edge-enhanced image is then limited to the value range of the image to obtain the target reconstructed image block x_hat_enhanced.

[0196] In a possible implementation, the encoder needs to encode multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder can adopt the following method.

[0197] The encoding end can configure multiple candidate image domain enhancement parameters. For each candidate image domain enhancement parameter, the candidate image domain enhancement parameter may include multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds. The cost value corresponding to each candidate image domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate image domain enhancement parameter, the image domain enhancement parameter corresponding to the current image block can be selected from all candidate image domain enhancement parameters, that is, the candidate image domain enhancement parameter with the smallest cost value. The encoding end can encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0198] For example, for each candidate image domain enhancement parameter, image adaptive edge enhancement is performed on the initial reconstructed image block x_hat based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target reconstructed image block x_hat_enhanced corresponding to the current image block. The image adaptive edge enhancement process is described in the above embodiment and is not further described here. A cost value corresponding to the candidate image domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced, and the method for determining this cost value is not limited.

[0199] For example, for each candidate image domain enhancement parameter, the candidate image domain enhancement parameters include magy-1, magy-2, ..., magy-m, thry-1, thry-2, ..., thry-m. Based on the candidate image domain enhancement parameters, the target reconstructed image block x_hat_enhanced can be obtained. The distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index. After obtaining the distortion index value corresponding to each candidate image domain enhancement parameter, the candidate image domain enhancement parameter corresponding to the minimum distortion index value can be selected as the image domain enhancement parameter corresponding to the current image block, that is, the optimal image domain enhancement parameter. In this way, the encoding end can encode the image domain enhancement parameter in the third code stream.

[0200] Example 13: For Example 5 to Example 12, if a feature domain enhancement module is added before the synthetic transformation network, then the feature domain enhancement module can perform feature domain enhancement on the luminance component, can also perform feature domain enhancement on the chrominance component, and can also perform feature domain enhancement on the luminance component and the chrominance component at the same time. In this embodiment, the feature domain enhancement of the luminance component is taken as an example. In order to perform feature domain enhancement on the luminance component, the initial reconstructed feature y_hat corresponding to the luminance component of the current image block can be feature domain enhanced based on the feature domain enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed feature y_hat_enhanced corresponding to the luminance component. The feature domain enhancement process can be referred to Example 5 to Example 12, and will not be repeated here.

[0201] For Example 5 to Example 12, if an image domain enhancement module is added after the synthetic transformation network, then the image domain enhancement module can perform image domain enhancement on the luminance component, can also perform image domain enhancement on the chrominance component, and can also perform image domain enhancement on the luminance component and the chrominance component at the same time. In this embodiment, the image domain enhancement of the luminance component and the chrominance component is taken as an example. For example, in order to perform image domain enhancement on the luminance component, the initial reconstructed image block x_hat corresponding to the luminance component of the current image block can be image-adaptive edge enhanced based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block x_hat_enhanced corresponding to the luminance component. The image domain enhancement process can be referred to Example 5 to Example 12, and will not be repeated here. In addition, in order to perform image domain enhancement on the chrominance component, the initial reconstructed image block x_hat corresponding to the chrominance component of the current image block can be image-adaptive edge enhanced based on the image domain enhancement parameters and the probability distribution parameters to obtain the target reconstructed image block x_hat_enhanced corresponding to the chrominance component.

[0202] When performing image-adaptive edge enhancement on the initial reconstructed image block x_hat corresponding to the chrominance component, a target probability distribution channel map for the chrominance component can be obtained based on the probability distribution parameters of the luminance component. For example, a significant probability distribution channel map is selected from all probability distribution channel maps of the luminance component's probability distribution parameters and upsampled to obtain a target probability distribution channel map for the chrominance component. The size of this target probability distribution channel map is the same as that of the initial reconstructed image block x_hat corresponding to the chrominance component. For example, the significant probability distribution channel map is upsampled using the nearest neighbor upsampling method to obtain the target probability distribution channel map for the chrominance component. For the nearest neighbor upsampling method, a pixel in the original low-resolution image (i.e., the significant probability distribution channel map) is selected as the center point of the corresponding region in the target high-resolution image (i.e., the target probability distribution channel map). The values ​​of the pixels closest to this pixel in the original low-resolution image are used as the values ​​of the corresponding pixels in the target high-resolution image. This process is repeated until all pixels in the target high-resolution image have been assigned values.

[0203] After obtaining the target probability distribution channel map of the chrominance component, for each probability distribution value in the target probability distribution channel map, an image enhancement segment intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value. Based on the image enhancement segment intensity value corresponding to each probability distribution value, image adaptive edge enhancement is performed on the initial reconstructed image block x_hat corresponding to the chrominance component to obtain a target reconstructed image block x_hat_enhanced corresponding to the chrominance component. The image adaptive edge enhancement process can be referred to in Example 12, and the relevant operations can be performed on the chrominance component.

[0204] Embodiment 14: This embodiment proposes a decoding method, which may include the following steps S31 to S38.

[0205] In step S31, the feature domain enhancement parameters and the image domain enhancement parameters are decoded from the Bitstream#3 (i.e., the third bitstream, which may also be referred to as the feature enhancement header information bitstream) corresponding to the current image block. The feature domain enhancement parameters may include an important channel identifier (important channel number important_channel), multiple edge enhancement segmentation strength values ​​(denoted as magl-1, magl-2, ..., magl-n), multiple edge enhancement segmentation thresholds (denoted as thrl-1, thrl-2, ..., thrl-n), and a stretching parameter value ρ; the image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​(denoted as magy-1, magy-2, ..., magy-m), and multiple image enhancement segmentation thresholds (denoted as thry-1, thry-2, ..., thry-m). The feature domain enhancement parameters may also include the number n of edge enhancement segmentation strength values, and the image domain enhancement parameters may also include the number m of image enhancement segmentation strength values.

[0206] In step S32, the input features of the feature domain enhancement module are the initial reconstruction feature y_hat and the probability distribution parameter sigma. The initial reconstruction feature y_hat is a feature with a size of C L ×H L ×W L The three-dimensional tensor, C L is the number of channels in the feature domain, namely the Latent domain, H L is the characteristic height of the Latent domain, W L is the characteristic width of the Latent domain. The probability distribution parameter sigma is the size of C L ×H L ×W L When the channel of the initial reconstruction feature y_hat is [1, 2, ..., C L ], the shape is H L ×W L The two-dimensional tensor y_hat_ch is called the feature channel map. When the channel of the probability distribution parameter sigma is [1, 2, ..., C L ], the shape is H L ×W L The two-dimensional tensor sigma_ch is called the probability distribution channel map.

[0207] In step S33, the initial reconstruction feature y_hat has a total of C LThere are C feature channel maps, and the probability distribution parameter sigma has a total of C probability distribution channel maps, C L The probability distribution channel graph and C L The probability distribution channel map is represented as sigma_ch. Feature adaptive edge enhancement is performed on the feature channel map where channel ch is an important channel marker important_channel, and feature adaptive stretching is performed on the feature channel map where channel ch is a non-important channel marker.

[0208] In step S34, for the feature adaptive edge enhancement process, the input data are the important feature channel map y_hat_ch, the important probability distribution channel map sigma_ch, multiple edge enhancement segmentation strength values ​​(magl-1, magl-2, ..., magl-n), and multiple edge enhancement segmentation thresholds (thrl-1, thrl-2, ..., thrl-n). Based on the above input data, the important feature channel map y_hat_ch can be feature-adaptively enhanced to obtain a reconstructed feature map after feature adaptive edge enhancement. The reconstructed feature map is called the first reconstructed feature y_hat_sharp. The feature adaptive edge enhancement process is described in Example 10 and will not be repeated here.

[0209] In step S35, for the feature adaptive stretching process, the input data is the non-important feature channel map (the remaining (C L -1) non-important feature channel maps), non-important probability distribution channel maps (the remaining (C L -1) non-important probability distribution channel maps), stretching parameter value ρ, based on the above input data, feature adaptive stretching can be performed on each non-important feature channel map to obtain a reconstructed feature map after feature adaptive stretching, and the reconstructed feature map is called the second reconstructed feature y_hat_scale. The feature adaptive stretching process is shown in Example 11 and will not be repeated here. Among them, the size of the non-important feature channel map subjected to feature adaptive stretching is (C L -1)×H L ×W L , the size of the corresponding probability distribution parameter sigma is also (C L -1)×H L ×W L Each element of this three-dimensional tensor can be subjected to the following adaptive stretching algorithm to obtain the stretched second reconstructed feature: y_hat_scale = y_hat + ρ*clip3(sigma*y_hat,-0.5,0.5).

[0210] Among them, clip3 is the limit operation.

[0211] In step S36, the second reconstructed feature y_hat_scale corresponds to the non-important channel enhancement of the initial reconstructed feature y_hat, and the first reconstructed feature y_hat_sharp corresponds to the important channel enhancement of the initial reconstructed feature y_hat. After the first reconstructed feature y_hat_sharp and the second reconstructed feature y_hat_scale are merged, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is obtained.

[0212] In step S37, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is input to the synthesis transformation network to obtain the initial reconstructed image block x_hat. The initial reconstructed image block x_hat is a two-dimensional tensor with a size of H×W. The important probability distribution channel map sigma_ch of the probability distribution parameter sigma is upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_ch_upscale.

[0213] In step S38, for the image domain enhancement process, the input data of the image domain enhancement module are the initial reconstructed image block x_hat, the target probability distribution channel map sigma_ch_upscale, multiple image enhancement segmentation intensity values ​​(magy-1, magy-2, ..., magy-m), and multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). Based on the above input data, the image domain enhancement module can perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. The image domain enhancement process is described in Example 12 and will not be repeated here.

[0214] Example 15: This example proposes an encoding method, which may include the following steps S41 to S45.

[0215] In step S41, based on C L feature channel maps y_hat and C L A probability distribution channel map sigma is used to determine the important channel identifier corresponding to the important feature channel map (the important channel identifier is also called the important channel number important_channel).

[0216] For example, slice y_hat and sigma along the channel dimension to get the current y_hat_ch and sigma_ch, and calculate the bits_per_ch of each feature channel map through these two tensors, as shown in the following expression. LThe bits_per_ch of each feature channel map is obtained, and the channel number corresponding to the maximum bits_per_ch is selected as the important channel identifier.

[0217] In step S42, a stretch parameter value ρ corresponding to the feature adaptive stretching process is determined. For example, N1 candidate stretch parameter values ​​ρ are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate stretch parameter value ρ are obtained. The distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index. The candidate stretch parameter value ρ corresponding to the minimum distortion index value is selected as the optimal stretch parameter value ρ corresponding to the feature adaptive stretching process.

[0218] In step S43, feature domain enhancement parameters corresponding to the feature adaptive edge enhancement process are determined. The feature domain enhancement parameters include multiple edge enhancement segmentation strength values ​​(mag1-1, magl-2, ..., magl-n) and multiple edge enhancement segmentation thresholds (thrl-1, thrl-2, ..., thrl-n). For example, N2 candidate feature domain enhancement parameters are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate feature domain enhancement parameter are obtained. The distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate feature domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal feature domain enhancement parameter corresponding to the feature adaptive edge enhancement process.

[0219] In step S44, image domain enhancement parameters corresponding to the image domain enhancement process are determined. The image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​(magy-1, magy-2, ..., magy-m) and multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). For example, N3 candidate image domain enhancement parameters are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate image domain enhancement parameter are obtained. The distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate image domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal image domain enhancement parameter corresponding to the image domain enhancement process.

[0220] In step S45, the important channel identifiers corresponding to the important feature channel map, the optimal stretch parameter value ρ, the optimal feature domain enhancement parameters, and the optimal image domain enhancement parameters are encoded into the header information bitstream (Bitstream #3, the third bitstream corresponding to the current image block). It should be noted that the feature domain enhancement module and the image domain enhancement module do not alter Bitstream #1 and Bitstream #2. However, the image reconstruction at both the encoder and decoder passes through the enhancement module, so the reconstructed image remains consistent between the encoder and decoder.

[0221] Example 16: A method for image adaptive edge enhancement, that is, an image domain enhancement module performs image domain enhancement on an initial reconstructed image block x_hat to obtain a target reconstructed image block x_hat_enhanced. The method may be a non-edge enhancement mask edge enhancement algorithm, also known as a USM edge enhancement algorithm (Unsharp Masking edge enhancement), and the process includes steps S51 to S54.

[0222] In step S51, a two-dimensional convolution operation is performed on the original reconstructed image and the Gaussian blurred convolution kernel to obtain a Gaussian blurred image.

[0223] In step S52, the Gaussian blurred image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0224] In step S53, the high-frequency detail image is multiplied by an edge enhancement coefficient (ie, the image enhancement segment intensity value corresponding to the probability distribution value corresponding to the eigenvalue), and added to the original reconstructed image to obtain a final edge-enhanced image.

[0225] In step S54, the edge-enhanced image is limited to the value range of the image.

[0226] Exemplarily, the original reconstructed image is the initial reconstructed image block x_hat. After limiting the edge-enhanced image to the value range of the image, the target reconstructed image block x_hat_enhanced can be obtained. This process can be referred to in Example 12.

[0227] Example 17: A method for feature adaptive edge enhancement, that is, a feature domain enhancement module performs feature adaptive edge enhancement on an important feature channel map based on feature domain enhancement parameters and an important probability distribution channel map to obtain a first reconstructed feature. The process can be a non-edge enhancement mask edge enhancement algorithm, that is, a USM edge enhancement algorithm, and the process includes steps S61 to S68.

[0228] In step S61, the important feature channel map y_hat_ch is subtracted from the mean and divided by the variance to obtain a normalized feature map.

[0229] In step S62, the normalized feature map is multiplied by 0.1 and added with 0.5, and the feature value is limited to between 0 and 1.

[0230] In step S63, a two-dimensional convolution operation is performed on the normalized feature map and the Gaussian blur convolution kernel to obtain a Gaussian blurred image.

[0231] In step S64, the Gaussian blurred image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0232] In step S65, the high-frequency detail image is multiplied by an edge enhancement coefficient (ie, the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue), and is added to the normalized feature map to obtain an edge enhancement feature map.

[0233] In step S66, the feature value of the edge enhancement feature map is limited to between 0 and 1.

[0234] In step S67, the edge-enhanced feature map is subtracted by 0.5 and divided by 0.1 to obtain a normalized feature map after edge enhancement.

[0235] In step S68, the normalized feature map after edge enhancement is multiplied by the variance and added with the mean to obtain the important feature channel edge enhancement feature map y_hat_sharp corresponding to the important feature channel map y_hat_ch, that is, the first reconstructed feature.

[0236] Exemplarily, the original reconstructed image is the important feature channel map y_hat_ch. The process can be referred to Example 10.

[0237] For example, the above embodiments can be implemented individually or in combination. For example, each embodiment in Embodiment 1 to Embodiment 17 can be implemented individually, and at least two embodiments in Embodiment 1 to Embodiment 17 can be implemented in combination.

[0238] For example, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way as the encoding end, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way as the decoding end.

[0239] Based on the same application concept as the above method, a decoding device is also proposed in the embodiment of the present application. The device is applied to the decoding end, and the device includes: a memory, which is configured to store video data; a decoder, which is configured to implement the decoding method in the above embodiments 1 to 17, that is, the processing flow of the decoding end.

[0240] For example, in one possible embodiment, the decoder is configured to: decode the first code stream corresponding to the current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode the second code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decode the third code stream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; and enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determine the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0241] Based on the same application concept as the above method, an encoding device is also proposed in an embodiment of the present application. The device is applied to the encoding end, and the device includes: a memory, which is configured to store video data; an encoder, which is configured to implement the encoding method in the above embodiments 1 to 17, that is, the processing flow of the encoding end.

[0242] For example, in one possible embodiment, the encoder is configured to: encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; for each candidate enhancement parameter, enhance the initial reconstruction features based on the candidate enhancement parameter and the probability distribution parameters to obtain an enhanced reconstruction feature, determine a target reconstructed image block based on the enhanced reconstruction feature, and determine a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encode the enhancement parameter, and obtain a third code stream corresponding to the current image block.

[0243] Based on the same application concept as the above-mentioned method, an embodiment of the present application provides a decoding end device (also referred to as a video decoder). From a hardware perspective, the hardware architecture diagram of the decoding end device can be seen in Figure 7A. The decoding end device includes: a processor 711 and a machine-readable storage medium 712, wherein the machine-readable storage medium 712 stores machine-executable instructions that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the decoding methods of the above-mentioned embodiments 1-17 of the present application.

[0244] For example, in one possible embodiment, the processor 711 executes machine-executable instructions to implement: decoding a first code stream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features, decoding a second code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decoding a third code stream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; and enhancing the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determining a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0245] Based on the same application concept as the above method, an embodiment of the present application provides an encoding end device (also referred to as a video encoder). From a hardware perspective, the hardware architecture diagram of the encoding end device can be seen in Figure 7B. The encoding end device includes: a processor 721 and a machine-readable storage medium 722, wherein the machine-readable storage medium 722 stores machine-executable instructions that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the encoding methods of the above embodiments 1-17 of the present application.

[0246] For example, in one possible embodiment, when the processor 721 executes the machine-executable instructions, it is used to implement: encoding the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features, and encoding the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; for each candidate enhancement parameter, enhancing the initial reconstruction features based on the candidate enhancement parameter and the probability distribution parameters to obtain an enhanced reconstruction feature, determining a target reconstructed image block based on the enhanced reconstruction feature, and determining a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; selecting an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encoding the enhancement parameter, and obtaining a third code stream corresponding to the current image block.

[0247] Based on the same application concept as the above method, an embodiment of the present application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the decoding method or encoding method of the above embodiments 1-17 of the present application.

[0248] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above embodiment of the present application can be implemented, such as the decoding method or encoding method in the above embodiments.

[0249] Based on the same application concept as the above method, the embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the decoding method or encoding method disclosed in the above embodiment of the present application.

[0250] Based on the same application concept as the above method, a decoding device is also proposed in an embodiment of the present application. The decoding device can be applied to a decoding end (also called a video decoder). The decoding device may include: a decoding module, used to decode a first code stream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode the second code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; decode the third code stream corresponding to the current image block to obtain enhancement parameters corresponding to the current image block; an enhancement module, used to enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features; and a determination module, used to determine the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0251] Exemplarily, the enhancement parameters include important channel identifiers, the initial reconstruction features include C feature channel maps, the probability distribution parameters include C probability distribution channel maps, the C probability distribution channel maps correspond one-to-one to the C feature channel maps, and the enhancement module is further used to select the feature channel map corresponding to the important channel identifier from the C feature channel maps as the important feature channel map, and select the remaining feature channel maps as non-important feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map.

[0252] Exemplarily, the initial reconstructed features include C feature channel maps, the probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps correspond one-to-one to the C feature channel maps. The enhancement module is further used to determine, for each feature channel map, the number of consumed bits corresponding to the feature channel map based on the eigenvalues ​​in the feature channel map and the probability distribution values ​​in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, select important feature channel maps from the C feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0253] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps, and the enhancement module enhances the initial reconstruction features based on the enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used for: performing feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature-adaptive edge enhancement; performing feature-adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature-adaptive stretching; and generating an enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature.

[0254] Exemplarily, the feature domain enhancement parameters include multiple edge enhancement segmentation strength values ​​and multiple edge enhancement segmentation thresholds, the multiple edge enhancement segmentation thresholds constitute multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values; the enhancement module performs feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map, and obtains the first reconstructed feature after feature-adaptive edge enhancement, which is specifically used for: if the important probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, determining the edge enhancement segmentation strength value corresponding to the probability distribution value based on the edge enhancement threshold interval corresponding to the probability distribution value; performing feature-adaptive edge enhancement on the important feature channel map based on the edge enhancement segmentation strength value corresponding to each probability distribution value, and obtaining the first reconstructed feature after feature-adaptive edge enhancement.

[0255] Exemplarily, the enhancement module performs feature-adaptive edge enhancement on the important feature channel map based on the edge enhancement segmented strength value corresponding to each probability distribution value, and obtains the first reconstructed feature after feature-adaptive edge enhancement, which is specifically used to: normalize the important feature channel map to obtain a normalized feature map; generate a high-frequency detail image based on the important feature channel map and the normalized feature map; for each eigenvalue in the high-frequency detail image, perform edge enhancement on the eigenvalue based on the edge enhancement segmented strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue; determine an edge enhancement feature map based on the edge enhancement eigenvalue corresponding to each eigenvalue; and denormalize the edge enhancement feature map to obtain the first reconstructed feature.

[0256] Exemplarily, the feature domain enhancement parameter includes a stretching parameter value, and the enhancement module performs feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain the second reconstructed feature after feature adaptive stretching, which is specifically used for: if the non-important feature channel map includes multiple eigenvalues, and the non-important probability distribution channel map includes multiple probability distribution values, then, for each eigenvalue in the non-important feature channel map, based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue, determine the stretched eigenvalue corresponding to the eigenvalue; and determine the second reconstructed feature based on the stretched eigenvalue corresponding to each eigenvalue in the non-important feature channel map.

[0257] Exemplarily, when the determination module determines the target reconstructed image block corresponding to the current image block based on the enhanced reconstruction feature, it is specifically used to: input the enhanced reconstruction feature to a synthetic transformation network to obtain the target reconstructed image block corresponding to the current image block; or input the enhanced reconstruction feature to a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; perform image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters to obtain the target reconstructed image block corresponding to the current image block; wherein the image domain enhancement parameters are obtained by decoding the third code stream corresponding to the current image block.

[0258] Exemplarily, the image domain enhancement parameters include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, the multiple image enhancement segmentation thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values; the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters, and obtains the target reconstructed image block corresponding to the current image block, which is specifically used for: obtaining a target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, determining the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value; performing image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value, and obtaining the target reconstructed image block corresponding to the current image block.

[0259] Exemplarily, when the determination module obtains the target probability distribution channel map based on the probability distribution parameters, it is specifically used to: if the probability distribution parameters include important probability distribution channel maps and unimportant probability distribution channel maps, then upsample the important probability distribution channel map to obtain the target probability distribution channel map; wherein, the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0260] Exemplarily, the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmentation strength value corresponding to each probability distribution value to obtain the target reconstructed image block corresponding to the current image block, and is specifically used to: generate a high-frequency detail image based on the initial reconstructed image block; for each eigenvalue in the high-frequency detail image, perform edge enhancement on the eigenvalue based on the image enhancement segmentation strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determine the target reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image.

[0261] Exemplarily, the enhancement module enhances the initial reconstructed features based on the enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used to: perform feature domain enhancement on the initial reconstructed features corresponding to the luminance component of the current image block based on the feature domain enhancement parameters and the probability distribution parameters, to obtain the enhanced reconstruction features corresponding to the luminance component; the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the probability distribution parameters, and when obtaining the target reconstructed image block corresponding to the current image block, is specifically used to: perform image adaptive edge enhancement on the initial reconstructed image block corresponding to the luminance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters, to obtain the target reconstructed image block corresponding to the luminance component; perform image adaptive edge enhancement on the initial reconstructed image block corresponding to the chrominance component of the current image block based on the image domain enhancement parameters and the probability distribution parameters, to obtain the target reconstructed image block corresponding to the chrominance component.

[0262] Exemplarily, the initial reconstructed features include multiple feature channel maps, the probability distribution parameters include multiple probability distribution channel maps, the multiple probability distribution channel maps correspond one-to-one to the multiple feature channel maps, and the decoding module is further used to: decode the code stream corresponding to the current image block to obtain an important channel identifier; the determination module is further used to: select the feature channel map corresponding to the important channel identifier from the multiple feature channel maps as the important feature channel map, and select the remaining feature channel maps as non-important feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map.

[0263] Exemplarily, the initial reconstructed features include multiple feature channel maps, the probability distribution parameters include multiple probability distribution channel maps, and the multiple probability distribution channel maps correspond one-to-one to the multiple feature channel maps. The determination module is further used to: for each feature channel map, determine the number of consumed bits corresponding to the feature channel map based on the characteristic value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; based on the number of consumed bits corresponding to each feature channel map, select important feature channel maps from the multiple feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0264] Exemplarily, by decoding a first code stream related to the current image block, the coefficient hyperparameter features and the probability distribution parameters corresponding to the current image block are obtained; by decoding a second code stream related to the current image block, the initial reconstruction features corresponding to the current image block are obtained; by decoding a third code stream related to the current image block, the important channel identifier is obtained; wherein, the first code stream, the second code stream, and the third code stream are code streams that encode different information.

[0265] Based on the same application concept as the above method, an encoding device is also proposed in an embodiment of the present application, which is applied to an encoding end (also known as a video encoder). The device may include: an encoding module, used to encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine the probability distribution parameters based on the coefficient hyperparameter features, and encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; an enhancement module, used to enhance the initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters for each candidate enhancement parameter to obtain an enhanced reconstruction feature; a determination module, used to determine the target reconstructed image block based on the enhanced reconstruction feature, and determine the cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block; select the enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter; the encoding module is also used to encode the enhancement parameter to obtain a third code stream corresponding to the current image block.

[0266] Exemplarily, the initial reconstruction features include important feature channel maps and unimportant feature channel maps, and the probability distribution parameters include important probability distribution channel maps corresponding to the important feature channel maps and unimportant probability distribution channel maps corresponding to the unimportant feature channel maps. The enhancement module enhances the initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstruction features, is specifically used for: performing feature-adaptive edge enhancement on the important feature channel map based on the candidate feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstruction feature after feature-adaptive edge enhancement; performing feature-adaptive stretching on the unimportant feature channel map based on the candidate feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstruction feature after feature-adaptive stretching; and generating an enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature.

[0267] Exemplarily, when determining the target reconstructed image block based on the enhanced reconstruction feature, the determination module is specifically configured to: input the enhanced reconstruction feature into a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; for each candidate image domain enhancement parameter, perform image adaptive edge enhancement on the initial reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block;

[0268] The encoding module is further configured to determine, for each candidate image domain enhancement parameter, a cost value corresponding to the candidate image domain enhancement parameter based on the target reconstructed image block; select, based on the cost value corresponding to each candidate image domain enhancement parameter, an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters, encode the image domain enhancement parameter, and obtain a third code stream corresponding to the current image block.

[0269] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The foregoing is merely an embodiment of the present application and is not intended to limit the present application.

[0270] For those skilled in the art, various modifications and variations are possible in this application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of the claims of this application.

Claims

1. A decoding method, It is characterized in that The method comprises: Decoding a bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; Determine probability distribution parameters based on the coefficient hyperparameter characteristics; Decoding the code stream corresponding to the current image block based on the probability distribution parameters to obtain an initial reconstruction feature corresponding to the current image block; and A target reconstructed image block corresponding to the current image block is determined based on the initial reconstruction feature.

2. The decoding method according to claim 1, It is characterized in that The determining, based on the initial reconstruction feature, a target reconstructed image block corresponding to the current image block comprises: The initial reconstruction features are input into a synthetic transformation network to obtain a target reconstructed image block corresponding to the current image block.

3. The decoding method according to claim 1, It is characterized in that The determining, based on the initial reconstruction feature, a target reconstructed image block corresponding to the current image block comprises: Inputting the initial reconstruction feature into a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; Performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block; The image domain enhancement parameter is obtained by decoding the code stream corresponding to the current image block.

4. The decoding method according to claim 3, It is characterized in that The image domain enhancement parameters include multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds, the multiple image enhancement segment thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segment strength values.

5. The decoding method according to claim 4, It is characterized in that The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block includes: Acquire a target probability distribution channel map based on the probability distribution parameters; If the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, an image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on an image enhancement threshold interval corresponding to the probability distribution value; Image adaptive edge enhancement is performed on the initial reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block.

6. The decoding method according to claim 5, It is characterized in that The obtaining of a target probability distribution channel map based on the probability distribution parameters includes: If the probability distribution parameters include an important probability distribution channel map and an unimportant probability distribution channel map, upsampling the important probability distribution channel map to obtain the target probability distribution channel map; The size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

7. The decoding method according to claim 5 or 6, It is characterized in that The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block includes: generating a high-frequency detail image based on the initial reconstructed image block; For each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the image enhancement segmentation strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; The target reconstructed image block is determined based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image.

8. The decoding method according to claim 3, It is characterized in that The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block includes: Based on the image domain enhancement parameters and the probability distribution parameters, image adaptive edge enhancement is performed on the initial reconstructed image block corresponding to the brightness component of the current image block to obtain a target reconstructed image block corresponding to the brightness component.

9. The decoding method according to claim 6, It is characterized in that The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one to the plurality of feature channel maps, and the method further includes: Decoding the code stream corresponding to the current image block to obtain an important channel identifier; According to the important channel identifier, a feature channel map corresponding to the important channel identifier is selected from the multiple feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps; The probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

10. The decoding method according to claim 6, It is characterized in that The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one to the plurality of feature channel maps, and the method further includes: For each feature channel map, determining the number of consumed bits corresponding to the feature channel map based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; Based on the number of consumed bits corresponding to each feature channel map, select an important feature channel map from the multiple feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; The probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

11. The decoding method according to claim 9, It is characterized in that Obtaining coefficient hyper-parameter features and the probability distribution parameters corresponding to the current image block by decoding a first code stream related to the current image block; Obtaining an initial reconstruction feature corresponding to the current image block by decoding a second code stream related to the current image block; Obtaining the important channel identifier by decoding a third code stream related to the current image block; The first code stream, the second code stream and the third code stream are code streams encoding different information.

12. The decoding method according to claim 11, It is characterized in that The third code stream is a header information code stream.

13. A method of encoding, It is characterized in that The method comprises: Encoding the coefficient hyper-parameter features corresponding to the current image block to obtain a first bitstream corresponding to the current image block; Determine probability distribution parameters based on the coefficient hyperparameter characteristics; Encoding the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; The important channel identifiers are encoded to obtain a third code stream corresponding to the current image block.

14. A decoding device, It is characterized in that The device comprises: A decoding module, configured to decode a bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode the bitstream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; and A determination module is used to determine a target reconstructed image block corresponding to the current image block based on the initial reconstruction feature.

15. The decoding device according to claim 14, It is characterized in that When the determination module determines the target reconstructed image block corresponding to the current image block based on the initial reconstruction feature, the determination module is further used to: The initial reconstruction features are input into a synthetic transformation network to obtain a target reconstructed image block corresponding to the current image block.

16. The decoding device according to claim 14, It is characterized in that When the determination module determines the target reconstructed image block corresponding to the current image block based on the initial reconstruction feature, the determination module is further used to: Inputting the initial reconstruction feature into a synthetic transformation network to obtain an initial reconstructed image block corresponding to the current image block; The initial reconstruction is performed based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter Performing image adaptive edge enhancement on the image block to obtain a target reconstructed image block corresponding to the current image block; The image domain enhancement parameter is obtained by decoding the code stream corresponding to the current image block.

17. The decoding device according to claim 16, It is characterized in that The image domain enhancement parameters include multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds, the multiple image enhancement segment thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segment strength values.

18. The decoding device according to claim 17, It is characterized in that When the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block, the determination module is further used to: Acquire a target probability distribution channel map based on the probability distribution parameters; If the target probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, an image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on an image enhancement threshold interval corresponding to the probability distribution value; Image adaptive edge enhancement is performed on the initial reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block.

19. The decoding device according to claim 18, It is characterized in that When the determination module obtains the target probability distribution channel map based on the probability distribution parameters, it is further used to: If the probability distribution parameters include an important probability distribution channel map and an unimportant probability distribution channel map, upsampling the important probability distribution channel map to obtain the target probability distribution channel map; The size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

20. The decoding device according to claim 18 or 19, It is characterized in that The determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segment strength value corresponding to each probability distribution value to obtain a target reconstructed image block corresponding to the current image block, further used to: generating a high-frequency detail image based on the initial reconstructed image block; For each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the image enhancement segmentation strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; The target reconstructed image block is determined based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image.

21. The decoding device according to claim 16, It is characterized in that When the determination module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter corresponding to the current image block and the probability distribution parameter to obtain a target reconstructed image block corresponding to the current image block, the determination module is further used to: Based on the image domain enhancement parameters and the probability distribution parameters, image adaptive edge enhancement is performed on the initial reconstructed image block corresponding to the brightness component of the current image block to obtain a target reconstructed image block corresponding to the brightness component.

22. The decoding device according to claim 19, It is characterized in that The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps correspond one to one to the plurality of feature channel maps. The decoding module is further used for: Decoding the code stream corresponding to the current image block to obtain an important channel identifier; The determination module is further used for: According to the important channel identifier, a feature channel map corresponding to the important channel identifier is selected from the multiple feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps; The probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

23. The decoding device according to claim 19, It is characterized in that The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps correspond one to one to the plurality of feature channel maps. The determination module is further used for: For each feature channel map, determining the number of consumed bits corresponding to the feature channel map based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map; Based on the number of consumed bits corresponding to each feature channel map, select an important feature channel map from the multiple feature channel maps, and select the remaining feature channel maps as unimportant feature channel maps; The probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

24. The decoding device according to claim 22, It is characterized in that Obtaining coefficient hyper-parameter features and the probability distribution parameters corresponding to the current image block by decoding a first code stream related to the current image block; Obtaining an initial reconstruction feature corresponding to the current image block by decoding a second code stream related to the current image block; Obtaining the important channel identifier by decoding a third code stream related to the current image block; The first code stream, the second code stream and the third code stream are code streams encoding different information.

25. The decoding device according to claim 24, It is characterized in that The third code stream is a header information code stream.

26. An encoding device, It is characterized in that The device comprises: The encoding module is used to encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine the probability distribution parameters based on the coefficient hyperparameter features; encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; encode the important channel identifier to obtain a third code stream corresponding to the current image block.

27. A decoding terminal device, It is characterized in that The decoding end device comprises: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement the decoding method described in any one of claims 1-12.

28. A coding terminal device, It is characterized in that The encoding end device includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement the encoding method described in claim 13.

29. A machine-readable storage medium, It is characterized in that The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by the processor, the decoding method described in any one of claims 1 to 12 is implemented, or, when the computer instructions are executed by the processor, the encoding method described in claim 13 is implemented.

30. A computer application program, It is characterized in that When the computer application is executed by a processor, it can implement the decoding method described in any one of claims 1 to 12 or the encoding method described in claim 13.