Decoding and encoding methods, apparatus, and related equipment

Abstract

The present invention provides decoding and encoding methods, apparatus, and related equipment. [Solution] The decoding method includes the steps of: decoding a bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block; determining probability distribution parameters based on the coefficient hyperparameter features; decoding a bitstream corresponding to the current image block based on the probability distribution parameters and obtaining initial reconstruction features corresponding to the current image block; and determining a target reconstructed image block corresponding to the current image block based on the initial reconstruction features. According to the technical solution of the present invention, encoding performance and decoding performance can be improved.

Description

[Technical Field]

[0001] The present invention relates to the technical field of encoding and decoding, and more particularly to decoding and encoding methods, apparatus, and devices thereof. [Background technology]

[0002] To save space, video images are transmitted after being encoded. The complete video encoding process may include prediction, transformation, quantization, entropy coding, filtering, etc. The prediction process may include intra-prediction and inter-prediction. Inter-prediction effectively removes temporal redundancy of video by utilizing the time-domain correlation of video and predicting the pixels of the current frame based on the pixels of adjacent encoded images. Intra-prediction, on the other hand, effectively removes spatial redundancy of video by utilizing the spatial-domain correlation of video and predicting the current pixels based on the pixels of encoded blocks of the current frame image.

[0003] With the rapid development of deep learning, it has achieved success in many high-level computer vision tasks such as image classification and object detection, and its application is beginning in the field of coding and decoding. In other words, it is possible to encode and decode images using neural networks. However, although neural network-based coding and decoding methods show high performance potential, they still suffer from low coding and decoding efficiency and high complexity. [Overview of the Initiative] [Means for solving the problem]

[0004] The present invention provides a decoding and encoding method, apparatus, and related equipment.

[0005] The present invention provides a decoding method applicable to the decoding side, and this method is The steps include decoding the bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block, The steps include determining probability distribution parameters based on the aforementioned coefficient hyperparameter features, The steps include decoding the bitstream corresponding to the current image block based on the probability distribution parameters and obtaining the initial reconstruction features corresponding to the current image block, The process includes the step of determining a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0006] The present invention provides an encoding method applied to the encoding side, and the method is The steps include encoding coefficient hyperparameter features corresponding to the current image block and obtaining a first bitstream corresponding to the current image block, The steps include determining probability distribution parameters based on the aforementioned coefficient hyperparameter features, The steps include: encoding the initial image features corresponding to the current image block based on the probability distribution parameters and obtaining a second bitstream corresponding to the current image block; The process includes the steps of encoding a critical channel identifier and obtaining a third bitstream corresponding to the current image block.

[0007] The present invention provides a decoding device to be applied to the decoding side, and the device is A decoding module configured to decode a bitstream corresponding to the current image block, obtain coefficient hyperparameter features corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, decode the bitstream corresponding to the current image block based on the probability distribution parameters, and obtain initial reconstruction features corresponding to the current image block, A determination module configured to determine a target reconstructed image block corresponding to the current image block based on the initial reconstruction features.

[0008] The present invention provides an encoding device applied to the encoding side, and the device includes An encoding module configured to encode coefficient hyperparameter features corresponding to a current image block, obtain a first bitstream corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, encode initial image features corresponding to the current image block based on the probability distribution parameters, obtain a second bitstream corresponding to the current image block, encode an important channel identifier, and obtain a third bitstream corresponding to the current image block.

[0009] The present invention provides a decoding-side device, and the decoding-side device includes a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor. The processor is configured to implement the above decoding method by executing the machine-executable instructions.

[0010] The present invention provides an encoding-side device, and the encoding-side device includes a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor. The processor is configured to implement the above encoding method by executing the machine-executable instructions.

[0011] The present invention provides an electronic device, and the electronic device includes a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor. The processor is configured to implement the above decoding method or the above encoding method by executing the machine-executable instructions.

[0012] The present invention provides a machine-readable storage medium in which a plurality of computer instructions are stored, and when the computer instructions are executed by a processor, the above-described decoding method is performed or the above-described encoding method is performed.

[0013] The present invention provides a computer program, and when the computer program is executed by a processor, the above-described decoding method or the above-described encoding method is performed. [Brief explanation of the drawing]

[0014] [Figure 1] This is an illustration of the 3D feature matrix in one embodiment of the present invention. [Figure 2] This is a flowchart of the decoding method in one embodiment of the present invention. [Figure 3] This is a flowchart of the encoding method in one embodiment of the present invention. [Figure 4] This is a schematic diagram of the encoding process in one embodiment of the present invention. [Figure 5] This is a schematic diagram of the decoding process in one embodiment of the present invention. [Figure 6A] This is a schematic diagram showing the location of the feature domain enhancement module in one embodiment of the present invention. [Figure 6B] This is a schematic diagram showing the locations of the feature domain enhancement module and the image domain enhancement module in one embodiment of the present invention. [Figure 6C] This is a schematic diagram showing the location of the image domain enhancement module in one embodiment of the present invention. [Figure 7A] This is a hardware structure diagram of the decoding device in one embodiment of the present invention. [Figure 7B] This is a hardware structure diagram of the encoding device in one embodiment of the present invention. [Modes for carrying out the invention]

[0015] The terms used in the embodiments of this invention are for illustrative purposes only and do not limit the invention. The singular terms “one,” “the said,” and “the said” used in the embodiments and claims of this invention are also intended to include the plural form unless the context clearly indicates otherwise. It should be understood that “and / or” in the terms used herein means any or all possible combination including one or more items listed in association. In the embodiments herein, terms such as “first,” “second,” and “third” may be used to describe various types of information, but it should be understood that this information should not be limited to these terms. These terms are used only to distinguish information of the same kind from one another. For example, without departing the scope of the embodiments of this invention, depending on the context, first information may be called second information, and similarly, second information may be called first information. Furthermore, the word “if” used may be interpreted as “when,” “in the case,” or “in response to having decided.”

[0016] Embodiments of the present invention provide decoding and encoding methods relating to the principles of entropy encoding, neural networks (NN), convolutional neural networks (CNN), deconvolution, generalization ability, features, and rate-distortion optimized (RDO).

[0017] Entropy coding is an encoding method that follows the principle of entropy during the encoding process, resulting in no information loss. Information entropy represents the average amount of information (a measure of uncertainty) in the information source. Entropy coding methods include, but are not limited to, Shannon coding, Huffman coding, and arithmetic coding.

[0018] A neural network refers to an artificial neural network, which is a computational model composed of numerous nodes (called neurons) connected to one another. In a neural network, neuron processing units can represent different objects, such as features, alphabets, concepts, or several meaningful abstract modes. There are three types of processing units in a neural network: input units, output units, and hidden units. Input units receive external signals and data, output units produce the output of the processing results, and hidden units are located between the input and output units and cannot be observed from outside the system. Connection weights between neurons reflect the strength of the connections between units, and the representation and processing of information are reflected in the connection relationships of the processing units. A neural network is a non-programmable, brain-like information processing method, and its essence is to acquire parallel and distributed information processing capabilities through the transformation and dynamic behavior of neural networks, mimicking the information processing capabilities of the human brain's nervous system to varying degrees and hierarchies. In the field of video processing, commonly used neural networks include, but are not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected networks.

[0019] Convolutional neural networks are feedforward neural networks and are one of the most representative network structures in deep learning technology. The artificial neurons in a convolutional neural network respond to surrounding units within a certain coverage area and have excellent representational capabilities in large-scale image processing. The basic structure of a convolutional neural network consists of two layers: a feature extraction layer (also called a convolutional layer) and a feature mapping layer (also called an activation layer). In the feature extraction layer, the input of each neuron is connected to the local receptive field of the previous layer, and the local features are extracted. Once the local features are extracted, their positional relationship with other features is also determined accordingly. In the feature mapping layer, each computational layer of the neural network consists of multiple feature mappings, and each feature mapping is a plane, with all neurons on the plane having equal weights. In feature mapping structures, functions such as the Sigmoid function, ReLU (Rectified Linear Unit) function, Leaky-ReLU function, PReLU (Parametric ReLU) function, and GDN (Generalized Divisive Normalization) function can be used as activation functions for the convolutional network. Furthermore, because neurons on a single mapping plane share weights, the number of free parameters in the network is reduced.

[0020] For example, one advantage of convolutional neural networks compared to image processing algorithms is that they can avoid complex pre-processing steps for images (such as extracting artificial features), directly inputting the original image and performing end-to-end learning. Another advantage of convolutional neural networks compared to general neural networks is that while general neural networks employ a fully connected architecture, meaning all neurons from the input layer to the hidden layer are interconnected, this results in a huge number of parameters, making network training time-consuming and ultimately difficult. Convolutional neural networks, however, avoid this difficulty through methods such as local connections and weight sharing.

[0021] A deconvolutional layer, also known as a transposed convolutional layer, operates similarly to a regular convolutional layer. The main difference is that a deconvolutional layer uses padding to make the output size larger than the input size (although it may remain the same size). A stride of 1 indicates that the output size is equal to the input size, while a stride of N indicates that the width of the output features is N times the width of the input features, and the height of the output features is N times the height of the input features.

[0022] Generalization ability refers to a machine learning algorithm's ability to adapt to untrained samples. The goal of learning is to learn the underlying rules of data pairs, and a trained network may also produce appropriate outputs for data other than the training set that shares the same rules; this ability can be called generalization ability.

[0023] The key feature of the present invention is a C × W × H three-dimensional feature matrix or tensor. Figure 1 illustrates the three-dimensional feature matrix, where C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix may be the input to a neural network or the output of a neural network.

[0024] Two key metrics for evaluating encoding efficiency are bitrate and PSNR (Peak Signal to Noise Ratio). A smaller bitstream results in higher compression, while a higher PSNR improves the reconstructed image quality. When selecting a mode, the cost function is essentially a combined evaluation of both. For example, the cost corresponding to a mode: J(mode) is expressed as J = D + λ × R.

[0025] Here, D represents Distortion, which is usually evaluated using the SSE (Sum of Squared Errors) metric. SSE is the sum of the squared differences between the reconstructed image block and the source image. When considering cost, the SAD (Sum of Absolute Differences) metric may also be used, where SAD is the sum of the absolute differences between the reconstructed image block and the source image. λ is the Lagrange multiplier, and R is the actual number of bits required to encode the image block in that mode, including the total number of bits required for encoding such as mode information, motion information, and residuals. When selecting a mode, comparing and evaluating encoding modes using the rate-distortion optimization principle usually guarantees optimal encoding performance.

[0026] Numerous encoding tools have been proposed for each module on the encoding side, and each tool has multiple modes. The encoding tool that yields optimal encoding performance often differs for different video sequences. Therefore, during the encoding process, the encoding performance of different tools and modes is usually compared using the RDO principle, and the optimal one is selected. After determining the optimal tool and mode, the decision information is transmitted by encoding flag information into the bitstream. Although this method results in high encoding complexity, it can adaptively select the optimal mode combination for different content to obtain optimal encoding performance. On the decoding side, the relevant mode information may be obtained by directly analyzing the flag information, which has little impact on the complexity of decoding.

[0027] The general-purpose end-to-end image coding framework primarily consists of a feature-oriented information portion and a hyperplier-oriented information portion. The feature-oriented information portion includes an analysis network, quantization, Gaussian entropy coding, Gaussian entropy decoding, and a synthesis network, while the hyperplier-oriented information portion includes a super pre-analysis network, quantization, factorized entropy coding, factorized entropy decoding, and a super pre-synthesis network. Image components undergo compression coding and reconstruction in the analysis and synthesis networks of the feature-oriented information portion. The hyperplier-oriented information portion is primarily used to model the probabilities of the feature-oriented information and guide the entropy coding and decoding of the feature-oriented information. However, a problem exists in general-purpose end-to-end image coding frameworks: the hyperplier-oriented information is not fully utilized, resulting in insufficient improvement in image reconstruction quality.

[0028] In view of this, in embodiments of the present invention, the features of an end-to-end image coding framework are utilized, and the decoding side improves the quality of features using probability distribution parameters, thereby improving the quality of the reconstructed image. Rather than directly modifying the main information, the coding side encodes enhancement parameters and incorporates them into the header information bitstream (third bitstream), and the decoding side enhances the features of the main information using the enhancement parameters.

[0029] The decoding method and encoding method in the embodiments of the present invention will be described in detail below based on several specific examples.

[0030] Example 1: An embodiment of the present invention provides a decoding method. Figure 2 is a flowchart of the decoding method. The method may be applied to the decoding side (also called a video decoder) and may include steps 201 to 204.

[0031] In step 201, the first bitstream corresponding to the current image block is decoded, and coefficient hyperparameter features corresponding to the current image block are obtained.

[0032] In step 202, probability distribution parameters are determined based on the coefficient hyperparameter features, a second bitstream corresponding to the current image block is decoded based on these probability distribution parameters, and initial reconstruction features corresponding to the current image block are obtained.

[0033] In step 203, the third bitstream corresponding to the current image block is decoded, and the enhancement parameters corresponding to the current image block are obtained. Here, the third bitstream is also called the header information bitstream corresponding to the current image block.

[0034] In step 204, the initial reconstruction features are enhanced based on the enhancement parameters and probability distribution parameters to obtain enhanced reconstruction features, and the target reconstruction image block corresponding to the current image block is determined based on the enhanced reconstruction features.

[0035] For example, the emphasis parameter includes important channel identifiers. The initial reconstruction features include C feature channel maps. The probability distribution parameter includes C probability distribution channel maps, and the C probability distribution channel maps correspond one-to-one with the C feature channel maps. From the C feature channel maps, the feature channel map corresponding to the important channel identifier is selected as the important feature channel map, the remaining feature channel maps are selected as the unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0036] Exemplary, the initial reconstruction features include C feature channel maps, the probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps correspond one-to-one with the C feature channel maps. Based on this, for each feature channel map, the number of bits consumed by that feature channel map is determined based on the feature values ​​of that feature channel map and the probability distribution values ​​of the probability distribution channel map corresponding to that feature channel map. Based on the number of bits consumed by each feature channel map, important feature channel maps are selected from the C feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps. Subsequently, the probability distribution channel maps corresponding to the important feature channel maps are selected as important probability distribution channel maps, and the probability distribution channel maps corresponding to the unimportant feature channel maps are selected as unimportant probability distribution channel maps.

[0037] Exemplary, the initial reconstructed features may include important feature channel maps and non-important feature channel maps, and the probability distribution parameters may include important probability distribution channel maps corresponding to the important feature channel maps and non-important probability distribution channel maps corresponding to the non-important feature channel maps. Refining enhanced reconstructed features by refining the initial reconstructed features based on the emphasis parameters and probability distribution parameters includes, but is not limited to, performing feature-adaptive edge enhancement on the important feature channel maps based on the feature domain emphasis parameters and important probability distribution channel maps to obtain a first reconstructed feature after feature-adaptive edge enhancement, performing feature-adaptive scaling on the non-important feature channel maps based on the feature domain emphasis parameters and non-important probability distribution channel maps to obtain a second reconstructed feature after feature-adaptive scaling, and generating enhanced reconstructed features based on the first and second reconstructed features.

[0038] Exemplary, a feature domain enhancement parameter includes multiple edge enhancement segmental intensity values ​​and multiple edge enhancement segmental thresholds, where the multiple edge enhancement segmental thresholds constitute multiple edge enhancement threshold intervals, and these intervals correspond one-to-one with the multiple edge enhancement segmental intensity values. Based on this, performing feature-adaptive edge enhancement on a key feature channel map based on the feature domain enhancement parameter and the key probability distribution channel map, and obtaining a first reconstructed feature after feature-adaptive edge enhancement, includes, but is not limited to, determining an edge enhancement segmental intensity value corresponding to each probability distribution value for each probability distribution value based on the edge enhancement threshold interval corresponding to that probability distribution value, and then performing feature-adaptive edge enhancement on the key feature channel map based on the edge enhancement segmental intensity value corresponding to each probability distribution value, and obtaining a first reconstructed feature after feature-adaptive edge enhancement.

[0039] Exemplary steps include, but are not limited to, performing feature-adaptive edge enhancement on a critical feature channel map based on edge-enhanced segmental intensity values ​​corresponding to each probability distribution value, and obtaining a first reconstructed feature after feature-adaptive edge enhancement, normalizing the critical feature channel map, obtaining a normalized feature map, generating a high-frequency detail image based on the critical feature channel map and the normalized feature map, performing edge enhancement on each feature value in the high-frequency detail image based on the edge-enhanced segmental intensity value corresponding to the probability distribution value corresponding to that feature value, obtaining an edge-enhanced feature value, determining an edge-enhanced feature map based on the edge-enhanced feature value corresponding to each feature value, performing inverse normalization on the edge-enhanced feature map, and obtaining a first reconstructed feature.

[0040] Exemplary, the feature domain enhancement parameter may include a scaling parameter value, and performing feature adaptive scaling on a non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map, and obtaining a second reconstructed feature after feature adaptive scaling, includes, but is not limited to, determining a scaled feature value corresponding to each feature value in the non-important feature channel map based on the feature value, the scaling parameter value, and the probability distribution value corresponding to the feature value, and then determining a second reconstructed feature based on the scaled feature value corresponding to each feature value in the non-important feature channel map, if the non-important feature channel map includes multiple feature values ​​and the non-important probability distribution channel map includes multiple probability distribution values, and then determining a second reconstructed feature based on the scaled feature value corresponding to each feature value in the non-important feature channel map.

[0041] Exemplary, determining a target reconstructed image block corresponding to the current image block based on enhanced reconstruction features includes, but is not limited to, inputting enhanced reconstruction features into a composite transformation network to obtain a target reconstructed image block corresponding to the current image block, or inputting enhanced reconstruction features into a composite transformation network to obtain an initial reconstructed image block corresponding to the current image block, performing image-adaptive edge enhancement on the initial reconstructed image block based on image domain enhancement parameters and probability distribution parameters corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block. Here, the image domain enhancement parameters are obtained by decoding a third bitstream corresponding to the current image block. That is, the third bitstream is decoded to obtain the image domain enhancement parameters corresponding to the current image block.

[0042] Exemplary, an image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, the plurality of image enhancement segmental thresholds constitute a plurality of image enhancement threshold intervals, and the plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values. Based on this, performing image adaptive edge enhancement on an initial reconstructed image block based on the image domain enhancement parameter and probability distribution parameter corresponding to the current image block and obtaining a target reconstructed image block corresponding to the current image block includes, but is not limited to, obtaining a target probability distribution channel map based on the probability distribution parameter, determining an image enhancement segmental intensity value corresponding to each probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, and performing image adaptive edge enhancement on an initial reconstructed image block based on the image enhancement segmental intensity value corresponding to each probability distribution value and obtaining a target reconstructed image block corresponding to the current image block.

[0043] Exemplary, obtaining a target probability distribution channel map based on probability distribution parameters includes, but is not limited to, upsampling the important probability distribution channel map to obtain the target probability distribution channel map, if the probability distribution parameters include important and unimportant probability distribution channel maps. Here, the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0044] Exemplary, performing image-adaptive edge enhancement on an initial reconstructed image block based on image enhancement segmental intensity values ​​corresponding to each probability distribution value, and obtaining a target reconstructed image block corresponding to the current image block, includes, but is not limited to, generating a high-frequency detail image based on the initial reconstructed image block, performing edge enhancement on each feature value in the high-frequency detail image based on the image enhancement segmental intensity values ​​corresponding to the probability distribution value of that feature value, and obtaining an image enhancement feature value, and determining a target reconstructed image block based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image.

[0045] Exemplarily, enhancing initial reconstructed features based on enhancement parameters and probability distribution parameters and obtaining enhanced reconstructed features includes, but is not limited to, performing feature domain enhancement on initial reconstructed features corresponding to the luminance components of the current image block based on feature domain enhancement parameters and probability distribution parameters and obtaining enhanced reconstructed features corresponding to the luminance components. Performing image-adaptive edge enhancement on initial reconstructed image blocks based on image domain enhancement parameters and probability distribution parameters corresponding to the current image block and obtaining target reconstructed image blocks corresponding to the current image block includes, but is not limited to, performing image-adaptive edge enhancement on initial reconstructed image blocks corresponding to the luminance components of the current image block based on image domain enhancement parameters and probability distribution parameters and obtaining target reconstructed image blocks corresponding to the luminance components, and performing image-adaptive edge enhancement on initial reconstructed image blocks corresponding to the chromaticity components of the current image block based on image domain enhancement parameters and probability distribution parameters and obtaining target reconstructed image blocks corresponding to the chromaticity components.

[0046] Exemplary, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and the method further includes the steps of: decoding the bitstream corresponding to the current image block to obtain a critical channel identifier; selecting, based on the critical channel identifier, a feature channel map corresponding to the critical channel identifier from the plurality of feature channel maps as a critical feature channel map, and selecting the remaining feature channel maps as non-critical feature channel maps; selecting a probability distribution channel map corresponding to the critical feature channel map as the critical probability distribution channel map, and selecting a probability distribution channel map corresponding to the non-critical feature channel map as the non-critical probability distribution channel map.

[0047] Exemplary, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and the method further includes, for each feature channel map, determining the number of bits consumed by the feature channel map based on the feature values ​​of the feature channel map and the probability distribution values ​​of the probability distribution channel map corresponding to the feature channel map; selecting important feature channel maps from the plurality of feature channel maps based on the number of bits consumed by each feature channel map, and selecting the remaining feature channel maps as unimportant feature channel maps; selecting the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0048] For example, by decoding the first bitstream associated with the current image block, the coefficient hyperparameter features and probability distribution parameters corresponding to the current image block are obtained. Furthermore, by decoding the second bitstream associated with the current image block, the initial reconstruction features corresponding to the current image block are obtained. Finally, by decoding the third bitstream associated with the current image block, the important channel identifier is obtained. Here, the first, second, and third bitstreams are bitstreams on which different information is encoded.

[0049] For illustrative purposes, the execution order described above is merely an example provided for the sake of explanation, and in actual applications, the order in which the steps are executed may be changed, and this order is not limiting. Furthermore, in other embodiments, the steps of the corresponding method may not necessarily be performed in the order shown and described herein, and the number of steps included in the method may be more or less than the number described herein. Also, a single step described herein may be broken down into multiple steps in other embodiments, and multiple steps described herein may be combined into a single step in other embodiments.

[0050] As can be seen from the above technical solution, in the embodiment of the present invention, after obtaining the initial reconstruction features corresponding to the current image block, the initial reconstruction features are enhanced based on enhancement parameters and probability distribution parameters, enhanced reconstruction features are obtained, and the target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction features. This realizes an end-to-end video image compression method based on a neural network. This method aims to improve coding and decoding efficiency by combining enhancement parameters and probability distribution parameters. Furthermore, by combining the design of the network structure with a header information bitstream (e.g., a third bitstream), the neural network can effectively guarantee the quality of the reconstructed image block while maintaining low complexity, achieving improved coding and decoding performance and reducing complexity. Feature quality enhancement is performed using enhancement parameters and probability distribution parameters, and the encoding side does not directly modify the feature information but rather encodes the enhancement parameters (e.g., important channel identifiers) and incorporates them into the header information bitstream. The decoding side improves coding performance and the quality of the reconstructed image by enhancing features with the enhancement parameters.

[0051] Example 2: An embodiment of the present invention provides an encoding method. Figure 3 is a flowchart of the encoding method. The encoding method may be applied to the encoding side (also called a video encoder) and may include steps 301 to 304.

[0052] In step 301, the coefficient hyperparameter features corresponding to the current image block are encoded, and a first bitstream corresponding to the current image block is obtained.

[0053] In step 302, probability distribution parameters are determined based on coefficient hyperparameter features, initial image features corresponding to the current image block are encoded based on these probability distribution parameters, and a second bitstream corresponding to the current image block is obtained.

[0054] In step 303, for each candidate enhancement parameter, the initial reconstruction features are enhanced based on the candidate enhancement parameter and probability distribution parameter, enhanced reconstruction features are obtained, the target reconstructed image block is determined based on the enhanced reconstruction features, and the cost value corresponding to the candidate enhancement parameter is determined based on the target reconstructed image block.

[0055] In step 304, based on the cost value corresponding to each candidate enhancement parameter, an enhancement parameter corresponding to the current image block is selected from all candidate enhancement parameters, the enhancement parameter is encoded, and a third bitstream corresponding to the current image block is obtained. Here, the third bitstream is also called the header information bitstream corresponding to the current image block.

[0056] Exemplary, the initial reconstructed features include a key feature channel map and a non-key feature channel map, and the probability distribution parameters include a key probability distribution channel map corresponding to the key feature channel map and a non-key probability distribution channel map corresponding to the non-key feature channel map. Refining the initial reconstructed features based on the candidate enhancement parameters and probability distribution parameters to obtain enhanced reconstructed features includes, but is not limited to, performing feature-adaptive edge enhancement on the key feature channel map based on the candidate feature domain enhancement parameters and the key probability distribution channel map to obtain a first reconstructed feature after feature-adaptive edge enhancement, performing feature-adaptive scaling on the non-key feature channel map based on the candidate feature domain enhancement parameters and the non-key probability distribution channel map to obtain a second reconstructed feature after feature-adaptive scaling, and generating enhanced reconstructed features based on the first and second reconstructed features.

[0057] Exemplary, determining a target reconstructed image block based on enhanced reconstruction features includes, but is not limited to, inputting the enhanced reconstruction features into a composite transformation network to obtain an initial reconstructed image block corresponding to the current image block, and performing image-adaptive edge enhancement on the initial reconstructed image block based on the candidate image domain enhancement parameter and probability distribution parameter for each candidate image domain enhancement parameter to obtain a target reconstructed image block corresponding to the current image block. Based on this, for each candidate image domain enhancement parameter, a cost value corresponding to that candidate image domain enhancement parameter may be determined based on the target reconstructed image block, and based on the cost value corresponding to each candidate image domain enhancement parameter, an image domain enhancement parameter corresponding to the current image block may be selected from all candidate image domain enhancement parameters, the image domain enhancement parameter may be encoded, and a third bitstream corresponding to the current image block may be obtained.

[0058] For illustrative purposes, the encoding process is similar to the decoding process, and the overlapping parts will be omitted from the explanation. The decoding process may also be applied to the encoding process; that is, the encoding side uses the same processing method as the decoding side.

[0059] For illustrative purposes, the execution order described above is merely an example provided for the sake of explanation, and in actual applications, the order in which the steps are executed may be changed, and this order is not limiting. Furthermore, in other embodiments, the steps of the corresponding method may not necessarily be performed in the order shown and described herein, and the number of steps included in the method may be more or less than the number described herein. Also, a single step described herein may be broken down into multiple steps in other embodiments, and multiple steps described herein may be combined into a single step in other embodiments.

[0060] As can be seen from the above technical solution, in the embodiment of the present invention, after obtaining the initial reconstruction features corresponding to the current image block, the initial reconstruction features are enhanced based on enhancement parameters and probability distribution parameters, enhanced reconstruction features are obtained, and the target reconstructed image block corresponding to the current image block is determined based on the enhanced reconstruction features. This realizes an end-to-end video image compression method based on a neural network. This method aims to improve coding and decoding efficiency by combining enhancement parameters and probability distribution parameters. Furthermore, by combining the design of the network structure with a header information bitstream (e.g., a third bitstream), the neural network can effectively guarantee the quality of the reconstructed image block while maintaining low complexity, achieving improved coding and decoding performance and reducing complexity. Feature quality enhancement is performed using enhancement parameters and probability distribution parameters, and the encoding side does not directly modify the feature information but rather encodes the enhancement parameters (e.g., important channel identifiers) and incorporates them into the header information bitstream. The decoding side improves coding performance and the quality of the reconstructed image by enhancing features with the enhancement parameters.

[0061] Example 3: For the encoding-side processing process related to Examples 1 and 2, please refer to Figure 4. Of course, Figure 4 is merely one example of the encoding-side processing process and is not limited to that process.

[0062] The encoding side may, after obtaining the current image block x (which may be the original image block x, i.e., the input image block), perform an analytical transformation on the current image block x using an analytical transformation network (i.e., a neural network) to obtain the image features y corresponding to the current image block x. Here, performing an analytical transformation on the current image block x using an analytical transformation network means transforming the current image block x into image features y in the latent domain, which makes it easier for all subsequent processes to operate in the latent domain.

[0063] Here, the image may be divided into one image block or into multiple image blocks. If the image is divided into one image block, the current image block x can be considered as the image itself, that is, the encoding process of the image block can be applied directly to the image.

[0064] The encoding side obtains image features y, then performs a coefficient hyperparameter feature transformation on image features y to obtain coefficient hyperparameter feature z. For example, image features y may be input to a hyperparameter coding network (i.e., a neural network), and the hyperparameter coding network may perform a coefficient hyperparameter feature transformation on image features y to obtain coefficient hyperparameter feature z. Here, the hyperparameter coding network may be a pre-trained neural network, and its training process is not limited; it just needs to be able to perform a coefficient hyperparameter feature transformation on image features y. Here, image features y from the latent domain are processed by the hyperparameter coding network to obtain hyperprior latent information z.

[0065] The encoding side may, after obtaining the coefficient hyperparameter feature z, quantize the coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z. That is, the Q operation in Figure 4 represents the quantization process. After obtaining the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain Bitstream#1 (i.e., the first bitstream) corresponding to the current image block. That is, the AE operation in Figure 4 represents an encoding process such as the entropy encoding process. Alternatively, the encoding side may directly encode the coefficient hyperparameter feature z to obtain Bitstream#1 corresponding to the current image block. Here, the hyperparameter quantization feature or coefficient hyperparameter feature z contained in Bitstream#1 is mainly used to obtain the mean and the parameters of the probability distribution model.

[0066] The encoding side may obtain Bitstream#1 corresponding to the current image block and then send Bitstream#1 corresponding to the current image block to the decoding side. For the decoding side's processing process for Bitstream#1 corresponding to the current image block, please refer to the subsequent embodiments.

[0067] The encoding side may obtain Bitstream#1 corresponding to the current image block, then decode Bitstream#1 to obtain the hyperparameter quantization feature. That is, AD in Figure 4 represents the decoding process. Next, the encoding side may perform inverse quantization on the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat may be the same as or different from the coefficient hyperparameter feature z. The IQ operation in Figure 4 represents the inverse quantization process. Alternatively, the encoding side may obtain Bitstream#1 corresponding to the current image block, then decode Bitstream#1 to obtain the coefficient hyperparameter feature z_hat, in which case no inverse quantization process is performed on the coefficient hyperparameter feature z_hat.

[0068] In the encoding process of Bitstream#1, an encoding method based on a fixed probability density model may be employed, and in the decoding process of Bitstream#1, a decoding method based on a fixed probability density model may be employed. These encoding and decoding processes are not limited.

[0069] The encoding side may, after obtaining the coefficient hyperparameter feature z_hat, perform a context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstructed feature y_hat of the previous image block (see subsequent examples for the process of determining the reconstructed feature y_hat), and obtain a predicted value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat may be input to a mean prediction network, and the mean prediction network may determine the predicted value mu based on the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat. This prediction process is not limited. Here, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstructed feature y_hat, and a more accurate predicted value mu is obtained by inputting both together. The predicted value mu is used to obtain the residual r by subtracting it from the original feature, and to obtain the reconstructed feature y_hat by adding it to the decoded residual feature r_hat.

[0070] Note that the mean prediction network is a selectable neural network; in other words, it is not necessary to have a mean prediction network. That is, it is not necessary to determine the predicted value mu using the mean prediction network. The dashed box in Figure 4 indicates that the mean prediction network is selectable.

[0071] The encoding side may, after obtaining the image feature y, determine the residual feature r based on the image feature y and the predicted value mu. For example, the residual feature r is defined as the difference between the image feature y and the predicted value mu. Then, feature processing is performed on the residual feature r to obtain the image feature s. This feature processing process is not limited and any feature processing method may be used. In this case, it is necessary to deploy an mean prediction network, which provides the predicted value mu. Alternatively, the encoding side may, after obtaining the image feature y, perform feature processing on the image feature y to obtain the image feature s. This feature processing process is not limited and any feature processing method may be used. In this case, it is not necessary to deploy an mean prediction network. The dashed box indicates that the residual process is a selectable process.

[0072] The encoding side may, after obtaining image features s, quantize the image features s to obtain image quantization features corresponding to the image features s. That is, the Q operation in Figure 4 represents the quantization process. After obtaining the image quantization features corresponding to the image features s, the encoding side may encode these image quantization features to obtain Bitstream#2 (i.e., the second bitstream) corresponding to the current image block. That is, the AE operation in Figure 4 represents an encoding process such as an entropy encoding process. Alternatively, the encoding side may directly encode the image features s to obtain Bitstream#2 corresponding to the current image block; in this case, no quantization process is performed on the image features s.

[0073] The encoding side may obtain Bitstream#2 corresponding to the current image block and then send Bitstream#2 corresponding to the current image block to the decoding side. For the decoding side's processing process for Bitstream#2 corresponding to the current image block, please refer to the subsequent embodiments.

[0074] The encoding side may obtain Bitstream#2 corresponding to the current image block, then decode Bitstream#2 to obtain the image quantization features; that is, AD in Figure 4 represents the decoding process. Next, the encoding side may dequantize the image quantization features to obtain image features s'. Image features s' may be the same as or different from image features s. The IQ operation in Figure 4 is the dequantization process. Alternatively, the encoding side may obtain Bitstream#2 corresponding to the current image block, then decode Bitstream#2 to obtain image features s'; in this case, no dequantization process is performed on the image quantization features.

[0075] The encoding side may, after obtaining image features s', perform feature reconstruction (i.e., the reverse process of feature processing) on ​​image features s' to obtain residual features r_hat. This feature reconstruction process is not limited and can be any feature reconstruction method, and residual features r_hat may be the same as or different from residual features r. After obtaining residual features r_hat, the encoding side may determine image features y_hat (i.e., reconstructed features) based on residual features r_hat and predicted values ​​mu, and image features y_hat may be the same as or different from image features y. For example, image features y_hat may be the sum of residual features r_hat and predicted values ​​mu. In this case, it is necessary to set up an mean prediction network, which provides the predicted values ​​mu. Alternatively, the encoding side may, after obtaining image features s', perform feature reconstruction (i.e., the reverse process of feature processing) on ​​image features s' to obtain image features y_hat, and image features y_hat may be the same as or different from image features y. In this case, it is not necessary to set up an mean prediction network. The dashed box indicates that the residual process is a selectable process.

[0076] The encoding side may obtain the image feature y_hat, then perform a composite transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat can be input into a composite transformation network, the composite transformation network can be performed on the image feature y_hat, and the reconstructed image block x_hat can be obtained. At this point, the image reconstruction process is complete.

[0077] In one embodiment, when the encoding side encodes image quantization features or image features s to obtain Bitstream#2 corresponding to the current image block, it is necessary to first determine a probability distribution model and then encode the image quantization features or image features s based on that probability distribution model. Furthermore, when decoding Bitstream#2, the encoding side is also required to first determine a probability distribution model and then decode Bitstream#2 based on that probability distribution model.

[0078] To obtain a probability distribution model, as shown in Figure 4, the encoding side may obtain the coefficient hyperparameter feature z_hat, then perform an inverse coefficient hyperparameter feature transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters. For example, the coefficient hyperparameter feature z_hat is input to a probabilistic hyperparameter decoding network, and the probabilistic hyperparameter decoding network performs an inverse coefficient hyperparameter feature transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. After obtaining the probability distribution parameter, a probability distribution model may be generated based on the probability distribution parameter. Here, the probabilistic hyperparameter decoding network may be a pre-trained neural network, and the training process of this probabilistic hyperparameter decoding network is not limited; it is sufficient to perform an inverse coefficient hyperparameter feature transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameters.

[0079] In one embodiment, the encoding process may be performed by a deep learning model or a neural network model. This enables an end-to-end image compression and encoding process. This encoding process is not limited to this embodiment.

[0080] Example 4: For Examples 1 and 2, please refer to Figure 5 for the decoding process. Figure 5 is merely one example of the decoding process, and the process is not limited to this example.

[0081] The decoding side may obtain Bitstream#1 corresponding to the current image block, then decode Bitstream#1 to obtain the hyperparameter quantization feature. That is, AD in Figure 5 represents the decoding process. Next, the decoding side may dequantize the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat may be the same as or different from the coefficient hyperparameter feature z. The IQ operation in Figure 5 represents the dequantization process. Alternatively, the decoding side may obtain Bitstream#1 corresponding to the current image block, then decode Bitstream#1 to obtain the coefficient hyperparameter feature z_hat, in which case no dequantization process is performed on the coefficient hyperparameter feature z_hat.

[0082] The decoding process for Bitstream#1 may, but is not limited to, a decoding method based on a fixed probability density model.

[0083] The image may be divided into one image block or into multiple image blocks. If the image is divided into one image block, the current image block x can be considered as the image itself, that is, the image block decoding process can be applied directly to the image.

[0084] The decoding side may, after obtaining the coefficient hyperparameter feature z_hat, perform a context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstructed feature y_hat of the previous image block (see subsequent examples for the process of determining the reconstructed feature y_hat), and obtain a predicted value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat may be input to a mean prediction network, and the mean prediction network may determine the predicted value mu based on the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat; however, this prediction process is not limited. Here, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstructed feature y_hat, and a more accurate predicted value mu is obtained by inputting both together.

[0085] Note that the mean prediction network is a selectable neural network; in other words, it is not necessary to have a mean prediction network. That is, it is not necessary to determine the predicted value mu using the mean prediction network. The dashed box in Figure 5 indicates that the mean prediction network is selectable.

[0086] The decoding side may obtain Bitstream#2 corresponding to the current image block, then decode Bitstream#2 to obtain the image quantization features; that is, AD in Figure 5 represents the decoding process. Next, the decoding side may dequantize the image quantization features to obtain image features s'. Image features s' may be the same as or different from image features s. The IQ operation in Figure 5 represents the dequantization process. Alternatively, the decoding side may obtain Bitstream#2 corresponding to the current image block, then decode Bitstream#2 to obtain image features s'; in this case, no dequantization process is performed on the image quantization features.

[0087] The decoding side may, after obtaining image features s', perform feature reconstruction (i.e., the reverse process of feature processing) on ​​image features s' to obtain residual features r_hat. Residual features r_hat may be the same as or different from residual features r. After obtaining residual features r_hat, the decoding side may determine image features y_hat (i.e., reconstructed features) based on residual features r_hat and predicted values ​​mu. Image features y_hat may be the same as or different from image features y. For example, image features y_hat may be the sum of residual features r_hat and predicted values ​​mu. In this case, an mean prediction network needs to be set up, and the mean prediction network provides the predicted value mu. Alternatively, the decoding side may, after obtaining image features s', perform feature reconstruction on image features s' to obtain image features y_hat. Image features y_hat may be the same as or different from image features y. In this case, an mean prediction network does not need to be set up. The dashed box indicates that the residual process is a selectable process.

[0088] The decoding side may obtain the image feature y_hat, then perform a composite transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat can be input into a composite transformation network, the composite transformation network can be performed on the image feature y_hat, and the reconstructed image block x_hat can be obtained. At this point, the image reconstruction process is complete.

[0089] In one embodiment, when decoding Bitstream#2, the decoding side must first determine a probability distribution model and then decode Bitstream#2 based on that probability distribution model. To obtain the probability distribution model, as shown in Figure 5, the decoding side may obtain the coefficient hyperparameter feature z_hat, then perform an inverse transformation of the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. For example, the coefficient hyperparameter feature z_hat is input to a probabilistic hyperparameter decoding network, and the probabilistic hyperparameter decoding network performs an inverse transformation of the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter. After obtaining the probability distribution parameter, a probability distribution model may be generated based on the probability distribution parameter. Here, the probabilistic hyperparameter decoding network may be a pre-trained neural network, and the training process of this probabilistic hyperparameter decoding network is not limited; the probability distribution parameter may be obtained by performing an inverse transformation of the coefficient hyperparameter feature z_hat.

[0090] In one embodiment, the decoding process may be performed by a deep learning model or a neural network model. This enables an end-to-end image compression and decoding process. This decoding process is not limited to this embodiment.

[0091] Example 5: Based on Examples 3 and 4, a feature domain enhancement module may be added before the composite transformation network, as shown in Figure 6A. The input feature to the feature domain enhancement module may be the image feature y_hat (hereinafter referred to as the initial reconstruction feature y_hat), and the output feature of the feature domain enhancement module may be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input to the composite transformation network, and the composite transformation network performs a composite transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0092] On the encoding side, after obtaining the current image block x, the current image block x is analyzed and transformed by an analysis-transformation network to obtain the image feature y corresponding to the current image block x. A hyperparameter coding network performs a coefficient hyperparameter feature transformation on the image feature y to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block (which may be the coefficient hyperparameter feature z itself, or the hyperparameter quantization feature of the coefficient hyperparameter feature z) is encoded to obtain the first bitstream corresponding to the current image block.

[0093] The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (for example, the coefficient hyperparameter feature z_hat itself is decoded from the first bitstream, or the hyperparameter quantization feature is decoded from the first bitstream and the hyperparameter quantization feature is inversely quantized to obtain the coefficient hyperparameter feature z_hat). Next, the coefficient hyperparameter feature inverse transform is performed on the coefficient hyperparameter feature z_hat using a probabilistic hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0094] An initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma, and a second bitstream corresponding to the current image block is obtained. Here, the initial image feature may be, but is not limited to, an image feature y, a residual feature r corresponding to image feature y, an image feature s after feature processing on image feature y or residual feature r, or an image quantization feature corresponding to image feature y, residual feature r, or image feature s.

[0095] Based on the probability distribution parameter sigma, a second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, if the initial image feature is image feature y, the initial reconstruction feature y_hat is decoded from the second bitstream. If the initial image feature is an image quantization feature corresponding to image feature y, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the initial reconstruction feature y_hat. Alternatively, for example, if the initial image feature is a residual feature r corresponding to image feature y, the residual feature r_hat is decoded from the second bitstream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is an image feature s corresponding to image feature y or residual feature r, the image feature s' may be decoded from the second bitstream, feature reconstruction may be performed on image feature s' to obtain the initial reconstructed feature y_hat or residual feature r_hat, and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat may be determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is an image quantization feature corresponding to image feature s, the image quantization feature may be decoded from the second bitstream, the image quantization feature may be dequantized to obtain the image feature s', feature reconstruction may be performed on image feature s' to obtain the initial reconstructed feature y_hat or residual feature r_hat, and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat may be determined based on the residual feature r_hat and the predicted value mu.

[0096] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input to the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced is input to the composite transformation network, which performs a composite transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0097] On the decoding side, the first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (for example, the coefficient hyperparameter feature z_hat itself is decoded from the first bitstream, or the hyperparameter quantization feature is decoded from the first bitstream and the hyperparameter quantization feature is inversely quantized to obtain the coefficient hyperparameter feature z_hat). Next, the probabilistic hyperparameter decoding network performs an inverse transformation of the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma.

[0098] Based on the probability distribution parameter sigma, a second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, the initial reconstruction feature y_hat is decoded from the second bitstream. Alternatively, the image quantization features are decoded from the second bitstream, and the image quantization features are dequantized to obtain the initial reconstruction feature y_hat. In another example, the residual feature r_hat is decoded from the second bitstream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization features are decoded from the second bitstream, the image quantization features are dequantized to obtain the residual feature r_hat, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. Alternatively, for example, the image features s' may be decoded from the second bitstream, feature reconstruction may be performed on the image features s' to obtain the initial reconstructed feature y_hat or residual feature r_hat, and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat may be determined based on the residual feature r_hat and the predicted value mu. Or, the image quantization features may be decoded from the second bitstream, the image quantization features may be dequantized to obtain the image features s', feature reconstruction may be performed on the image features s' to obtain the initial reconstructed feature y_hat or residual feature r_hat, and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat may be determined based on the residual feature r_hat and the predicted value mu.

[0099] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input to the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced is input to the composite transformation network, which performs a composite transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0100] Example 6: Based on Examples 3 and 4, a feature domain enhancement module may be added before the composite transformation network and an image domain enhancement module may be added after the composite transformation network, as shown in Figure 6B. The input feature to the feature domain enhancement module may be the initial reconstruction feature y_hat, and the output feature of the feature domain enhancement module may be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input to the composite transformation network, and the composite transformation network performs a composite transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the initial reconstruction image block x_hat corresponding to the current image block x. The input feature to the image domain enhancement module may be the initial reconstruction image block x_hat, and the output feature of the image domain enhancement module may be the target reconstruction image block x_hat_enhanced. That is, the image domain enhancement module may perform image domain enhancement on the initial reconstruction image block x_hat to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0101] On the encoding side, the current image block x is analyzed and transformed by an analysis-transformation network to obtain the image feature y corresponding to the current image block x. A coefficient hyperparameter feature transformation is performed on the image feature y by a hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first bitstream corresponding to the current image block. The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and a probabilistic hyperparameter decoding network performs an inverse coefficient hyperparameter feature transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second bitstream corresponding to the current image block. Based on the probability distribution parameter sigma, the second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For detailed processes, please refer to Example 5.

[0102] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input to the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced is input to the composite transformation network, which performs a composite transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0103] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat and obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0104] On the decoding side, the first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block. The probabilistic hyperparameter decoding network then performs an inverse transformation of the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For details of these processes, please refer to Example 5.

[0105] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input to the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced is input to the composite transformation network, which performs a composite transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x.

[0106] After obtaining the initial reconstructed image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat and obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0107] Example 7: Based on Examples 3 and 4, an image domain enhancement module may be added after the composite transformation network, as shown in Figure 6C. The input feature to the image domain enhancement module is the initial reconstructed image block x_hat, and the output feature of the image domain enhancement module is the target reconstructed image block x_hat_enhanced. That is, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0108] On the encoding side, the current image block x is analyzed and transformed by an analysis-transformation network to obtain the image feature y corresponding to the current image block x. A coefficient hyperparameter feature transformation is performed on the image feature y by a hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first bitstream corresponding to the current image block. The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and a probabilistic hyperparameter decoding network performs an inverse coefficient hyperparameter feature transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second bitstream corresponding to the current image block. Based on the probability distribution parameter sigma, the second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For detailed processes, please refer to Example 5.

[0109] After obtaining the initial reconstruction feature y_hat, the initial reconstruction feature y_hat may be input to a composite transformation network, and the composite transformation network may perform a composite transformation on the initial reconstruction feature y_hat to obtain the initial reconstruction image block x_hat corresponding to the current image block x. After obtaining the initial reconstruction image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstruction image block x_hat to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0110] On the decoding side, the first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block. The probabilistic hyperparameter decoding network then performs an inverse transformation of the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the second bitstream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For details of these processes, please refer to Example 5.

[0111] After obtaining the initial reconstruction feature y_hat, the initial reconstruction feature y_hat may be input to a composite transformation network, and the composite transformation network may perform a composite transformation on the initial reconstruction feature y_hat to obtain the initial reconstruction image block x_hat corresponding to the current image block x. After obtaining the initial reconstruction image block x_hat corresponding to the current image block x, the image domain enhancement module may perform image domain enhancement on the initial reconstruction image block x_hat to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0112] Example 8: This section describes the process in which, in Examples 5 and 6, the initial reconstructed feature y_hat is input to the feature domain enhancement module, feature domain enhancement is performed on the initial reconstructed feature y_hat by the feature domain enhancement module, and the enhanced reconstructed feature y_hat_enhanced is obtained. For example, feature domain enhancement may be performed on the initial reconstructed feature y_hat based on the feature domain enhancement parameter and probability distribution parameter sigma corresponding to the current image block, and the enhanced reconstructed feature y_hat_enhanced may be obtained. Feature domain enhancement may include feature adaptive edge enhancement and feature adaptive scaling. The probability distribution parameter sigma is used to assist in the calculation, determining which channels require feature adaptive edge enhancement and which channels require feature adaptive scaling, and also determining the intensity of the scaling. This process is described below.

[0113] The initial reconstruction feature y_hat is,

number

number

number

number

number

[0114] The probability distribution parameter sigma is similar

number

number

number

number

number

[0115] C L For \(C\) feature channel maps, L the \(C\) feature channel maps can be classified into important feature channel maps and unimportant feature channel maps. There may be at least one important feature channel map, and there may be multiple unimportant feature channel maps. For example, L at least one feature channel map can be selected from the \(C\) feature channel maps as an important feature channel map, and the remaining feature channel maps can be selected as unimportant feature channel maps. L The \(C\) probability distribution channel maps L correspond one-to-one with the \(C\) feature channel maps. Therefore, the probability distribution channel maps corresponding to the important feature channel maps can be selected as important probability distribution channel maps, and the probability distribution channel maps corresponding to the unimportant feature channel maps can be selected as unimportant probability distribution channel maps. For example, if the first feature channel map is an important feature channel map, the first probability distribution channel map corresponding to the first feature channel map can be selected as an important probability distribution channel map.

[0116] As described above, the initial reconstructed feature y_hat may include important feature channel maps and non-important feature channel maps, and the probability distribution parameter sigma may include important probability distribution channel maps corresponding to the important feature channel maps and non-important probability distribution channel maps corresponding to the non-important feature channel maps. For important feature channel maps, feature-adaptive edge enhancement may be performed on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map, and a first reconstructed feature after feature-adaptive edge enhancement may be obtained. For non-important feature channel maps, feature-adaptive scaling may be performed on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map, and a second reconstructed feature after feature-adaptive scaling may be obtained. Based on this, an enhanced reconstructed feature y_hat_enhanced may be generated based on the first reconstructed feature and the second reconstructed feature.

[0117] For example, consider the case where Feature Channel Map 1 is an important feature channel map, Feature Channel Maps 2-4 are unimportant feature channel maps, Probability Distribution Channel Map 1 is an important probability distribution channel map, and Probability Distribution Channel Maps 2-4 are unimportant probability distribution channel maps. Feature adaptive edge enhancement may be performed on Feature Channel Map 1 based on the feature domain enhancement parameters and Probability Distribution Channel Map 1 to obtain the first reconstructed feature 1 (i.e., the reconstructed feature of the first channel) after feature adaptive edge enhancement. Feature adaptive scaling may be performed on Feature Channel Map 2 based on the feature domain enhancement parameters and Probability Distribution Channel Map 2 to obtain the second reconstructed feature 2 (i.e., the reconstructed feature of the second channel) after feature adaptive scaling. Feature adaptive scaling may be performed on Feature Channel Map 3 based on the feature domain enhancement parameters and Probability Distribution Channel Map 3 to obtain the second reconstructed feature 3 (i.e., the reconstructed feature of the third channel) after feature adaptive scaling. Feature adaptive scaling may be performed on the feature channel map 4 based on the feature domain enhancement parameters and probability distribution channel map 4 to obtain the second reconstructed feature 4 after feature adaptive scaling (i.e., the reconstructed feature of the fourth channel). Then, the first reconstructed feature 1, the second reconstructed feature 2, the second reconstructed feature 3, and the second reconstructed feature 4 may be combined according to the channels (for example, concatenated according to the channels), that is, the reconstructed features of the four channels may be combined to obtain the enhanced reconstructed feature y_hat_enhanced.

[0118] Example 9: In Example 8, C L It is necessary to classify individual feature channel maps into important feature channel maps and non-important feature channel maps. For example, C L Individual feature channel maps are classified into important feature channel maps and non-important feature channel maps.

[0119] Method 1: The encoding side encodes the important channel identifier in a third bitstream corresponding to the current image block, and the decoding side decodes the important channel identifier from the third bitstream corresponding to the current image block.

[0120] For example, the encoding side is C L The individual feature channel maps are classified into important feature channel maps and non-important feature channel maps (the classification method is not limited), and the important channel identifier (also called the important channel number, denoted as important_channel) is encoded into a third bitstream corresponding to the current image block. After receiving the third bitstream, the decoding side decodes the important channel identifier from the third bitstream and C L From the individual feature channel maps, select the feature channel map corresponding to the important channel identifier as the important feature channel map, select the remaining feature channel maps as the unimportant feature channel maps, select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0121] Here, the encoding side is C L When classifying individual feature channel maps into important feature channel maps and non-important feature channel maps, C is used based on the feature values ​​of each feature channel map and the probability distribution values ​​of each probability distribution channel map. L You may select important feature channel maps from the individual feature channel maps and select the remaining feature channel maps as non-important feature channel maps.

[0122] Method 2: Both the encoding and decoding sides use C based on the feature values ​​of each feature channel map and the probability distribution values ​​of each probability distribution channel map. L Select the important feature channel map from the individual feature channel maps.

[0123] For example, for each feature channel map, the number of bits consumed by that feature channel map may be determined based on the feature values ​​of that feature channel map and the probability distribution values ​​of the probability distribution channel map corresponding to that feature channel map. For example, the number of bits consumed by that feature channel map may be determined using the following formula, and of course, the following formula is just one example.

[0124]

number

[0125] In the above formula, bits_per_ch represents the number of bits consumed by the feature channel map, y_hat_ch(i,j) represents the feature value of the feature point (i,j) in the feature channel map ch, sigma_ch(i,j) represents the probability distribution value of the feature point (i,j) in the probability distribution channel map ch, and Φ(.) is the standard normal distribution function.

[0126] After performing the above process on each feature channel map, the number of bits consumed by each feature channel map is obtained. Based on the number of bits consumed by each feature channel map, C L Important feature channel maps may be selected from the number of feature channel maps. For example, the feature channel map with the largest number of bits consumed may be designated as the important feature channel map, or the K feature channel maps with the largest number of bits consumed may be designated as the important feature channel map. Here, K may be a positive integer greater than 1.

[0127] C L After selecting important feature channel maps from the individual feature channel maps, the remaining feature channel maps may be selected as unimportant feature channel maps, the probability distribution channel maps corresponding to the important feature channel maps may be selected as important probability distribution channel maps, and the probability distribution channel maps corresponding to the unimportant feature channel maps may be selected as unimportant probability distribution channel maps.

[0128] Method 3: Both the encoding and decoding sides use C based on the bitrate consumed by each feature channel map. L Select important feature channel maps from the individual feature channel maps. For example, based on the bitrate consumption of each feature channel map, sort them in descending order of bitrate consumption. L The feature channel maps may be sorted, and the top K feature channel maps may be selected as important feature channel maps. Alternatively, based on the bitrate consumption of each feature channel map, C may be sorted in ascending order of lowest bitrate consumption. L You may sort the feature channel maps and select the bottom K feature channel maps as important feature channel maps.

[0129] C L After selecting important feature channel maps from the individual feature channel maps, the remaining feature channel maps may be selected as unimportant feature channel maps, the probability distribution channel maps corresponding to the important feature channel maps may be selected as important probability distribution channel maps, and the probability distribution channel maps corresponding to the unimportant feature channel maps may be selected as unimportant probability distribution channel maps.

[0130] Method 4: Both the encoding and decoding sides use a default feature channel map (i.e., a fixed feature channel map) as the important feature channel map. For example, the first feature channel map may be predefined as the important feature channel map, or the sixth feature channel map may be predefined as the important feature channel map, or the tenth feature channel map may be predefined as the important feature channel map. Of course, any feature channel map may be used as the important feature channel map, and is not limited to this.

[0131] Example 10: In Example 8, feature-adaptive edge enhancement is performed on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain the first reconstructed feature. The feature-adaptive edge enhancement process is described below.

[0132] For example, the feature domain enhancement parameter may include multiple edge enhancement intensity values ​​and multiple edge enhancement thresholds. For instance, the encoding side encodes multiple edge enhancement intensity values ​​and multiple edge enhancement thresholds in a third bitstream corresponding to the current image block. The decoding side decodes the multiple edge enhancement intensity values ​​and multiple edge enhancement thresholds from the third bitstream corresponding to the current image block and uses the multiple edge enhancement intensity values ​​and multiple edge enhancement thresholds as the feature domain enhancement parameter.

[0133] The number of edge-enhanced segmental intensity values ​​may be the same as or different from the number of edge-enhanced segmental thresholds. For example, if they are the same, the encoding side may encode the number of edge-enhanced segmental intensity values ​​(or edge-enhanced segmental thresholds) in a third bitstream corresponding to the current image block, and the decoding side may decode the number of edge-enhanced segmental intensity values ​​from the third bitstream corresponding to the current image block. If the number of edge-enhanced segmental intensity values ​​is n, the n edge-enhanced segmental intensity values ​​are denoted as magl-1, magl-2, ..., magl-n, and the n edge-enhanced segmental thresholds are denoted as thrl-1, thrl-2, ..., thrl-n.

[0134] For example, multiple edge-enhancing thresholds constitute multiple edge-enhancing threshold intervals. These multiple edge-enhancing threshold intervals correspond one-to-one with multiple edge-enhancing intensity values. Table 1 shows an example of this correspondence.

[0135] [Table 1]

[0136] For example, the important probability distribution channel map may include multiple probability distribution values. For each probability distribution value, first, an edge enhancement threshold interval corresponding to that probability distribution value is determined, and then, based on that edge enhancement threshold interval, an edge enhancement segmental intensity value corresponding to that probability distribution value is determined. For example, if the probability distribution value is within [thrl-2, thrl-3), the edge enhancement segmental intensity value corresponding to that probability distribution value is magl-2, and if the probability distribution value is within [thrl-3, thrl-4), the edge enhancement segmental intensity value corresponding to that probability distribution value is magl-3, and so on. After obtaining the edge enhancement segmental intensity value corresponding to each probability distribution value, feature-adaptive edge enhancement may be performed on the important feature channel map based on the edge enhancement segmental intensity value corresponding to each probability distribution value, and the first reconstructed feature after feature-adaptive edge enhancement may be obtained.

[0137] For example, feature-adaptive edge enhancement may be performed on the important feature channel map by following steps S11 to S15.

[0138] In step S11, the important feature channel map is normalized, and the normalized feature map is obtained.

[0139] For example, a normalized feature map may be obtained by performing normalization based on the important feature channel map, the mean feature corresponding to the important feature channel map, and the standard deviation feature corresponding to the important feature channel map. For example, a normalized feature map may be obtained by subtracting the mean feature from the important feature channel map and dividing by the standard deviation feature. Alternatively, for example, an intermediate feature may be obtained by subtracting the mean feature from the important feature channel map and dividing by the standard deviation feature, and then the intermediate feature may be transformed to obtain a normalized feature map. For example, the intermediate feature may be multiplied by 0.1 and added by 0.5 (0.1 and 0.5 are just examples), and then the feature value may be restricted to the range from 0 to 1. Of course, the above are just some examples of how to normalize an important feature channel map, and the normalization method is not limited to these.

[0140] In step S12, a high-frequency detail image is generated based on the important feature channel map and the normalized feature map.

[0141] For example, a convolution operation (e.g., a two-dimensional convolution) may be performed on a normalized feature map and a Gaussian blurred convolution kernel (which may be a 3x3 convolution kernel or a convolution kernel of other sizes; this convolution kernel is not limited) to obtain a Gaussian blurred image, and then the high-frequency detail image may be obtained by subtracting the Gaussian blurred image from the important feature channel map. Of course, the above is just one example of a method for generating a high-frequency detail image, and the method is not limited to this, as long as the high-frequency details of the important feature channel map can be obtained.

[0142] In step S13, edge enhancement is performed on each feature value in the high-frequency detail image based on the edge-enhanced segmental intensity value corresponding to the probability distribution value corresponding to that feature value, and edge-enhanced feature values ​​are obtained.

[0143] For example, a critical feature channel map contains multiple feature values, and a critical probability distribution channel map contains multiple probability distribution values, with each of these probability distribution values ​​corresponding one-to-one with a given feature value. Furthermore, since a high-frequency detail image contains multiple feature values ​​(corresponding one-to-one with the multiple feature values ​​in the critical feature channel map), the multiple probability distribution values ​​in the critical probability distribution channel map correspond one-to-one with the multiple feature values ​​in the high-frequency detail image. Based on this, for each feature value in the high-frequency detail image, the probability distribution value corresponding to that feature value may be determined from the critical probability distribution channel map, the edge enhancement threshold interval corresponding to that probability distribution value may be determined, and the edge enhancement segmental intensity value corresponding to that probability distribution value may be determined based on that edge enhancement threshold interval.

[0144] After obtaining the edge-enhanced segmental intensity value corresponding to the probability distribution value, edge enhancement may be performed on the feature value based on the edge-enhanced segmental intensity value to obtain the edge-enhanced feature value. For example, the feature value is multiplied by the edge-enhanced segmental intensity value (e.g., magl-1, magl-2, etc.) to obtain the edge-enhanced feature value. Clearly, if the edge-enhanced threshold interval is "less than thrl-1", the feature value is multiplied by 1, i.e., the feature value does not change (no edge enhancement is performed on the feature value). If the edge-enhanced threshold interval is [thrl-1, thrl-2], the feature value is multiplied by magl-1 and edge enhancement is performed on the feature value. Here, magl-1 is a number greater than 1. Similarly, other edge-enhanced segmental intensity values ​​such as magl-2 are greater than 1, so edge enhancement can be achieved.

[0145] For each feature value in the high-frequency detail image, an edge-enhanced segmental intensity value corresponding to that feature value is obtained using the method described above. Furthermore, edge enhancement is performed on the feature value based on the edge-enhanced segmental intensity value, and an edge-enhanced feature value is obtained.

[0146] In step S14, an edge-enhanced feature map is determined based on the edge-enhanced feature value corresponding to each feature value. For example, the edge-enhanced feature map may be obtained by combining the edge-enhanced feature values ​​corresponding to all feature values.

[0147] In step S15, the edge-enhanced feature map is denormalized to obtain the first reconstructed feature (i.e., the first reconstructed feature map). The first reconstructed feature (map) is the reconstructed feature (map) after applying feature-adaptive edge enhancement to the important feature channel map.

[0148] After obtaining the edge-enhanced feature map, the edge-enhanced feature map may be denormalized to obtain the first reconstructed feature map. Alternatively, after obtaining the edge-enhanced feature map, the edge-enhanced feature map and the normalized feature map may be added together to obtain the modified edge-enhanced feature map, and the modified edge-enhanced feature map may be denormalized to obtain the first reconstructed feature map.

[0149] For example, an edge-enhanced normalized feature map may be determined based on an edge-enhanced feature map, and a first reconstructed feature map may be determined based on the edge-enhanced normalized feature map, the mean feature corresponding to the edge-enhanced normalized feature map, and the standard deviation feature corresponding to the edge-enhanced normalized feature map. The first reconstructed feature may be called y_hat_sharp.

[0150] For example, the edge-enhanced feature map can be transformed to obtain a normalized feature map after edge enhancement. For instance, the feature values ​​of the edge-enhanced feature map can be restricted to a range of 0 to 1, and then 0.5 can be subtracted from the feature values ​​of the edge-enhanced feature map and divided by 0.1 (0.1 and 0.5 are just examples) to obtain a normalized feature map after edge enhancement.

[0151] The edge-enhanced normalized feature map is multiplied by the standard deviation feature, and the mean feature is added to obtain the edge-enhanced feature map corresponding to the important feature channel map. This edge-enhanced feature map is denoted as the first reconstructed feature y_hat_sharp.

[0152] Of course, the above is merely an example of denormalizing an edge-enhanced feature map, and this denormalization method is not limited to this.

[0153] In one embodiment, the encoding side needs to encode multiple edge-enhanced intensity values ​​and multiple edge-enhanced threshold values ​​in a third bitstream corresponding to the current image block. For this process, the encoding side may employ the following method.

[0154] The encoding side may set multiple candidate feature domain enhancement parameters, and each candidate feature domain enhancement parameter may include multiple edge enhancement segmental intensity values ​​and multiple edge enhancement segmental thresholds. A cost value corresponding to each candidate feature domain enhancement parameter may be determined, and based on the cost value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block, i.e., the candidate feature domain enhancement parameter with the minimum cost value, may be selected from all candidate feature domain enhancement parameters. The encoding side may encode the feature domain enhancement parameter and obtain a third bitstream corresponding to the current image block.

[0155] For example, for each candidate feature domain enhancement parameter, feature domain enhancement may be performed on the initial reconstructed feature based on the candidate feature domain enhancement parameter and probability distribution parameter to obtain enhanced reconstructed features. Refer to the above-described embodiment for the feature domain enhancement process. Based on the enhanced reconstructed features, the target reconstructed image block x_hat_enhanced is determined, and based on the target reconstructed image block x_hat_enhanced, the cost value corresponding to the candidate feature domain enhancement parameter is determined. The method for determining the cost value is not limited.

[0156] For example, for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes magl-1, magl-2, ..., magl-n and thrl-1, thrl-2, ..., thrl-n. Based on the candidate feature domain enhancement parameter, enhanced reconstruction features are obtained, and the target reconstructed image block x_hat_enhanced is obtained. Using the distortion index, the distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated. After obtaining the distortion index value corresponding to each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter corresponding to the smallest distortion index value may be selected as the feature domain enhancement parameter corresponding to the current image block, i.e., the optimal feature domain enhancement parameter. In this way, the encoding side encodes the feature domain enhancement parameter in the third bitstream.

[0157] Example 11: In Example 8, it is necessary to perform feature-adaptive scaling on the non-important feature channel map (i.e., the non-important probability distribution channel map corresponding to the non-important feature channel map) based on the feature domain enhancement parameters and the non-important probability distribution channel map (i.e., the non-important probability distribution channel map corresponding to the non-important feature channel map) to obtain the second reconstructed feature after feature-adaptive scaling. The process of feature-adaptive scaling is described below.

[0158] For example, the feature domain enhancement parameter may include a scaling parameter value. For instance, the encoding side may encode the scaling parameter value into a third bitstream corresponding to the current image block, and the decoding side may decode the scaling parameter value from the third bitstream corresponding to the current image block. This scaling parameter value is then used as the feature domain enhancement parameter. The scaling parameter value is denoted as ρ.

[0159] For example, a non-important feature channel map may contain multiple feature values, and a non-important probability distribution channel map corresponding to a non-important feature channel map may contain multiple probability distribution values, with multiple probability distribution values ​​corresponding one-to-one with multiple feature values. Based on this, for each feature value in a non-important feature channel map, a scaled feature value corresponding to that feature value may be determined based on the feature value, the scaling parameter value, and the probability distribution value corresponding to that feature value. For example, the scaled feature value corresponding to a feature value may be determined using the following formula, although of course, the following formula is merely an example and not limited to it.

[0160]

number

[0161] In the above formula, y_hat_scale represents the scaled feature value. y_hat represents the feature value of the non-important feature channel map. ρ represents the scaling parameter value. sigma represents the probability distribution value of the non-important probability distribution channel map, and the probability distribution value sigma corresponds to the feature value y_hat. clip3 is a clipping operation that restricts sigma*y_hat to a value between -0.5 and 0.5 (-0.5 and 0.5 are configurable values ​​and are not limited to these). For example, if sigma*y_hat is less than -0.5, the value is clipped to -0.5; if sigma*y_hat is greater than 0.5, the value is clipped to 0.5; and if sigma*y_hat is between -0.5 and 0.5, the value is left as is.

[0162] After performing the above process on each feature value in the non-essential feature channel map, the scaled feature value corresponding to each feature value may be obtained, and a second reconstructed feature (i.e., a second reconstructed feature map) may be determined based on the scaled feature value corresponding to each feature value. The second reconstructed feature (map) is the reconstructed feature (map) after performing feature adaptive scaling on the non-essential feature channel map. For example, the second reconstructed feature map may be obtained by combining the scaled feature values ​​corresponding to all feature values. The second reconstructed feature is also called y_hat_scale.

[0163] The second reconstruction feature y_hat_scale corresponds to the non-important feature channel map of the initial reconstruction feature y_hat; that is, the non-important feature channel map is scaled to obtain the second reconstruction feature y_hat_scale. The first reconstruction feature y_hat_sharp corresponds to the important feature channel map of the initial reconstruction feature y_hat; that is, the important feature channel map is enhanced to obtain the first reconstruction feature y_hat_sharp. The enhanced reconstruction feature y_hat_enhanced is obtained by combining the second reconstruction feature y_hat_scale and the first reconstruction feature y_hat_sharp.

[0164] In one embodiment, the encoding side employs the following method to encode the scaling parameter value in a third bitstream corresponding to the current image block. The encoding side may set a plurality of candidate scaling parameter values, determine a cost value corresponding to each candidate scaling parameter value, and, based on the cost value corresponding to each candidate scaling parameter value, select from all candidate scaling parameter values ​​the scaling parameter value corresponding to the current image block, i.e., the candidate scaling parameter value with the minimum cost value. The encoding side may encode the scaling parameter value and obtain a third bitstream corresponding to the current image block.

[0165] For example, for each candidate scaling parameter value, feature-adaptive scaling is performed on the non-important feature channel map based on the candidate scaling parameter value and the non-important probability distribution channel map, and a second reconstructed feature is obtained after feature-adaptive scaling. Furthermore, an enhanced reconstructed feature is obtained based on the second reconstructed feature, and the target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed feature. Finally, a cost value corresponding to the candidate scaling parameter value may be determined based on the target reconstructed image block x_hat_enhanced.

[0166] For example, for each candidate scaling parameter value, enhanced reconstruction features are obtained based on that candidate scaling parameter value, and the target reconstructed image block x_hat_enhanced is obtained. Using the distortion index, the distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated. After obtaining the distortion index value corresponding to each candidate scaling parameter value, the encoding side may select the candidate scaling parameter value corresponding to the smallest distortion index value as the scaling parameter value corresponding to the current image block, i.e., the optimal scaling parameter value. The encoding side encodes this scaling parameter value in the third bitstream.

[0167] In another embodiment, the encoding side sets up a plurality of candidate feature domain enhancement parameters, and for each candidate feature domain enhancement parameter, the parameter includes a plurality of edge enhancement segmental intensity values, a plurality of edge enhancement segmental thresholds, and one scaling parameter value. The encoding side determines a cost value corresponding to each candidate feature domain enhancement parameter and, based on the cost value corresponding to each candidate feature domain enhancement parameter, selects from all candidate feature domain enhancement parameters the feature domain enhancement parameter corresponding to the current image block, i.e., the candidate feature domain enhancement parameter with the minimum cost value. The encoding side encodes the feature domain enhancement parameters (i.e., a plurality of edge enhancement segmental intensity values, a plurality of edge enhancement segmental thresholds, and one scaling parameter value) and obtains a third bitstream corresponding to the current image block. For example, for each candidate feature domain enhancement parameter, feature domain enhancement is performed on the initial reconstructed features based on the candidate feature domain enhancement parameter and probability distribution parameters to obtain enhanced reconstructed features, the target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed features, and the cost value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced.

[0168] Example 12: This section describes the process in which, in Examples 6 and 7, the initial reconstructed image block x_hat is input to the image domain enhancement module, image domain enhancement is performed on the initial reconstructed image block x_hat by the image domain enhancement module, and the target reconstructed image block x_hat_enhanced corresponding to the current image block x is obtained. For example, image adaptive edge enhancement may be performed on the initial reconstructed image block x_hat based on the image domain enhancement parameter and probability distribution parameter sigma corresponding to the current image block, and the target reconstructed image block x_hat_enhanced corresponding to the current image block is obtained. The image adaptive edge enhancement process is described below.

[0169] For example, the image domain enhancement parameter may include multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds. For instance, the encoding side encodes multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds in a third bitstream corresponding to the current image block. The decoding side decodes the multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds from the third bitstream corresponding to the current image block, and uses the multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds as the image domain enhancement parameter.

[0170] The number of image enhancement segmental intensity values ​​may be the same as or different from the number of image enhancement segmental thresholds. For example, if they are the same, the encoding side may encode the number of image enhancement segmental intensity values ​​(or image enhancement segmental thresholds) in a third bitstream corresponding to the current image block, and the decoding side may decode the number of image enhancement segmental intensity values ​​from the third bitstream corresponding to the current image block. If the number of image enhancement segmental intensity values ​​is m, the m image enhancement segmental intensity values ​​are denoted as magy-1, magy-2, ..., magy-m, and the m image enhancement segmental thresholds are denoted as thry-1, thry-2, ..., thry-m.

[0171] For example, multiple image enhancement thresholds constitute multiple image enhancement threshold intervals. These multiple image enhancement threshold intervals correspond one-to-one with multiple image enhancement intensity values. Table 2 shows an example of this correspondence.

[0172] [Table 2]

[0173] For example, image-adaptive edge enhancement may be performed on the initial reconstructed image block x_hat by the following steps S21 to S23.

[0174] In step S21, a target probability distribution channel map is obtained based on the probability distribution parameters.

[0175] The initial reconstructed image block x_hat is a two-dimensional tensor of size H × W, where H is the image height of the initial reconstructed image block x_hat and W is the image width of the initial reconstructed image block x_hat. The probability distribution parameter sigma is

number

number

number

number

[0176] To perform image-adaptive edge enhancement on the initial reconstructed image block x_hat using probability distribution parameters, it is necessary to obtain a target probability distribution channel map based on the probability distribution parameters. The target probability distribution channel map is a two-dimensional tensor of size H x W.

[0177] For example, C L You may select one probability distribution channel map from among the number of probability distribution channel maps. Since the probability distribution parameter sigma includes important probability distribution channel maps (at least one) and unimportant probability distribution channel maps (multiple), CL From among the available probability distribution channel maps, one important probability distribution channel map may be selected, or one unimportant probability distribution channel map may be selected. The following explanation will use the case of selecting an important probability distribution channel map as an example.

[0178] Next, the important probability distribution channel map is upsampled to obtain the target probability distribution channel map. The method of this upsampling is not particularly limited, as long as the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block x_hat.

[0179] For example, the nearest neighbor upsampling method may be used to upsample the important probability distribution channel map and obtain the target probability distribution channel map. In the nearest neighbor upsampling method, one pixel may be selected from the original low-resolution image (i.e., the important probability distribution channel map) and used as the center point of the corresponding region in the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to that pixel in the original low-resolution image is used as the value of the corresponding pixel in the target high-resolution image. This process is repeated until values ​​are assigned to all pixels in the target high-resolution image.

[0180] In summary, one approach is to upsample the important probability distribution channel map to the same size as the initial reconstructed image block x_hat, and then use the upsampled probability distribution channel map as the target probability distribution channel map sigma_ch_upscale.

[0181] In step S22, if the target probability distribution channel map includes multiple probability distribution values, for each probability distribution value, the corresponding image enhancement segmental intensity value is determined based on the image enhancement threshold interval corresponding to that probability distribution value.

[0182] For example, a target probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the corresponding image enhancement threshold interval may be determined first, and then the corresponding image enhancement segmental intensity value may be determined based on that image enhancement threshold interval. For example, if the probability distribution value is within [thry-2, thry-3), the corresponding image enhancement segmental intensity value will be magy-2, and if the probability distribution value is within [thry-3, thry-4), the corresponding image enhancement segmental intensity value will be magy-3, and so on.

[0183] In step S23, image-adaptive edge enhancement is performed on the initial reconstructed image block x_hat based on the image enhancement segmental intensity value corresponding to each probability distribution value, and the target reconstructed image block x_hat_enhanced corresponding to the current image block is obtained.

[0184] First, a high-frequency detail image may be generated based on the initial reconstructed image block x_hat. For example, a convolution operation (e.g., a two-dimensional convolution operation) may be performed on the initial reconstructed image block x_hat and a Gaussian blur convolution kernel (which may be a 3x3 convolution kernel or a convolution kernel of other sizes; this convolution kernel is not limited) to obtain a Gaussian blur image, and then the high-frequency detail image may be obtained by subtracting the Gaussian blur image from the initial reconstructed image block x_hat. Of course, the above is just one example of a method for generating a high-frequency detail image, and it is sufficient to obtain the high-frequency details of the initial reconstructed image block x_hat, and the method is not limited to this.

[0185] Next, for each feature value in the high-frequency detail image, edge enhancement is performed on that feature value based on the image enhancement segmental intensity value corresponding to the probability distribution value corresponding to that feature value, and an image enhancement feature value is obtained. For example, the initial reconstructed image block x_hat contains multiple feature values, the target probability distribution channel map contains multiple probability distribution values, and multiple probability distribution values ​​correspond one-to-one with multiple feature values. Also, since the high-frequency detail image contains multiple feature values ​​(corresponding one-to-one with multiple feature values ​​in the initial reconstructed image block x_hat), multiple probability distribution values ​​in the target probability distribution channel map correspond one-to-one with multiple feature values ​​in the high-frequency detail image. Based on this, for each feature value in the high-frequency detail image, the probability distribution value corresponding to that feature value may be determined from the target probability distribution channel map, the image enhancement threshold interval corresponding to that probability distribution value may be determined, and the image enhancement segmental intensity value corresponding to that probability distribution value may be determined based on the image enhancement threshold interval.

[0186] After obtaining the image enhancement segmental intensity value corresponding to the probability distribution value, edge enhancement may be performed on the feature value based on the image enhancement segmental intensity value to obtain the image enhancement feature value. For example, the image enhancement feature value is obtained by multiplying the feature value by the image enhancement segmental intensity value (e.g., magy-1, magy-2, etc.). Clearly, if the image enhancement threshold interval is "less than thry-1", the feature value is multiplied by 1, i.e., the feature value does not change (edge ​​enhancement is not performed on the feature value). If the image enhancement threshold interval is [thry-1, thry-2], the feature value is multiplied by magy-1 and edge enhancement is performed on the feature value. Here, magy-1 is a number greater than 1. Similarly, other image enhancement segmental intensity values ​​such as magy-2 are greater than 1, so edge enhancement can be achieved.

[0187] Subsequently, the target reconstructed image block x_hat_enhanced is determined based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image. For example, for each feature value in the high-frequency detail image, the image enhancement feature value corresponding to that feature value may be obtained using the method described above, and an image enhancement feature map may be determined based on the image enhancement feature value corresponding to each feature value. For example, an image enhancement feature map may be obtained by combining the image enhancement feature values ​​corresponding to all feature values. Next, the image enhancement feature map and the initial reconstructed image block x_hat are added together to obtain the final edge-enhanced image, and then the target reconstructed image block x_hat_enhanced is obtained by clipping the edge-enhanced image within the image's range.

[0188] In one embodiment, the encoding side needs to encode multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds in a third bitstream corresponding to the current image block. For this process, the encoding side may employ the following method.

[0189] The encoding side may set multiple candidate image domain enhancement parameters, and each candidate image domain enhancement parameter may include multiple image enhancement segmental intensity values ​​and multiple image enhancement segmental thresholds. A cost value corresponding to each candidate image domain enhancement parameter may be determined, and based on the cost value corresponding to each candidate image domain enhancement parameter, the encoding side may select from all candidate image domain enhancement parameters the image domain enhancement parameter corresponding to the current image block, i.e., the candidate image domain enhancement parameter with the minimum cost value. The encoding side may encode the image domain enhancement parameter and obtain a third bitstream corresponding to the current image block.

[0190] For example, for each candidate image domain enhancement parameter, image-adaptive edge enhancement is performed on the initial reconstructed image block x_hat based on the candidate image domain enhancement parameter and probability distribution parameter to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block. The image-adaptive edge enhancement process is described in the above example and is omitted here. Based on the target reconstructed image block x_hat_enhanced, a cost value corresponding to the candidate image domain enhancement parameter is determined. The method for determining this cost value is not limited.

[0191] For example, for each candidate image domain enhancement parameter, the candidate image domain enhancement parameter includes magy-1, magy-2, ..., magy-m, thry-1, thry-2, ..., thry-m. Based on the candidate image domain enhancement parameter, the target reconstructed image block x_hat_enhanced is obtained. Using the distortion index, the distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated. After obtaining the distortion index value corresponding to each candidate image domain enhancement parameter, the candidate image domain enhancement parameter corresponding to the smallest distortion index value may be selected as the image domain enhancement parameter corresponding to the current image block, i.e., the optimal image domain enhancement parameter. In this way, the encoding side encodes the image domain enhancement parameter in the third bitstream.

[0192] Example 13: In Examples 5 to 12, if a feature domain enhancement module is added before the composite transformation network, the feature domain enhancement module may perform feature domain enhancement on the luminance component, on the chromaticity component, or on both the luminance and chromaticity components simultaneously. In this example, we will use the example of performing feature domain enhancement on the luminance component. To perform feature domain enhancement on the luminance component, feature domain enhancement may be performed on the initial reconstruction feature y_hat corresponding to the luminance component of the current image block based on the feature domain enhancement parameter and the probability distribution parameter, thereby obtaining an enhanced reconstruction feature y_hat_enhanced corresponding to the luminance component. The feature domain enhancement process can be found in Examples 5 to 12, and will not be explained here.

[0193] In Examples 5 to 12, when an image domain enhancement module is added after the composite transformation network, the image domain enhancement module may perform image domain enhancement on the luminance component, on the chromaticity component, or on both the luminance and chromaticity components simultaneously. In this example, we will use the example of performing image domain enhancement on both the luminance and chromaticity components simultaneously. For example, to perform image domain enhancement on the luminance component, image adaptive edge enhancement may be performed on the initial reconstructed image block x_hat corresponding to the luminance component of the current image block based on the image domain enhancement parameter and the probability distribution parameter to obtain the target reconstructed image block x_hat_enhanced corresponding to the luminance component. The image domain enhancement process can be found in Examples 5 to 12 and will not be described again here. Alternatively, to perform image domain enhancement on the chromaticity component, image adaptive edge enhancement may be performed on the initial reconstructed image block x_hat corresponding to the chromaticity component of the current image block based on the image domain enhancement parameter and the probability distribution parameter to obtain the target reconstructed image block x_hat_enhanced corresponding to the chromaticity component.

[0194] When performing image-adaptive edge enhancement on the initial reconstructed image block x_hat corresponding to the chromaticity component, the target probability distribution channel map of the chromaticity component may be obtained based on the probability distribution parameters of the luminance component. For example, an important probability distribution channel map may be selected from all the probability distribution channel maps of the luminance component's probability distribution parameters, and this important probability distribution channel map may be upsampled to obtain the target probability distribution channel map of the chromaticity component. The size of this target probability distribution channel map is the same as the size of the initial reconstructed image block x_hat corresponding to the chromaticity component. For example, the important probability distribution channel map may be upsampled using the nearest neighbor upsampling method to obtain the target probability distribution channel map of the chromaticity component. In the nearest neighbor upsampling method, one pixel is selected from the original low-resolution image (i.e., the important probability distribution channel map) and set as the center point of the corresponding region in the target high-resolution image (i.e., the target probability distribution channel map). In the original low-resolution image, the value of the pixel closest to that pixel is set as the value of the corresponding pixel in the target high-resolution image. This process is repeated until values ​​are assigned to all pixels in the target high-resolution image.

[0195] After obtaining the target probability distribution channel map for the chromaticity component, for each probability distribution value in the target probability distribution channel map, the corresponding image enhancement segmental intensity value is determined based on the corresponding image enhancement threshold interval. Then, based on the image enhancement segmental intensity value corresponding to each probability distribution value, image adaptive edge enhancement is performed on the initial reconstructed image block x_hat corresponding to the chromaticity component, and the target reconstructed image block x_hat_enhanced corresponding to the chromaticity component is obtained. For a detailed process of image adaptive edge enhancement, please refer to Example 12.

[0196] Example 14: This example provides a decoding method. The decoding method may include the following steps S31 to S38.

[0197] In step S31, the feature domain enhancement parameter and the image domain enhancement parameter are decoded from Bitstream #3 (i.e., the third bitstream, also called the feature enhancement header information bitstream) corresponding to the current image block. Here, the feature domain enhancement parameter may include an important channel identifier (important channel number important_channel), multiple edge enhancement compartmental intensity values ​​(denoted as magl-1, magl-2, ..., magl-n), multiple edge enhancement compartmental thresholds (denoted as thrl-1, thrl-2, ..., thrl-n), and a scaling parameter value ρ. The image domain enhancement parameter may include multiple image enhancement compartmental intensity values ​​(denoted as magy-1, magy-2, ..., magy-m) and multiple image enhancement compartmental thresholds (denoted as thry-1, thry-2, ..., thry-m). The feature domain enhancement parameter may further include a number n of edge enhancement compartmental intensity values, and the image domain enhancement parameter may further include a number m of image enhancement compartmental intensity values.

[0198] In step S32, the input features to the feature domain enhancement module are the initial reconstructed feature y_hat and the probability distribution parameter sigma. The initial reconstructed feature y_hat is,

number

number

number

number

number

number

number

[0199] In step S33, the initial reconstructed feature y_hat has a total of C L There are individual feature channel maps, and the probability distribution parameter sigma has a total of C L There are several probability distribution channel maps. L Individual probability distribution channel maps and C L Each feature channel map has a one-to-one correspondence, and the probability distribution channel map is denoted as sigma_ch. Feature adaptive edge enhancement is performed on feature channel maps where channel ch corresponds to the important channel identifier important_channel, and feature adaptive scaling is performed on feature channel maps where channel ch corresponds to the unimportant channel identifier.

[0200] In step S34, the feature-adaptive edge enhancement process uses the following input data: a key feature channel map y_hat_ch, a key probability distribution channel map sigma_ch, multiple edge enhancement segmental intensity values ​​(magl-1,magl-2,…,magl-n), and multiple edge enhancement segmental thresholds (thrl-1,thrl-2,…,thrl-n). Based on the above input data, feature-adaptive edge enhancement may be performed on the key feature channel map y_hat_ch to obtain a reconstructed feature map after feature-adaptive edge enhancement. This reconstructed feature map is called the first reconstructed feature y_hat_sharp. For a detailed explanation of the feature-adaptive edge enhancement process, please refer to Example 10, and the explanation will be omitted here.

[0201] In step S35, the feature-adaptive scaling process uses non-essential feature channel maps (the remaining (C) other than the important feature channel maps). L (C) non-important feature channel maps, non-important probability distribution channel maps (the remaining (C) non-important probability distribution channel maps L -1) non-important probability distribution channel maps, and a scaling parameter value ρ. Based on the above input data, feature adaptive scaling may be performed on each non-important feature channel map to obtain a reconstructed feature map after feature adaptive scaling. This reconstructed feature map is called the second reconstructed feature y_hat_scale. For a detailed process of feature adaptive scaling, please refer to Example 11, and the explanation will be omitted here. Here, the size of the non-important feature channel map on which feature adaptive scaling is performed is

number

number

[0202]

number

[0203] Here, clip3 is a clipping operation.

[0204] In step S36, the second reconstructed feature y_hat_scale corresponds to the enhancement of non-important channels of the initial reconstructed feature y_hat, and the first reconstructed feature y_hat_sharp corresponds to the enhancement of important channels of the initial reconstructed feature y_hat. After the first reconstructed feature y_hat_sharp and the second reconstructed feature y_hat_scale are combined, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is obtained.

[0205] In step S37, the enhanced reconstructed feature y_hat_enhanced after feature domain enhancement is input to the composite transformation network to obtain the initial reconstructed image block x_hat. The initial reconstructed image block x_hat is a two-dimensional tensor of size H × W. In addition, the important probability distribution channel map sigma_ch of the probability distribution parameter sigma is upsampled to the same size as the initial reconstructed image block x_hat, and the probability distribution channel map after upsampling is set as the target probability distribution channel map sigma_ch_upscale.

[0206] In step S38, the image domain enhancement process includes the following input data to the image domain enhancement module: the initial reconstructed image block x_hat, the target probability distribution channel map sigma_ch_upscale, multiple image enhancement segmental intensity values ​​(magy-1,magy-2,…,magy-m), and multiple image enhancement segmental thresholds (thry-1,thry-2,…,thry-m). Based on the above input data, the image domain enhancement module may perform image domain enhancement on the initial reconstructed image block x_hat and obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. A detailed explanation of the image domain enhancement process can be found in Example 12 and is omitted here.

[0207] Example 15: This embodiment provides an encoding method. The encoding method may include the following steps S41 to S45.

[0208] In step S41, C L Individual feature channel map y_hat and C L Based on the individual probability distribution channel map sigma, the important channel identifier (also called the important channel number important_channel) corresponding to the important feature channel map is determined.

[0209] For example, slice y_hat and sigma along the channel dimension to obtain the current y_hat_ch and sigma_ch. Using these two tensors, calculate bits_per_ch for each feature channel map according to the following formula. Process the above in C L The process is repeated several times to obtain the bits_per_ch value for each feature channel map. The channel number corresponding to the largest bits_per_ch value is selected as the important channel identifier.

[0210]

number

[0211] In step S42, the scaling parameter value ρ corresponding to the feature-adaptive scaling process is determined. For example, N1 candidate scaling parameter values ​​ρ are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate scaling parameter value ρ are obtained. Then, the distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate scaling parameter value ρ corresponding to the smallest distortion index value is selected as the optimal scaling parameter value ρ corresponding to the feature-adaptive scaling process.

[0212] In step S43, feature domain enhancement parameters corresponding to the feature-adaptive edge enhancement process are determined. These feature domain enhancement parameters include multiple edge enhancement segmental intensity values ​​(magl-1, magli-2, ..., magli-n) and multiple edge enhancement segmental thresholds (thrl-1, thrl-2, ..., thrl-n). For example, N2 candidate feature domain enhancement parameters are selected, and the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate feature domain enhancement parameter are obtained. The distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate feature domain enhancement parameter corresponding to the smallest distortion index value is selected as the optimal feature domain enhancement parameter corresponding to the feature-adaptive edge enhancement process.

[0213] In step S44, the image domain enhancement parameter corresponding to the image domain enhancement process is determined. This image domain enhancement parameter includes multiple image enhancement segmental intensity values ​​(magy-1,magy-2,...,magy-m) and multiple image enhancement segmental thresholds (thry-1,thry-2,...,thry-m). For example, N3 candidate image domain enhancement parameters are selected, and the enhancement reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate image domain enhancement parameter are obtained. The distortion index value between the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate image domain enhancement parameter corresponding to the smallest distortion index value is selected as the optimal image domain enhancement parameter corresponding to the image domain enhancement process.

[0214] In step S45, the important channel identifiers corresponding to the important feature channel map, the optimal scaling parameter value ρ, the optimal feature domain enhancement parameters, and the optimal image domain enhancement parameters are encoded into the header information bitstream (the third bitstream Bitstream#3 corresponding to the current image block). Note that the feature domain enhancement module and the image domain enhancement module do not change Bitstream#1 and Bitstream#2, but since the enhancement modules are passed through during image reconstruction on both the encoding and decoding sides, the reconstructed images will match on both sides.

[0215] Example 16: The image adaptive edge enhancement method, i.e., the process in which an image domain enhancement module performs image domain enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced, may be an edge enhancement algorithm using unsharp masking, i.e., a USM edge enhancement algorithm (Unsharp Masking edge enhancement), and this process includes steps S51 to S54.

[0216] In step S51, a two-dimensional convolution operation is performed on the original reconstructed image and the Gaussian blur convolution kernel to obtain a Gaussian blur image.

[0217] In step S52, the Gaussian blurred image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0218] In step S53, the high-frequency detail image is multiplied by an edge enhancement coefficient (i.e., an image enhancement segmental intensity value corresponding to a probability distribution value corresponding to a feature value), and this is added to the original reconstructed image to obtain the final edge-enhanced image.

[0219] In step S54, the edge-enhanced image is clipped to the value range of the image.

[0220] For example, the original reconstructed image is the initial reconstructed image block x_hat, and after clipping the edge-enhanced image within the image's range, the target reconstructed image block x_hat_enhanced is obtained. See Example 12 for this process.

[0221] Example 17: Feature-adaptive edge enhancement method, i.e., a process in which a feature domain enhancement module performs feature-adaptive edge enhancement on an important feature channel map based on feature domain enhancement parameters and an important probability distribution channel map to obtain a first reconstructed feature, may be an edge enhancement algorithm using unsharp masking, i.e., a USM edge enhancement algorithm, and this process includes steps S61 to S68.

[0222] In step S61, the mean is subtracted from the important feature channel map y_hat_ch and divided by the standard deviation to obtain a normalized feature map.

[0223] In step S62, the normalized feature map is multiplied by 0.1 and 0.5 is added to clip the feature values ​​to the range of 0 to 1.

[0224] In step S63, a two-dimensional convolution operation is performed on the normalized feature map and the Gaussian blur convolution kernel to obtain a Gaussian blur image.

[0225] In step S64, the Gaussian blur image is subtracted from the original reconstructed image to obtain a high-frequency detail image.

[0226] In step S65, the high-frequency detail image is multiplied by an edge enhancement coefficient (i.e., the edge enhancement segmented intensity value corresponding to the probability distribution value corresponding to the eigenvalue), and added to the normalized feature map to obtain an edge-enhanced feature map.

[0227] In step S66, the edge-enhanced feature map eigenvalues are clipped to the range from 0 to 1.

[0228] In step S67, 0.5 is subtracted from the edge-enhanced feature map and divided by 0.1 to obtain a normalized feature map after edge enhancement.

[0229] In step S68, the normalized feature map after edge enhancement is multiplied by the standard deviation and added with the mean value to obtain an edge-enhanced feature map of the important feature channel corresponding to the important feature channel map y_hat_ch, that is, the first reconstructed feature.

[0230] Exemplarily, the original reconstructed image is the important feature channel map y_hat_ch. For this process, please refer to Example 10.

[0231] Exemplarily, each of the above embodiments may be realized alone or in combination. For example, each of the embodiments in Embodiments 1 to 17 may be realized alone, or at least two of the embodiments in Embodiments 1 to 17 may be realized in combination.

[0232] For example, in each of the embodiments described above, the contents of the encoded side are applicable to the decoded side, that is, the decoded side may be processed in the same way as the encoded side. The contents of the decoded side are applicable to the encoded side, that is, the encoded side may be processed in the same way as the decoded side.

[0233] Based on the same patent concept as described above, embodiments of the present invention further provide a decoding device applied to the decoding side, the decoding device including a memory configured to store video data and a decoder configured to implement the decoding method in Embodiments 1 to 17, i.e., the decoding side processing flow.

[0234] For example, in one embodiment, the decoding side is configured to decode a first bitstream corresponding to the current image block, obtain coefficient hyperparameter features corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, decode a second bitstream corresponding to the current image block based on the probability distribution parameters, obtain initial reconstruction features corresponding to the current image block, decode a third bitstream corresponding to the current image block, obtain enhancement parameters corresponding to the current image block, enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determine a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0235] Based on a similar patent concept as described above, embodiments of the present invention further provide an encoding device applied to the encoding side, the encoding device comprising a memory configured to store video data and an encoder configured to implement the encoding method in Embodiments 1 to 17, i.e., the encoding side processing flow.

[0236] For example, in one embodiment, the encoding side is configured to encode coefficient hyperparameter features corresponding to the current image block, obtain a first bitstream corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, encode initial image features corresponding to the current image block based on the probability distribution parameters, obtain a second bitstream corresponding to the current image block, enhance the initial reconstruction features for each candidate enhancement parameter based on the candidate enhancement parameter and the probability distribution parameters to obtain an enhanced reconstruction feature, determine the target reconstructed image block based on the enhanced reconstruction feature, determine the cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block, select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encode the enhancement parameter, and obtain a third bitstream corresponding to the current image block.

[0237] Based on a similar application concept as described above, an embodiment of the present invention provides a decoding device (also called a video decoder). From a hardware perspective, please refer to Figure 7A for a structural diagram of the hardware architecture of the decoding device. The decoding device comprises a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 storing machine-executable instructions that can be executed by the processor 711, and the processor 711 is configured to perform the decoding methods of embodiments 1 to 17 of the present invention by executing the machine-executable instructions.

[0238] For example, in one embodiment, the processor 711 executes machine-executable instructions to decode a first bitstream corresponding to the current image block, obtain coefficient hyperparameter features corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, decode a second bitstream corresponding to the current image block based on the probability distribution parameters, obtain initial reconstruction features corresponding to the current image block, decode a third bitstream corresponding to the current image block, obtain enhancement parameters corresponding to the current image block, enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, and determine a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0239] Based on a similar application concept as described above, an embodiment of the present invention provides an encoding device (also called a video encoder), and from a hardware perspective, please refer to Figure 7B for a structural diagram of the hardware architecture of the encoding device. The encoding device comprises a processor 721 and a machine-readable storage medium 722, the machine-readable storage medium 722 storing machine-executable instructions that can be executed by the processor 721, and the processor 721 is configured to perform the encoding method of Embodiments 1 to 17 of the present invention by executing the machine-executable instructions.

[0240] For example, in one embodiment, the processor 721 performs the following actions by executing machine-executable instructions: encode coefficient hyperparameter features corresponding to the current image block, obtain a first bitstream corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, encode initial image features corresponding to the current image block based on the probability distribution parameters, obtain a second bitstream corresponding to the current image block, enhance the initial reconstruction features for each candidate enhancement parameter based on the candidate enhancement parameter and the probability distribution parameters to obtain an enhanced reconstruction feature, determine a target reconstructed image block based on the enhanced reconstruction feature, determine a cost value corresponding to the candidate enhancement parameter based on the target reconstructed image block, select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter, encode the enhancement parameter, and obtain a third bitstream corresponding to the current image block.

[0241] Based on a patent application concept similar to the method described above, an embodiment of the present invention provides an electronic device comprising a processor and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor, wherein the processor is configured to perform the decoding method of Embodiments 1 to 17 of the present invention by executing the machine-executable instructions.

[0242] Based on a similar patent concept as described above, embodiments of the present invention further provide a machine-readable storage medium in which several computer instructions are stored, and when the computer instructions are executed by a processor, the methods disclosed in the above embodiments of the present invention, such as the decoding method or encoding method in each of the above embodiments, are carried out.

[0243] Based on a similar patent application concept as described above, embodiments of the present invention further provide a computer program which, when executed by a processor, performs the decoding or encoding method disclosed in each of the above embodiments.

[0244] Based on a similar patent concept as described above, embodiments of the present invention further provide a decoding device to be applied to the decoding side (also called a video decoder), the decoding device is A decoding module configured to decode a first bitstream corresponding to the current image block, obtain coefficient hyperparameter features corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, decode a second bitstream corresponding to the current image block based on the probability distribution parameters, obtain initial reconstruction features corresponding to the current image block, decode a third bitstream corresponding to the current image block, and obtain enhancement parameters corresponding to the current image block, An enhancement module configured to enhance the initial reconstruction features based on the enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features, The system includes a decision module configured to determine a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features.

[0245] Exemplary, the emphasis parameter includes a key channel identifier, the initial reconstruction feature includes C feature channel maps, the probability distribution parameter includes C probability distribution channel maps, the C probability distribution channel maps correspond one-to-one with the C feature channel maps, and the emphasis module is further configured to select from the C feature channel maps the feature channel map corresponding to the key channel identifier as the key feature channel map, select the remaining feature channel maps as the non-key feature channel map, select the probability distribution channel map corresponding to the key feature channel map as the key probability distribution channel map, and select the probability distribution channel map corresponding to the non-key feature channel map as the non-key probability distribution channel map.

[0246] Exemplarily, the initial reconstruction feature includes C feature channel maps, the probability distribution parameter includes C probability distribution channel maps, the C probability distribution channel maps correspond one-to-one to the C feature channel maps, and the enhancement module further determines, for each feature channel map, the number of consumed bits of the feature channel map based on the feature value of the feature channel map and the probability distribution value of the probability distribution channel map corresponding to the feature channel map, and selects important feature channel maps from the C feature channel maps based on the number of consumed bits of each feature channel map, selects the remaining feature channel maps as non-important feature channel maps, selects the probability distribution channel map corresponding to the important feature channel map as an important probability distribution channel map, and selects the probability distribution channel map corresponding to the non-important feature channel map as a non-important probability distribution channel map.

[0247] Exemplarily, the initial reconstruction feature includes important feature channel maps and non-important feature channel maps, the probability distribution parameter includes important probability distribution channel maps corresponding to the important feature channel maps and non-important probability distribution channel maps corresponding to the non-important feature channel maps, and when the enhancement module obtains an enhanced reconstruction feature by enhancing the initial reconstruction feature based on the enhancement parameter and the probability distribution parameter, specifically, performs feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map, obtains a first reconstruction feature after the feature-adaptive edge enhancement, performs feature-adaptive scaling on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map, obtains a second reconstruction feature after the feature-adaptive scaling, and generates an enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature.

[0248] Exemplary, the feature domain enhancement parameter includes a plurality of edge enhancement segmental intensity values ​​and a plurality of edge enhancement segmental thresholds, the plurality of edge enhancement segmental thresholds constitute a plurality of edge enhancement threshold intervals, the plurality of edge enhancement threshold intervals correspond one-to-one with the plurality of edge enhancement segmental intensity values, and the enhancement module performs feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map, and obtains a first reconstructed feature after feature-adaptive edge enhancement. Specifically, if the important probability distribution channel map includes a plurality of probability distribution values, for each probability distribution value, the edge enhancement segmental intensity value corresponding to that probability distribution value is determined based on the edge enhancement threshold interval corresponding to that probability distribution value, and feature-adaptive edge enhancement is performed on the important feature channel map based on the edge enhancement segmental intensity value corresponding to each probability distribution value, and the first reconstructed feature after feature-adaptive edge enhancement is obtained.

[0249] Exemplary, the enhancement module is configured to perform feature-adaptive edge enhancement on the important feature channel map based on edge-enhancement segmental intensity values ​​corresponding to each probability distribution value, and to obtain a first reconstructed feature after feature-adaptive edge enhancement. Specifically, it is configured to normalize the important feature channel map, obtain a normalized feature map, generate a high-frequency detail image based on the important feature channel map and the normalized feature map, perform edge enhancement on each feature value in the high-frequency detail image based on the edge-enhancement segmental intensity value corresponding to the probability distribution value corresponding to that feature value, obtain an edge-enhancement feature value, determine an edge-enhancement feature map based on the edge-enhancement feature value corresponding to each feature value, perform inverse normalization on the edge-enhancement feature map, and obtain the first reconstructed feature.

[0250] Exemplary, the feature domain enhancement parameter includes a scaling parameter value, and the enhancement module performs feature adaptive scaling on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain a second reconstructed feature after feature adaptive scaling. Specifically, if the non-important feature channel map includes multiple feature values ​​and the non-important probability distribution channel map includes multiple probability distribution values, the module is configured to determine a scaled feature value corresponding to each feature value in the non-important feature channel map based on the feature value, the scaling parameter value, and the probability distribution value corresponding to the feature value, and then determine the second reconstructed feature based on the scaled feature value corresponding to each feature value in the non-important feature channel map.

[0251] Exemplary, when the decision module determines a target reconstructed image block corresponding to the current image block based on the enhanced reconstruction features, it is configured to input the enhanced reconstruction features into a composite transformation network to obtain a target reconstructed image block corresponding to the current image block, or to input the enhanced reconstruction features into a composite transformation network to obtain an initial reconstructed image block corresponding to the current image block, perform image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, and obtain a target reconstructed image block corresponding to the current image block. Here, the image domain enhancement parameters are obtained by decoding a third bitstream corresponding to the current image block.

[0252] Exemplary, the image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, the plurality of image enhancement segmental thresholds constitute a plurality of image enhancement threshold intervals, the plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values, and the decision module is configured to perform image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block, and to obtain a target reconstructed image block corresponding to the current image block, specifically, to obtain a target probability distribution channel map based on the probability distribution parameter, and if the target probability distribution channel map includes a plurality of probability distribution values, for each probability distribution value, to determine an image enhancement segmental intensity value corresponding to that probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, and to perform image adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmental intensity value corresponding to each probability distribution value, and to obtain a target reconstructed image block corresponding to the current image block.

[0253] Exemplary, when the decision module obtains a target probability distribution channel map based on the probability distribution parameters, specifically, if the probability distribution parameters include important probability distribution channel maps and non-important probability distribution channel maps, it is configured to upsample the important probability distribution channel map to obtain the target probability distribution channel map, where the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0254] Exemplary, the decision module is configured to perform image-adaptive edge enhancement on the initial reconstructed image block based on image enhancement segmental intensity values ​​corresponding to each probability distribution value, and to obtain a target reconstructed image block corresponding to the current image block. Specifically, it generates a high-frequency detail image based on the initial reconstructed image block, performs edge enhancement on each feature value in the high-frequency detail image based on the image enhancement segmental intensity values ​​corresponding to the probability distribution value corresponding to that feature value, obtains an image enhancement feature value, and determines the target reconstructed image block based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image.

[0255] Exemplary, when the enhancement module enhances the initial reconstructed features based on the enhancement parameter and the probability distribution parameter and obtains enhanced reconstructed features, it is configured to specifically perform feature domain enhancement on the initial reconstructed features corresponding to the luminance components of the current image block based on the feature domain enhancement parameter and the probability distribution parameter and obtain enhanced reconstructed features corresponding to the luminance components. When the decision module performs image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block and obtains a target reconstructed image block corresponding to the current image block, it is configured to specifically perform image adaptive edge enhancement on the initial reconstructed image block corresponding to the luminance components of the current image block based on the image domain enhancement parameter and the probability distribution parameter and obtain a target reconstructed image block corresponding to the luminance components, and to perform image adaptive edge enhancement on the initial reconstructed image block corresponding to the chromaticity components of the current image block based on the image domain enhancement parameter and the probability distribution parameter and obtain a target reconstructed image block corresponding to the chromaticity components.

[0256] Exemplary, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, the decoding module is further configured to decode the bitstream corresponding to the current image block to obtain important channel identifiers, the decision module is further configured to select, based on the important channel identifiers, the feature channel map corresponding to the important channel identifiers from the plurality of feature channel maps as important feature channel maps, the remaining feature channel maps as non-important feature channel maps, the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and the probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map.

[0257] Exemplary, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and the decision module is configured to further determine the number of bits consumed for each feature channel map based on the feature value of the feature channel map and the probability distribution value of the probability distribution channel map corresponding to the feature channel map, select important feature channel maps from the plurality of feature channel maps based on the number of bits consumed for each feature channel map, select the remaining feature channel maps as unimportant feature channel maps, select the probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and select the probability distribution channel map corresponding to the unimportant feature channel map as the unimportant probability distribution channel map.

[0258] For example, by decoding the first bitstream associated with the current image block, the coefficient hyperparameter features and probability distribution parameters corresponding to the current image block are obtained. Furthermore, by decoding the second bitstream associated with the current image block, the initial reconstruction features corresponding to the current image block are obtained. Finally, by decoding the third bitstream associated with the current image block, the important channel identifier is obtained. Here, the first, second, and third bitstreams are bitstreams on which different information is encoded.

[0259] Based on a similar application concept as described above, embodiments of the present invention further provide an encoding device to be applied to an encoding side (also called a video encoder), the device comprising: an encoding module configured to encode coefficient hyperparameter features corresponding to the current image block, obtain a first bitstream corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, encode initial image features corresponding to the current image block based on the probability distribution parameters, and obtain a second bitstream corresponding to the current image block; an enhancement module configured to enhance initial reconstruction features based on the candidate enhancement parameters and the probability distribution parameters for each candidate enhancement parameter to obtain an enhanced reconstruction feature; and a determination module configured to determine a target reconstruction image block based on the enhanced reconstruction feature, determine a cost value corresponding to the candidate enhancement parameter based on the target reconstruction image block, and select an enhancement parameter corresponding to the current image block from all candidate enhancement parameters based on the cost value corresponding to each candidate enhancement parameter. The encoding module is further configured to encode the enhancement parameters and obtain a third bitstream corresponding to the current image block.

[0260] Exemplary, the initial reconstruction feature includes a key feature channel map and a non-key feature channel map, and the probability distribution parameter includes a key probability distribution channel map corresponding to the key feature channel map and a non-key probability distribution channel map corresponding to the non-key feature channel map. The enhancement module is configured to enhance the initial reconstruction feature based on the candidate enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, specifically by performing feature-adaptive edge enhancement on the key feature channel map based on the candidate feature domain enhancement parameter and the key probability distribution channel map, obtaining a first reconstruction feature after feature-adaptive edge enhancement, performing feature-adaptive scaling on the non-key feature channel map based on the candidate feature domain enhancement parameter and the non-key probability distribution channel map, obtaining a second reconstruction feature after feature-adaptive scaling, and generating an enhanced reconstruction feature based on the first and second reconstruction features.

[0261] Exemplary, when the decision module determines a target reconstructed image block based on the enhanced reconstruction features, it is configured to input the enhanced reconstruction features into a composite transformation network, obtain an initial reconstructed image block corresponding to the current image block, perform image-adaptive edge enhancement on the initial reconstructed image block for each candidate image domain enhancement parameter based on the candidate image domain enhancement parameter and the probability distribution parameter, and obtain a target reconstructed image block corresponding to the current image block.

[0262] The encoding module is further configured to determine a cost value corresponding to each candidate image domain enhancement parameter based on the target reconstructed image block, select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the cost value corresponding to each candidate image domain enhancement parameter, encode that image domain enhancement parameter, and obtain a third bitstream corresponding to the current image block.

[0263] Those skilled in the art will understand that embodiments of the present invention may be provided as methods, systems, or computer program products. The present invention can be provided as complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can be provided in the form of computer program products implemented on one or more computer-compatible storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-compatible program code.

[0264] The above are merely embodiments of the present invention and are not intended to limit it. The present invention can be modified in various ways by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of the claims.

[0265] In the first aspect, embodiments will be further described with reference to the following examples.

[0266] 1. An image decoding method applied to the decoding side, The steps include inputting the initial reconstruction features corresponding to the current image block into a composite transformation network to obtain the initial reconstructed image block corresponding to the current image block, The steps include obtaining image domain enhancement parameters corresponding to the current image block, An image decoding method comprising the steps of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block.

[0267] 2. In the method described in Example 1, Before the step of inputting the initial reconstruction features into the composite transformation network and obtaining the initial reconstruction image block corresponding to the current image block, the method further: The steps include decoding the bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block, The process includes the steps of determining probability distribution parameters based on the coefficient hyperparameter features, decoding the bitstream corresponding to the current image block based on the probability distribution parameters, and obtaining initial reconstruction features corresponding to the current image block.

[0268] 3. In the method described in Example 1, The aforementioned image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, The aforementioned multiple image enhancement thresholds constitute multiple image enhancement threshold intervals, The plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values.

[0269] 4. In the method described in Example 3, The step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block, is: The steps include determining a target probability distribution channel map based on the aforementioned probability distribution parameters, If the target probability distribution channel map includes multiple probability distribution values, the step of determining the corresponding image enhancement segmental intensity value for each probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, The process includes the step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmental intensity value corresponding to each probability distribution value, and obtaining the target reconstructed image block corresponding to the current image block.

[0270] 5. In the method described in Example 4, The step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmental intensity values ​​corresponding to each of the aforementioned probability distribution values, and obtaining a target reconstructed image block corresponding to the current image block, is: The steps include generating a high-frequency detail image based on the initial reconstructed image block, The steps include: determining an image enhancement segmental intensity value for each feature value in the high-frequency detail image based on the probability distribution value corresponding to the feature value; performing edge enhancement on the feature value using the image enhancement segmental intensity value to obtain an image enhancement feature value; The step includes determining the target reconstructed image block based on image enhancement feature values ​​corresponding to each feature value in the high-frequency detail image.

[0271] 6. In the method of Example 2, By decoding the first bitstream corresponding to the current image block, coefficient hyperparameter features corresponding to the current image block are obtained. By decoding the second bitstream corresponding to the current image block, the initial reconstruction features corresponding to the current image block are obtained. Here, the first bitstream and the second bitstream are bitstreams on which different information is encoded.

[0272] 7. An image encoding method applied to the encoding side, The steps include inputting the initial reconstruction features corresponding to the current image block into a composite transformation network to obtain the initial reconstructed image block corresponding to the current image block, The steps include obtaining image domain enhancement parameters corresponding to the current image block, An image encoding method comprising the steps of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block.

[0273] In the second aspect, embodiments will be further described with reference to the following examples.

[0274] 1. An image decoding method applied to the decoding side, The steps include inputting the initial reconstruction features corresponding to the current image block into a composite transformation network to obtain the initial reconstructed image block corresponding to the current image block, The steps include selecting a key probability distribution channel map from the probability distribution channel map of the probability distribution parameters corresponding to the current image block, and generating a target probability distribution channel map based on the key probability distribution channel map, An image decoding method comprising the steps of performing image adaptive edge enhancement on the initial reconstructed image block based on image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining a target reconstructed image block corresponding to the current image block.

[0275] 2. In the method described in Example 1, The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and selecting an important probability distribution channel map from the probability distribution channel map of the probability distribution parameter corresponding to the current image block is: Decode the bitstream corresponding to the current image block to obtain the important channel identifier corresponding to the current image block, Based on the aforementioned important channel identifier, a feature channel map corresponding to the aforementioned important channel identifier is selected from the plurality of feature channel maps as the important feature channel map, This includes selecting a probability distribution channel map corresponding to the aforementioned important feature channel map as the aforementioned important probability distribution channel map.

[0276] 3. In the method described in Example 1, Generating a target probability distribution channel map based on the aforementioned important probability distribution channel map is: This includes upsampling the aforementioned important probability distribution channel map to obtain the aforementioned target probability distribution channel map. Here, the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0277] 4. In the method described in any one of Examples 1 to 3, Prior to the step of inputting the initial reconstruction features corresponding to the current image block into the composite transformation network and obtaining the initial reconstructed image block corresponding to the current image block, the method further: The steps include decoding the bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block, The process includes the steps of determining probability distribution parameters based on the coefficient hyperparameter features, decoding the bitstream corresponding to the current image block based on the probability distribution parameters, and obtaining initial reconstruction features corresponding to the current image block.

[0278] 5. In the method described in Example 4, By decoding the first bitstream corresponding to the current image block, coefficient hyperparameter features corresponding to the current image block are obtained. By decoding the second bitstream corresponding to the current image block, the initial reconstruction features corresponding to the current image block are obtained. By decoding the third bitstream corresponding to the current image block, the important channel identifier corresponding to the current image block is obtained. Here, the first bitstream, the second bitstream, and the third bitstream are bitstreams on which different information is encoded.

[0279] 6. In the method described in Example 1, The aforementioned image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, The aforementioned multiple image enhancement thresholds constitute multiple image enhancement threshold intervals, The plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values, The step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining the target reconstructed image block corresponding to the current image block, is: If the target probability distribution channel map includes multiple probability distribution values, the step of determining the corresponding image enhancement segmental intensity value for each probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, The steps include generating a high-frequency detail image based on the initial reconstructed image block, The steps include: determining an image enhancement segmental intensity value for each feature value in the high-frequency detail image based on the probability distribution value corresponding to the feature value; performing edge enhancement on the feature value using the image enhancement segmental intensity value to obtain an image enhancement feature value; The step includes determining the target reconstructed image block based on image enhancement feature values ​​corresponding to each feature value in the high-frequency detail image.

[0280] 7. An image encoding method applied to the encoding side, The steps include inputting the initial reconstruction features corresponding to the current image block into a composite transformation network to obtain the initial reconstructed image block corresponding to the current image block, The steps include selecting a key probability distribution channel map from the probability distribution channel map of the probability distribution parameters corresponding to the current image block, and generating a target probability distribution channel map based on the key probability distribution channel map, An image encoding method comprising the steps of performing image adaptive edge enhancement on the initial reconstructed image block based on image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining a target reconstructed image block corresponding to the current image block.

[0281] In a third aspect, embodiments will be further described with reference to the following examples.

[0282] 1. An image decoding method applied to the decoding side, A step of obtaining initial reconstruction features corresponding to the current image block, The steps include inputting the initial reconstruction features into the composite transformation network and obtaining the initial reconstruction image block corresponding to the current image block, The steps include selecting a key probability distribution channel map from the probability distribution channel map of the probability distribution parameters corresponding to the current image block, and generating a target probability distribution channel map based on the key probability distribution channel map, The steps include: performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining a target reconstructed image block corresponding to the current image block; Herein, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, in an image decoding method.

[0283] 2. In the method described in Example 1, Selecting a key probability distribution channel map from the probability distribution channel maps of the probability distribution parameters corresponding to the current image block is, For each feature channel map, the number of bits consumed by that feature channel map is determined based on the feature values ​​of that feature channel map and the probability distribution values ​​of the probability distribution channel map corresponding to that feature channel map. This includes selecting important probability distribution channel maps from all probability distribution channel maps based on the number of bits consumed by each feature channel map.

[0284] 3. In the method described in Example 2, Selecting a significant probability distribution channel map from all probability distribution channel maps based on the number of bits consumed by each of the aforementioned feature channel maps is: Based on the number of bits consumed by each feature channel map, an important feature channel map is selected from the multiple feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps. This includes selecting a probability distribution channel map corresponding to the aforementioned important feature channel map as the aforementioned important probability distribution channel map, and selecting a probability distribution channel map corresponding to the aforementioned unimportant feature channel map as the unimportant probability distribution channel map, Here, the feature channel map with the largest number of bits consumed is defined as the important feature channel map.

[0285] 4. In the method described in Example 1, Generating a target probability distribution channel map based on the aforementioned important probability distribution channel map is: This includes upsampling the aforementioned important probability distribution channel map to obtain the aforementioned target probability distribution channel map. Here, the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block.

[0286] 5. In the method described in any one of Examples 1 to 4, The step of obtaining the initial reconstruction features corresponding to the current image block is: The steps include decoding the bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block, The process includes the steps of determining probability distribution parameters based on the coefficient hyperparameter features, decoding the bitstream corresponding to the current image block based on the probability distribution parameters, and obtaining initial reconstruction features corresponding to the current image block.

[0287] 6. In the method described in Example 1, The aforementioned image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, The aforementioned multiple image enhancement thresholds constitute multiple image enhancement threshold intervals, The plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values, The step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining the target reconstructed image block corresponding to the current image block, is: If the target probability distribution channel map includes multiple probability distribution values, the step of determining the corresponding image enhancement segmental intensity value for each probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, The steps include generating a high-frequency detail image based on the initial reconstructed image block, For each feature value in the aforementioned high-frequency detail image, edge enhancement is performed on that feature value based on the image enhancement segmental intensity value corresponding to the probability distribution value corresponding to that feature value, and an image enhancement feature value is obtained. The step includes determining the target reconstructed image block based on image enhancement feature values ​​corresponding to each feature value in the high-frequency detail image.

[0288] 7. An image encoding method applied to the encoding side, A step of obtaining initial reconstruction features corresponding to the current image block, The steps include inputting the initial reconstruction features into the composite transformation network and obtaining the initial reconstruction image block corresponding to the current image block, The steps include selecting a key probability distribution channel map from the probability distribution channel map of the probability distribution parameters corresponding to the current image block, and generating a target probability distribution channel map based on the key probability distribution channel map, The steps include: performing image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters corresponding to the current image block and the target probability distribution channel map, and obtaining a target reconstructed image block corresponding to the current image block; Herein, the initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, and the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, in an image coding method. [Explanation of symbols]

[0289] 711 Processor 712 Machine-readable storage medium 721 Processor 722 Machine-readable storage medium

Claims

1. A decoding method, The steps include decoding the bitstream corresponding to the current image block and obtaining coefficient hyperparameter features corresponding to the current image block, The steps include determining probability distribution parameters based on the aforementioned coefficient hyperparameter features, The steps include decoding the bitstream corresponding to the current image block based on the probability distribution parameters and obtaining the initial reconstruction features corresponding to the current image block, The step of determining a target reconstructed image block corresponding to the current image block based on the initial reconstruction features, The step of determining the target reconstructed image block corresponding to the current image block based on the initial reconstruction features is: The steps include inputting the initial reconstruction features into the composite transformation network and obtaining the initial reconstruction image block corresponding to the current image block, The steps include: performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block; Here, the image domain enhancement parameter is obtained by decoding the bitstream corresponding to the current image block. A decoding method characterized by the following:

2. The step of determining the target reconstructed image block corresponding to the current image block based on the initial reconstruction features is: The steps include inputting the initial reconstruction features into a composite transformation network and obtaining a target reconstruction image block corresponding to the current image block, The decoding method according to feature 1.

3. The aforementioned image domain enhancement parameter includes a plurality of image enhancement segmental intensity values ​​and a plurality of image enhancement segmental thresholds, The aforementioned multiple image enhancement thresholds constitute multiple image enhancement threshold intervals, The plurality of image enhancement threshold intervals correspond one-to-one with the plurality of image enhancement segmental intensity values, The decoding method according to feature 1.

4. The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block, is: A step of obtaining a target probability distribution channel map based on the aforementioned probability distribution parameters, If the target probability distribution channel map includes multiple probability distribution values, the step of determining the corresponding image enhancement segmental intensity value for each probability distribution value based on the image enhancement threshold interval corresponding to that probability distribution value, The process includes the steps of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmental intensity value corresponding to each probability distribution value, and obtaining a target reconstructed image block corresponding to the current image block. The decoding method according to feature 3.

5. The step of obtaining a target probability distribution channel map based on the aforementioned probability distribution parameters is: If the probability distribution parameters include important probability distribution channel maps and non-important probability distribution channel maps, the step includes upsampling the important probability distribution channel map to obtain the target probability distribution channel map. Here, the size of the target probability distribution channel map is the same as the size of the initial reconstructed image block. The decoding method according to feature 4.

6. The step of performing image-adaptive edge enhancement on the initial reconstructed image block based on the image enhancement segmental intensity values ​​corresponding to each of the aforementioned probability distribution values, and obtaining a target reconstructed image block corresponding to the current image block, is: The steps include generating a high-frequency detail image based on the initial reconstructed image block, For each feature value in the aforementioned high-frequency detail image, edge enhancement is performed on the feature value based on the image enhancement segmental intensity value corresponding to the probability distribution value corresponding to the feature value, and an image enhancement feature value is obtained. The steps include determining the target reconstructed image block based on image enhancement feature values ​​corresponding to each feature value in the high-frequency detail image, The decoding method according to feature 4.

7. The step of performing image adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, and obtaining a target reconstructed image block corresponding to the current image block, is: The process includes performing image-adaptive edge enhancement on an initial reconstructed image block corresponding to the luminance component of the current image block, based on the image domain enhancement parameter and the probability distribution parameter, and obtaining a target reconstructed image block corresponding to the luminance component. The decoding method according to feature 1.

8. The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and the decoding method is The steps include decoding the bitstream corresponding to the current image block to obtain an important channel identifier, Based on the aforementioned important channel identifier, the step of selecting the feature channel map corresponding to the aforementioned important channel identifier from the plurality of feature channel maps as the important feature channel map, and selecting the remaining feature channel maps as the unimportant feature channel maps, The further step includes selecting a probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting a probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map. The decoding method according to feature 5.

9. The initial reconstruction feature includes a plurality of feature channel maps, the probability distribution parameter includes a plurality of probability distribution channel maps, the plurality of probability distribution channel maps correspond one-to-one with the plurality of feature channel maps, and the decoding method is For each feature channel map, the step of determining the number of bits consumed by the feature channel map based on the feature values ​​of the feature channel map and the probability distribution values ​​of the probability distribution channel map corresponding to the feature channel map, The steps include selecting important feature channel maps from the plurality of feature channel maps based on the number of bits consumed by each feature channel map, and selecting the remaining feature channel maps as unimportant feature channel maps, The further step includes selecting a probability distribution channel map corresponding to the important feature channel map as the important probability distribution channel map, and selecting a probability distribution channel map corresponding to the non-important feature channel map as the non-important probability distribution channel map. The decoding method according to feature 5.

10. By decoding the first bitstream associated with the current image block, the coefficient hyperparameter features and the probability distribution parameters corresponding to the current image block are obtained. By decoding the second bitstream associated with the current image block, the initial reconstruction features corresponding to the current image block are obtained. By decoding the third bitstream associated with the current image block, the important channel identifier is obtained. Here, the first bitstream, the second bitstream, and the third bitstream are bitstreams on which different information is encoded. The decoding method according to feature 8.

11. The third bitstream mentioned above is a header information bitstream. The decoding method according to feature 10.

12. An encoding method, The steps include encoding coefficient hyperparameter features corresponding to the current image block and obtaining a first bitstream corresponding to the current image block, The steps include determining probability distribution parameters based on the aforementioned coefficient hyperparameter features, The steps include: encoding the initial image features corresponding to the current image block based on the probability distribution parameters and obtaining a second bitstream corresponding to the current image block; The process includes the steps of encoding a critical channel identifier and obtaining a third bitstream corresponding to the current image block, The step of encoding the important channel identifier and obtaining a third bitstream corresponding to the current image block is: The process includes the steps of encoding image domain enhancement parameters and obtaining the third bitstream, Here, the image domain enhancement parameter is used by the decoding side to perform image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter, and to obtain the target reconstructed image block corresponding to the current image block. An encoding method characterized by the following.

13. A decoding device, A decoding module configured to decode a bitstream corresponding to the current image block, obtain coefficient hyperparameter features corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, decode the bitstream corresponding to the current image block based on the probability distribution parameters, and obtain initial reconstruction features corresponding to the current image block, The system comprises a decision module configured to determine a target reconstructed image block corresponding to the current image block based on the initial reconstruction features, When the decision module determines the target reconstructed image block corresponding to the current image block based on the initial reconstruction features, it further determines the target reconstructed image block. The initial reconstruction features are input to the composite transformation network, and the initial reconstruction image block corresponding to the current image block is obtained. Based on the image domain enhancement parameters and probability distribution parameters corresponding to the current image block, the system is configured to perform image adaptive edge enhancement on the initial reconstructed image block and obtain a target reconstructed image block corresponding to the current image block. Here, the image domain enhancement parameter is obtained by decoding the bitstream corresponding to the current image block. A decoding device characterized by the following features.

14. The encoding module is configured to encode coefficient hyperparameter features corresponding to the current image block, obtain a first bitstream corresponding to the current image block, determine probability distribution parameters based on the coefficient hyperparameter features, encode initial image features corresponding to the current image block based on the probability distribution parameters, obtain a second bitstream corresponding to the current image block, encode important channel identifiers, and obtain a third bitstream corresponding to the current image block. When the encoding module encodes the important channel identifier and obtains a third bitstream corresponding to the current image block, It is configured to encode image domain enhancement parameters and obtain the third bitstream, Here, the image domain enhancement parameter is used by the decoding side to perform image-adaptive edge enhancement on the initial reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter, and to obtain the target reconstructed image block corresponding to the current image block. An encoding device characterized by the following features.

15. Decoding device, Processor and The system comprises a machine-readable storage medium in which machine-executable instructions that can be executed by the processor are stored, The processor is configured to perform the decoding method described in any one of claims 1 to 11 by executing the machine-executable instructions. A decoding device characterized by the following features.

16. Encoding device, Processor and The system comprises a machine-readable storage medium in which machine-executable instructions that can be executed by the processor are stored, The processor is configured to perform the encoding method described in claim 12 by executing the machine-executable instructions. An encoding device characterized by the following features.

17. A machine-readable storage medium, Multiple computer instructions are stored in the aforementioned machine-readable storage medium. When the computer instruction is executed by the processor, the method according to any one of claims 1 to 12 is performed. A machine-readable storage medium characterized by the following features.

18. It is a computer program, When the computer program is executed by the processor, the method according to any one of claims 1 to 12 is carried out. A computer program characterized by the following features.

Images