Decoding and coding method, device and equipment

Abstract

The invention provides a decoding method and device, a coding method and device and equipment. The decoding method comprises the following steps: decoding a first code stream corresponding to a current image block to obtain a coefficient hyper-parameter feature; determining a probability distribution parameter based on the coefficient hyper-parameter feature, and decoding a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature; determining an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature, wherein the initial reconstruction image block comprises an initial brightness reconstruction image block and an initial chroma reconstruction image block; and performing brightness edge enhancement on the initial brightness reconstruction image block to obtain a target brightness reconstruction image block. Through the technical scheme of the invention, the coding performance and the decoding performance can be improved.

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to a decoding and coding method, device and equipment thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. Complete video encoding can include prediction, transformation, quantization, entropy coding, filtering and other processes. For the prediction process, the prediction process can include intra-frame prediction and inter-frame prediction. Inter-frame prediction refers to the use of the correlation in the video time domain to use the pixels of the adjacent encoded images to predict the current pixel, so as to effectively remove the redundancy in the video time domain. Intra-frame prediction refers to the use of the correlation in the video space domain to use the pixels of the encoded blocks of the current frame image to predict the current pixel, so as to remove the redundancy in the video space domain.

[0003] With the rapid development of deep learning, deep learning has achieved success in many high-level computer vision problems, such as image classification and target detection. Deep learning has also gradually begun to be applied in the field of encoding and decoding, that is, neural networks can be used to encode and decode images. Although the encoding and decoding methods based on neural networks have shown great performance potential, they still have problems such as poor encoding performance, poor decoding performance and high complexity. Summary of the invention

[0004] In view of this, the present application provides a decoding and encoding method, device and equipment thereof to improve encoding performance and decoding performance.

[0005] The present application provides a decoding method, which is applied to a decoding end, and the method includes:

[0006] Decoding a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block;

[0007] Determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the second bitstream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block;

[0008] Determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0009] Brightness edge enhancement is performed on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block.

[0010] The present application provides an encoding method, which is applied to an encoding end, and the method includes:

[0011] Encoding the coefficient hyper-parameter features corresponding to the current image block to obtain a first bitstream corresponding to the current image block;

[0012] Determine probability distribution parameters based on the coefficient hyperparameter features, and encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block;

[0013] Determining an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature corresponding to the initial image feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0014] For each candidate image domain enhancement parameter, based on the candidate image domain enhancement parameter and the probability distribution parameter, an initial brightness reconstructed image block is enhanced in brightness to obtain a target brightness reconstructed image block; and based on the target brightness reconstructed image block, a cost value corresponding to the candidate image domain enhancement parameter is determined;

[0015] An image domain enhancement parameter corresponding to the current image block is selected from all candidate image domain enhancement parameters based on a cost value corresponding to each candidate image domain enhancement parameter, and the image domain enhancement parameter is encoded to obtain a third code stream corresponding to the current image block.

[0016] The present application provides a decoding device, applied to a decoding end, the device comprising:

[0017] A decoding module, configured to decode a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode a second bitstream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block;

[0018] A determination module, configured to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0019] The enhancement module is used to perform brightness edge enhancement on the initial brightness reconstruction image block to obtain a target brightness reconstruction image block.

[0020] The present application provides an encoding device, applied to an encoding end, the device comprising:

[0021] An encoding module, configured to encode coefficient hyperparameter features corresponding to a current image block to obtain a first bitstream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second bitstream corresponding to the current image block;

[0022] A determination module, configured to determine an initial reconstructed image block corresponding to the current image block based on an initial reconstruction feature corresponding to the initial image feature; the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0023] An enhancement module, configured to perform brightness edge enhancement on an initial brightness reconstructed image block based on each candidate image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block;

[0024] The determination module is further used to determine a cost value corresponding to the candidate image domain enhancement parameter based on the target brightness reconstructed image block; and select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the cost value corresponding to each candidate image domain enhancement parameter;

[0025] The encoding module is further used to encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0026] The present application provides a decoding end device, the decoding end device comprising: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor;

[0027] The processor is used to execute machine executable instructions to implement the above decoding method.

[0028] The present application provides an encoding end device, the decoding end device comprising: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor;

[0029] The processor is used to execute machine executable instructions to implement the above encoding method.

[0030] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned decoding method; or, the processor is used to execute the machine-executable instructions to implement the above-mentioned encoding method.

[0031] The present application provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the above-mentioned decoding method is implemented; or, the above-mentioned encoding method is implemented.

[0032] As can be seen from the above technical solutions, in the embodiment of the present application, an end-to-end video image compression method is proposed, which can realize the encoding and decoding of video images based on a neural network, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network can effectively ensure the quality of the reconstructed image block while maintaining low complexity, thereby achieving the purpose of improving the encoding performance and decoding performance, and reducing the complexity. The image quality of the features is enhanced using image domain enhancement parameters and probability distribution parameters. The encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream. The decoding end enhances the reconstructed image block through the image domain enhancement parameters, thereby improving the encoding performance and the quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic diagram of a three-dimensional feature matrix in one embodiment of the present application;

[0034] Figure 2 is a flowchart of a decoding method in one embodiment of the present application;

[0035] Figure 3 It is a schematic diagram of a flow chart of an encoding method in one embodiment of the present application;

[0036] Figure 4 is a schematic diagram of a processing process of an encoding end in one implementation manner of the present application;

[0037] Figure 5 is a schematic diagram of a processing process of a decoding end in one embodiment of the present application;

[0038] Fig. 6A is a schematic diagram of the location of a feature domain enhancement module in one embodiment of the present application;

[0039] Figure 6B is a schematic diagram of the positions of a feature domain enhancement module and an image domain enhancement module in one embodiment of the present application;

[0040] Figure 6C is a schematic diagram of the location of an image domain enhancement module in one embodiment of the present application;

[0041] Fig.6D is a schematic diagram of cross-channel filtering in one embodiment of the present application;

[0042] Fig. 6E is a schematic diagram of nonlinear chromaticity filtering in one embodiment of the present application;

[0043] Fig. 7A is a hardware structure diagram of a decoding end device in one embodiment of the present application;

[0044] Figure 7B It is a hardware structure diagram of an encoding end device in one implementation of the present application. DETAILED DESCRIPTION

[0045] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, depending on the context. In addition, the word "if" used can be interpreted as "at ... ", or "when ... ", or "in response to determination".

[0046] In the embodiments of the present application, a decoding method and an encoding method are proposed, which may involve the following concepts:

[0047] Entropy coding: Entropy coding is the coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information of the source (a measure of uncertainty). Entropy coding methods may include but are not limited to: Shannon coding, Huffman coding, and arithmetic coding.

[0048] Neural Network (NN): Neural network refers to artificial neural network. Neural network is a computing model composed of a large number of nodes (or neurons) connected to each other. In a neural network, neuron processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in a neural network can be divided into three categories: input units, output units, and hidden units. Input units receive signals and data from the outside world; output units realize the output of processing results; hidden units are units between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units, and the representation and processing of information are reflected in the connection relationship of processing units. Neural network is a non-programmed, brain-like information processing method. The essence of neural network is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of neural network, and imitate the information processing function of human brain nervous system to different degrees and levels. In the field of video processing, commonly used neural networks may include but are not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0049] Convolutional Neural Network (CNN): Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. The artificial neurons of convolutional neural network can respond to surrounding units within a certain coverage range and have excellent performance in large-scale image processing. The basic structure of convolutional neural network includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer and extracts the local features. Once the local feature is extracted, its positional relationship with other features is also determined. The second is the feature mapping layer (also called activation layer). Each computing layer of the neural network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since the neurons on a mapping surface share weights, the number of free parameters of the network is reduced.

[0050] For example, one of the advantages of convolutional neural networks over image processing algorithms is that they avoid the complex pre-processing of images (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of convolutional neural networks over ordinary neural networks is that ordinary neural networks use a fully connected approach, that is, all neurons from the input layer to the hidden layer are connected, which will result in a huge number of parameters, making network training time-consuming or even difficult to train, while convolutional neural networks avoid this difficulty through local connections, weight sharing and other methods.

[0051] Deconvolution layer: Deconvolution layer is also called transposed convolution layer. The working process of deconvolution layer and convolution layer is very similar. The main difference is that deconvolution layer will make the output larger than the input through padding (of course, it can also remain the same). If stride is 1, it means that the output size is equal to the input size; if stride is N, it means that the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0052] Generalization Ability: Generalization ability refers to the ability of a machine learning algorithm to adapt to fresh samples. The purpose of learning is to learn the rules hidden behind the data pairs. For data outside the learning set with the same rules, the trained network can also give appropriate outputs. This ability can be called generalization ability.

[0053] Feature: The feature involved in this application is a three-dimensional feature matrix or tensor of C*W*H, see Figure 1 As shown, it is a schematic diagram of a three-dimensional feature matrix, in which C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be the input of a neural network, or the output of a neural network.

[0054] Rate-Distortion Optimized: There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode: J(mode) = D + λ*R, where D represents Distortion, which can usually be measured using the SSE indicator. SSE refers to the mean square sum of the difference between the reconstructed image block and the source image. In order to achieve cost considerations, the SAD indicator can also be used. SAD refers to the sum of the absolute values ​​of the difference between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc. When selecting a mode, if the rate-distortion principle is used to make a comparison decision on the encoding mode, the best encoding performance can usually be guaranteed.

[0055] A large number of coding tools have been proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the coding tools that can achieve the best coding performance are often different. Therefore, in the encoding process, RDO (Rate-Distortion Optimize) is usually used to compare the coding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the marker information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to obtain the best coding performance. The decoding end can obtain relevant mode information by directly parsing the flag information, and the complexity impact is small.

[0056] In the general framework of end-to-end image coding, it mainly includes the feature main information part and the super prior side information part. The feature main information part includes analysis network, quantization, normal entropy coding, normal entropy decoding and synthesis network, and the super prior side information includes super prior analysis network, quantization, factored entropy coding, factored entropy decoding and super prior synthesis network. The image component x is compressed and encoded and reconstructed and restored by the analysis network and synthesis network of the feature main information part respectively; the super prior side information part is mainly used to model the probability of the feature main information and guide the entropy coding and decoding of the feature main information. In the general framework of end-to-end image coding, there is a problem that the super prior side information is not fully utilized, and the image reconstruction quality is not fully improved.

[0057] In view of the above findings, this embodiment utilizes the characteristics of the end-to-end image coding framework, and the decoding end uses the probability distribution parameters to enhance the quality of the reconstructed image to improve the quality of the reconstructed image. The encoding end does not directly change the main information, but encodes the image domain enhancement parameters into the header information bitstream (the third bitstream), and the decoding end enhances the reconstructed image through the image domain enhancement parameters.

[0058] The decoding method and encoding method in the embodiments of the present application are described in detail below in combination with several specific embodiments.

[0059] Example 1: A decoding method is proposed in the embodiment of the present application, see Figure 2 As shown, it is a flowchart of the decoding method, which can be applied to a decoding end (also called a video decoder), and the method may include:

[0060] Step 201: decode the first bitstream corresponding to the current image block to obtain coefficient hyper-parameter features corresponding to the current image block.

[0061] Step 202: Determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block.

[0062] Step 203: Determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block may include an initial brightness reconstructed image block and an initial chrominance reconstructed image block.

[0063] Step 204: Perform brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block.

[0064] Exemplarily, after determining the initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature, the initial chromaticity reconstructed image block can also be subjected to chromaticity filtering enhancement to obtain a target chromaticity reconstructed image block. For example, the chromaticity filtering enhancement and the brightness edge enhancement are synchronized and in parallel, and the chromaticity filtering enhancement and the brightness edge enhancement can be performed simultaneously, or the brightness edge enhancement can be performed first and then the chromaticity filtering enhancement, or the chromaticity filtering enhancement can be performed first and then the brightness edge enhancement, and there is no restriction on the enhancement order of brightness and chromaticity.

[0065] Exemplarily, after obtaining the target brightness reconstructed image block and the target chrominance reconstructed image block, the target reconstructed image block may be generated based on the target brightness reconstructed image block and the target chrominance reconstructed image block.

[0066] Exemplarily, in order to obtain a target chromaticity reconstructed image block, chromaticity filtering enhancement can be performed based on the initial luminance reconstructed image block and the initial chromaticity reconstructed image block to obtain the target chromaticity reconstructed image block, or chromaticity filtering enhancement can be performed based on the target luminance reconstructed image block and the initial chromaticity reconstructed image block to obtain the target chromaticity reconstructed image block.

[0067] Exemplarily, performing brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block may include, but is not limited to: decoding a third code stream corresponding to the current image block (the third code stream is also called the header information code stream corresponding to the current image block) to obtain image domain enhancement parameters corresponding to the current image block; performing brightness edge enhancement on the initial brightness reconstructed image block based on the image domain enhancement parameters and probability distribution parameters to obtain a target brightness reconstructed image block corresponding to the current image block.

[0068] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, the multiple image enhancement segmentation thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values. Based on this, the initial brightness reconstructed image block is enhanced in brightness edge based on the image domain enhancement parameters and the probability distribution parameters to obtain the target brightness reconstructed image block corresponding to the current image block, which may include but is not limited to: obtaining a target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then for each probability distribution value, the image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value; the initial brightness reconstructed image block is enhanced in brightness edge based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain the target brightness reconstructed image block corresponding to the current image block.

[0069] Exemplarily, obtaining a target probability distribution channel map based on probability distribution parameters may include, but is not limited to: decoding a third code stream corresponding to a current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameters include multiple probability distribution channel maps, selecting a probability distribution channel map corresponding to an important channel identifier from the multiple probability distribution channel maps as an important probability distribution channel map; and upsampling the important probability distribution channel map to obtain a target probability distribution channel map.

[0070] Exemplarily, performing brightness edge enhancement on an initial brightness reconstructed image block to obtain a target brightness reconstructed image block may include, but is not limited to: decoding a third code stream corresponding to a current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameters include multiple probability distribution channel maps, selecting a probability distribution channel map corresponding to the important channel identifier from the multiple probability distribution channel maps included in the probability distribution parameters as the important probability distribution channel map; upsampling the important probability distribution channel map to obtain a target probability distribution channel map; and performing brightness edge enhancement on the initial brightness reconstructed image block according to the target probability distribution channel map to obtain a target brightness reconstructed image block corresponding to the current image block.

[0071] Exemplarily, based on the image enhancement segmented intensity value corresponding to each probability distribution value, the initial brightness reconstructed image block is subjected to brightness edge enhancement to obtain a target brightness reconstructed image block corresponding to the current image block, which may include but is not limited to: generating a high-frequency detail image based on the initial brightness reconstructed image block; for each eigenvalue in the high-frequency detail image, performing brightness edge enhancement on the eigenvalue based on the image enhancement segmented intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determining the target brightness reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue.

[0072] Exemplarily, generating a high-frequency detail image based on the initial brightness reconstruction image block may include, but is not limited to: performing a convolution operation on the initial brightness reconstruction image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; and subtracting the Gaussian blur image from the initial brightness reconstruction image block to obtain a high-frequency detail image. For each value in the Gaussian blur convolution kernel, the denominator of the value may be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc.

[0073] Exemplarily, the Gaussian blur convolution kernel may be an N*N convolution kernel, where N may be a positive integer. If the value of N is 3, that is, the Gaussian blur convolution kernel is a 3*3 convolution kernel, then the Gaussian blur convolution kernel may include but is not limited to:

[0074]

[0075] A may be a positive integer, B may be a positive integer, and M may be a power of 2, such as M may be 2, 4, 8, 16, 32, 64, or 128. For example, A may be 5, B may be 24, and M may be 64.

[0076] Exemplarily, for step 204, the initial chromaticity reconstructed image block is subjected to chromaticity filtering enhancement to obtain a target chromaticity reconstructed image block, which may include but is not limited to: the initial chromaticity reconstructed image block may be subjected to chromaticity filtering enhancement based on the target brightness reconstructed image block or the initial brightness reconstructed image block to obtain the target chromaticity reconstructed image block.

[0077] Exemplarily, adaptive upsampling can be performed based on the initial chroma reconstructed image block and the initial luminance reconstructed image block to obtain an upsampled chroma reconstructed image block, and the resolution of the upsampled chroma reconstructed image block and the initial luminance reconstructed image block can be the same. A filtering operation is performed on the upsampled chroma reconstructed image block to obtain a target chroma reconstructed image block; wherein the filtering operation may include but is not limited to cross-channel filtering and / or nonlinear chroma filtering. For example, the upsampled chroma reconstructed image block can be cross-channel filtered based on the initial brightness reconstructed image block to obtain a target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be nonlinearly filtered based on the initial brightness reconstructed image block to obtain a target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be cross-channel filtered based on the initial brightness reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and the cross-channel filtered chroma reconstructed image block can be nonlinearly filtered based on the initial brightness reconstructed image block to obtain a target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be nonlinearly filtered based on the initial brightness reconstructed image block to obtain a nonlinear chroma reconstructed image block, and the nonlinear chroma reconstructed image block can be cross-channel filtered based on the initial brightness reconstructed image block to obtain a target chroma reconstructed image block.

[0078] Exemplarily, adaptive upsampling can be performed based on the initial chroma reconstructed image block and the target brightness reconstructed image block to obtain an upsampled chroma reconstructed image block, and the resolution of the upsampled chroma reconstructed image block and the target brightness reconstructed image block can be the same. A filtering operation is performed on the upsampled chroma reconstructed image block to obtain a target chroma reconstructed image block; wherein the filtering operation may include but is not limited to cross-channel filtering and / or nonlinear chroma filtering. For example, the upsampled chroma reconstructed image block can be cross-channel filtered based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be nonlinearly filtered based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be cross-channel filtered based on the target brightness reconstructed image block to obtain the cross-channel filtered chroma reconstructed image block, and the cross-channel filtered chroma reconstructed image block can be nonlinearly filtered based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, the upsampled chroma reconstructed image block can be nonlinearly filtered based on the target brightness reconstructed image block to obtain the nonlinear chroma reconstructed image block, and the nonlinear chroma reconstructed image block can be cross-channel filtered based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block.

[0079] Exemplarily, determining the initial reconstructed image block corresponding to the current image block based on the initial reconstructed features may include, but is not limited to: inputting the initial reconstructed features into a synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block. Alternatively, decoding the third bitstream corresponding to the current image block to obtain feature domain enhancement parameters corresponding to the current image block; performing feature domain enhancement on the initial reconstructed features based on the feature domain enhancement parameters and probability distribution parameters to obtain enhanced reconstruction features; and inputting the enhanced reconstruction features into a synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block.

[0080] Exemplarily, the third bitstream corresponding to the current image block can also be decoded to obtain the important channel identifier corresponding to the current image block; wherein the initial reconstruction feature includes C feature channel maps, the probability distribution parameter includes C probability distribution channel maps, and the C probability distribution channel maps and the C feature channel Figure 1 One by one, the feature channel map corresponding to the important channel identifier is used as the important feature channel map, the remaining feature channel maps are used as the unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is used as the important probability distribution channel map, and the remaining probability distribution channel maps are used as the unimportant probability distribution channel maps.

[0081] The initial reconstructed feature is enhanced in the feature domain based on the feature domain enhancement parameters and the probability distribution parameters to obtain an enhanced reconstructed feature, which may include but is not limited to: performing feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstructed feature after feature adaptive edge enhancement; performing feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching; and generating an enhanced reconstruction feature based on the first reconstructed feature and the second reconstructed feature.

[0082] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segmentation strength values ​​and multiple edge enhancement segmentation thresholds, and the multiple edge enhancement segmentation thresholds may form multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. On this basis, based on the feature domain enhancement parameters and the important probability distribution channel map, the important feature channel map is subjected to feature adaptive edge enhancement to obtain the first reconstructed feature after feature adaptive edge enhancement, which may include but is not limited to: if the important probability distribution channel map includes multiple probability distribution values, then, for each probability distribution value, the edge enhancement segmentation strength value corresponding to the probability distribution value may be determined based on the edge enhancement threshold interval corresponding to the probability distribution value; then, based on the edge enhancement segmentation strength value corresponding to each probability distribution value, the important feature channel map is subjected to feature adaptive edge enhancement to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0083] Exemplarily, based on the edge enhancement segmented strength value corresponding to each probability distribution value, feature-adaptive edge enhancement is performed on the important feature channel map to obtain the first reconstructed feature after feature-adaptive edge enhancement, which may include but is not limited to: normalizing the important feature channel map to obtain a normalized feature map; generating a high-frequency detail image based on the important feature channel map and the normalized feature map; for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segmented strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhanced eigenvalue; based on the edge enhancement eigenvalue corresponding to each eigenvalue, an edge enhancement feature map is determined; and the edge enhancement feature map is denormalized to obtain the first reconstructed feature.

[0084] Exemplarily, the feature domain enhancement parameters may include a stretching parameter value, and feature adaptive stretching is performed on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching, which may include but is not limited to: if the unimportant feature channel map may include multiple eigenvalues, and the unimportant probability distribution channel map may include multiple probability distribution values, then, for each eigenvalue in the unimportant feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue; then, the second reconstructed feature is determined based on the stretched eigenvalue corresponding to each eigenvalue in the unimportant feature channel map.

[0085] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0086] It can be seen from the above technical scheme that in the embodiment of the present application, after obtaining the initial brightness reconstruction image block and the initial chroma reconstruction image block corresponding to the current image block, the initial brightness reconstruction image block can be enhanced by brightness edge enhancement to obtain the target brightness reconstruction image block, the initial chroma reconstruction image block can be enhanced by chroma filtering to obtain the target chroma reconstruction image block, and the target reconstruction image block can be generated based on the target brightness reconstruction image block and the target chroma reconstruction image block. An end-to-end video image compression method is proposed, which can realize the encoding and decoding of video images based on neural networks, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network effectively guarantees the quality of the reconstructed image block while maintaining low complexity, so as to achieve the purpose of improving the encoding performance and decoding performance, and reduce the complexity. The image quality of the features is enhanced by using the image domain enhancement parameters and the probability distribution parameters. The encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream. The decoding end enhances the reconstructed image block through the image domain enhancement parameters, improves the encoding performance, and improves the quality of the reconstructed image.

[0087] Example 2: This application proposes an encoding method, see Figure 3 As shown, it is a schematic diagram of the process of the encoding method, which can be applied to an encoding end (also called a video encoder), and the method may include:

[0088] Step 301: Encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block.

[0089] Step 302: Determine a probability distribution parameter based on the coefficient hyperparameter feature, and encode the initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block.

[0090] Step 303: Determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature corresponding to the initial image feature; wherein the initial reconstructed image block may include an initial brightness reconstructed image block and an initial chrominance reconstructed image block.

[0091] Step 304: for each candidate image domain enhancement parameter, the initial brightness reconstructed image block is enhanced in brightness edge based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block; and the cost value corresponding to the candidate image domain enhancement parameter is determined based on the target brightness reconstructed image block. For example, the initial chroma reconstructed image block may be enhanced by chroma filtering to obtain a target chroma reconstructed image block, a target reconstructed image block is generated based on the target brightness reconstructed image block and the target chroma reconstructed image block, and the cost value corresponding to the candidate image domain enhancement parameter is determined based on the target reconstructed image block.

[0092] Step 305: Select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the cost value corresponding to each candidate image domain enhancement parameter, encode the image domain enhancement parameter, and obtain a third code stream corresponding to the current image block (the third code stream may also be referred to as a header information code stream corresponding to the current image block).

[0093] Exemplarily, the processing process of the encoding end is similar to the processing process of the decoding end, and the similarities are not repeated here. The processing process of the decoding end can be applied to the encoding end, that is, the encoding end adopts the same processing method.

[0094] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0095] It can be seen from the above technical scheme that in the embodiment of the present application, after obtaining the initial brightness reconstruction image block and the initial chroma reconstruction image block corresponding to the current image block, the initial brightness reconstruction image block can be enhanced by brightness edge enhancement to obtain the target brightness reconstruction image block, the initial chroma reconstruction image block can be enhanced by chroma filtering to obtain the target chroma reconstruction image block, and the target reconstruction image block can be generated based on the target brightness reconstruction image block and the target chroma reconstruction image block. An end-to-end video image compression method is proposed, which can realize the encoding and decoding of video images based on neural networks, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network effectively guarantees the quality of the reconstructed image block while maintaining low complexity, so as to achieve the purpose of improving the encoding performance and decoding performance, and reduce the complexity. The image quality of the features is enhanced by using the image domain enhancement parameters and the probability distribution parameters. The encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream. The decoding end enhances the reconstructed image block through the image domain enhancement parameters, improves the encoding performance, and improves the quality of the reconstructed image.

[0096] Embodiment 3: For Embodiment 1 and Embodiment 2, the processing process of the encoding end can be seen in Figure 4 As shown, of course, Figure 4 This is just an example of the processing process at the encoding end, and there is no limitation on the processing process at the encoding end.

[0097] After obtaining the current image block x (the current image block x may be the original image block x, i.e., the input image block), the encoder may analyze and transform the current image block x through an analysis transformation network (i.e., a neural network) to obtain an image feature y corresponding to the current image block x. Among them, performing feature transformation on the current image block x through the analysis transformation network means: transforming the current image block x to an image feature y in the latent domain, thereby facilitating all subsequent processes to operate on the latent domain.

[0098] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the encoding and decoding process for the image block can also be directly used for the image.

[0099] After obtaining the image feature y, the encoding end performs coefficient hyperparameter feature transformation on the image feature y to obtain coefficient hyperparameter feature z. For example, the image feature y can be input into a hyperparameter encoding network (i.e., a neural network), and the hyperparameter encoding network performs coefficient hyperparameter feature transformation on the image feature y to obtain coefficient hyperparameter feature z. The hyperparameter encoding network can be a trained neural network, and there is no restriction on the training process of the hyperparameter encoding network, as long as the coefficient hyperparameter feature transformation can be performed on the image feature y. The image feature y in the latent domain obtains the super prior latent information z after passing through the hyperparameter encoding network.

[0100] After obtaining the coefficient hyperparameter feature z, the encoder can quantize the coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, that is, Figure 4 The Q operation in is the quantization process. After obtaining the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain Bitstream#1 (i.e., the first bitstream) corresponding to the current image block, i.e. Figure 4 The AE operation in represents the encoding process, such as the entropy encoding process. Alternatively, the encoding end can also directly encode the coefficient hyperparameter feature z to obtain the Bitstream#1 corresponding to the current image block. Among them, the hyperparameter quantization feature or coefficient hyperparameter feature z carried in Bitstream#1 is mainly used to obtain the parameters of the mean and probability distribution model.

[0101] After obtaining the Bitstream#1 corresponding to the current image block, the encoder may send the Bitstream#1 corresponding to the current image block to the decoder. For the processing process of the decoder for the Bitstream#1 corresponding to the current image block, refer to the subsequent embodiments.

[0102] After obtaining the Bitstream#1 corresponding to the current image block, the encoder can also decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 4 AD in it represents the decoding process. Then, the hyperparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat can be the same as or different from the coefficient hyperparameter feature z. Figure 4 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current image block, the encoder can also decode the Bitstream#1 to obtain the coefficient hyperparameter feature z_hat without involving the inverse quantization process of the coefficient hyperparameter feature z_hat.

[0103] For the encoding process of Bitstream#1, the encoding method of the fixed probability density model can be adopted, and for the decoding process of Bitstream#1, the decoding method of the fixed probability density model can be adopted. There is no restriction on this encoding and decoding process.

[0104] After obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (the determination process of the reconstruction feature y_hat is described in the subsequent embodiments) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat, and there is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu, and the prediction value mu is used to obtain the residual r_hat by subtracting from the original feature and adding it to the decoded residual to obtain the reconstructed feature y_hat.

[0105] It should be noted that the mean prediction network is an optional neural network, that is, there is no mean prediction network, that is, there is no need to determine the predicted value mu through the mean prediction network. Figure 4 The dashed box in indicates that the mean prediction network is optional.

[0106] After obtaining the image feature y, the encoder can determine the residual feature r based on the image feature y and the prediction value mu, such as taking the difference between the image feature y and the prediction value mu as the residual feature r. Then, the residual feature r is feature processed to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the prediction value mu. Alternatively, after obtaining the image feature y, the encoder can perform feature processing on the image feature y to obtain the image feature s. There is no restriction on this feature processing process, and it can be any feature processing method. In this case, there is no need to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0107] After obtaining the image feature s, the encoder can quantize the image feature s to obtain the image quantization feature corresponding to the image feature s, that is, Figure 4 The Q operation in is the quantization process. After obtaining the image quantization feature corresponding to the image feature s, the encoding end can encode the image quantization feature to obtain Bitstream#2 (i.e., the second bitstream) corresponding to the current image block, i.e. Figure 4 The AE operation in represents an encoding process, such as an entropy encoding process. Alternatively, the encoding end may directly encode the image feature s to obtain Bitstream#2 corresponding to the current image block without involving the quantization process of the image feature s.

[0108] After obtaining Bitstream#2 corresponding to the current image block, the encoder may send Bitstream#2 corresponding to the current image block to the decoder. For the processing of Bitstream#2 corresponding to the current image block by the decoder, refer to the subsequent embodiments.

[0109] After obtaining the Bitstream#2 corresponding to the current image block, the encoder can also decode Bitstream#2 to obtain the image quantization feature, that is, Figure 4 AD in the figure represents the decoding process. Then, the encoder can dequantize the image quantization feature to obtain the image feature s'. The image feature s' can be the same as or different from the image feature s. Figure 4 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the encoder can also decode the Bitstream#2 to obtain the image feature s' without involving the inverse quantization process of the image quantization feature.

[0110] After obtaining the image feature s', the encoder can perform feature recovery (i.e., the inverse process of feature processing) on ​​the image feature s'. There is no restriction on this feature recovery process, and it can be any feature recovery method to obtain the residual feature r_hat. The residual feature r_hat and the residual feature r can be the same or different. After obtaining the residual feature r_hat, the encoder determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the prediction value mu. The image feature y_hat is the same or different from the image feature y. For example, the sum of the residual feature r_hat and the prediction value mu is used as the image feature y_hat. In this case, it is necessary to deploy a mean prediction network, and the mean prediction network provides the prediction value mu. Alternatively, after obtaining the image feature s', the encoder performs feature recovery (i.e., the inverse process of feature processing) on ​​the image feature s' to obtain the image feature y_hat. The image feature y_hat and the image feature y can be the same or different. In this case, it is not necessary to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0111] After obtaining the image feature y_hat, the encoding end can perform a synthetic transformation on the image feature y_hat to obtain a reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input into the synthetic transformation network, and the synthetic transformation network performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat. At this point, the image reconstruction process is completed.

[0112] In a possible implementation, when the encoding end encodes the image quantization feature or the image feature s to obtain the Bitstream#2 corresponding to the current image block, the encoding end needs to first determine the probability distribution model, and then encode the image quantization feature or the image feature s based on the probability distribution model. In addition, when the encoding end decodes Bitstream#2, it also needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model.

[0113] To obtain the probability distribution model, continue to refer to Figure 4 As shown, after obtaining the coefficient hyperparameter feature z_hat, the encoding end can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter. For example, the coefficient hyperparameter feature z_hat is input to the probability hyperparameter decoding network, and the probability hyperparameter decoding network performs a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. After obtaining the probability distribution parameter, a probability distribution model can be generated based on the probability distribution parameter. Among them, the probability hyperparameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability hyperparameter decoding network, and it is sufficient to be able to perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat.

[0114] In a possible implementation, the processing process at the encoding end may be performed by a deep learning model or a neural network model, thereby realizing an end-to-end image compression and encoding process, without any limitation on the encoding process.

[0115] Embodiment 4: For Embodiment 1 and Embodiment 2, the processing process of the decoding end can be seen in Figure 5 As shown, of course, Figure 5 This is just an example of the processing process at the decoding end, and there is no limitation on the processing process at the decoding end.

[0116] After obtaining the Bitstream#1 corresponding to the current image block, the decoding end can also decode Bitstream#1 to obtain the hyperparameter quantization feature, that is, Figure 5 AD in it represents the decoding process. Then, the hyperparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat. The coefficient hyperparameter feature z_hat can be the same as or different from the coefficient hyperparameter feature z. Figure 5 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current image block, the decoder can also decode the Bitstream#1 to obtain the coefficient hyperparameter feature z_hat without involving the inverse quantization process of the coefficient hyperparameter feature z_hat.

[0117] For the decoding process of Bitstream#1, a decoding method of a fixed probability density model may be used, and there is no restriction on this.

[0118] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be an image, that is, the decoding process for the image block can also be directly used for the image.

[0119] After obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform context-based prediction based on the coefficient hyperparameter feature z_hat of the current image block and the reconstruction feature y_hat of the previous image block (the determination process of the reconstruction feature y_hat is shown in the subsequent embodiments) to obtain the prediction value mu (i.e., mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat are input to the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstruction feature y_hat. There is no restriction on this prediction process. Among them, for the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstruction feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu.

[0120] It should be noted that the mean prediction network is an optional neural network, that is, there is no mean prediction network, that is, there is no need to determine the predicted value mu through the mean prediction network. Figure 5 The dashed box in indicates that the mean prediction network is optional.

[0121] After obtaining the Bitstream#2 corresponding to the current image block, the decoding end can also decode Bitstream#2 to obtain the image quantization feature, that is, Figure 5 AD in the figure represents the decoding process. Then, the decoding end can dequantize the image quantization feature to obtain the image feature s'. The image feature s' can be the same as or different from the image feature s. Figure 5 The IQ operation in is the inverse quantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the decoder can also decode the Bitstream#2 to obtain the image feature s' without involving the inverse quantization process of the image quantization feature.

[0122] After obtaining the image feature s', the decoder can perform feature recovery on the image feature s' (i.e., the inverse process of feature processing) to obtain the residual feature r_hat, which is the same as or different from the residual feature r. After obtaining the residual feature r_hat, the decoder determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the prediction value mu. The image feature y_hat is the same as or different from the image feature y, such as taking the sum of the residual feature r_hat and the prediction value mu as the image feature y_hat. In this case, it is necessary to deploy a mean prediction network, which provides the prediction value mu. Alternatively, after obtaining the image feature s', the decoder can perform feature recovery on the image feature s' to obtain the image feature y_hat, which is the same as or different from the image feature y. In this case, there is no need to deploy a mean prediction network, and the dotted box indicates that the residual process is an optional process.

[0123] After obtaining the image feature y_hat, the decoding end can perform a synthetic transformation on the image feature y_hat to obtain a reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat is input into the synthetic transformation network, and the synthetic transformation network performs a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat. At this point, the image reconstruction process is completed.

[0124] In a possible implementation, when decoding Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model. To obtain the probability distribution model, continue to refer to Figure 5As shown, after obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. For example, the coefficient hyperparameter feature z_hat is input into the probability hyperparameter decoding network, and the probability hyperparameter decoding network performs a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter. After obtaining the probability distribution parameter, a probability distribution model can be generated based on the probability distribution parameter. Among them, the probability hyperparameter decoding network can be a trained neural network, and there is no restriction on the training process of this probability hyperparameter decoding network. It can perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter.

[0125] In a possible implementation, the processing process at the above decoding end can be performed by a deep learning model or a neural network model, thereby realizing an end-to-end image compression and encoding process, without limiting the decoding process.

[0126] Embodiment 5: Based on Embodiment 3 and Embodiment 4, a feature domain enhancement module can be added before the synthesis transformation network. Fig. 6A As shown, the input feature of the feature domain enhancement module can be the image feature y_hat (hereinafter referred to as the initial reconstruction feature y_hat), and the output feature of the feature domain enhancement module can be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input to the synthesis transformation network, and the synthesis transformation network performs a synthesis transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0127] For the encoding end: After obtaining the current image block x, the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block (which can be the coefficient hyperparameter feature z itself or the hyperparameter quantization feature of the coefficient hyperparameter feature z) is encoded to obtain the first code stream corresponding to the current image block.

[0128] The first bitstream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (e.g., the coefficient hyperparameter feature z_hat itself is decoded from the first bitstream, or the hyperparameter quantization feature is decoded from the first bitstream, and the hyperparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat). Then, the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0129] Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded (the initial image feature can be the image feature y, or the residual feature r corresponding to the image feature y, or the image feature s after feature processing of the image feature y or the residual feature r, or the image quantization feature corresponding to the image feature y, or the residual feature r, or the image feature s, and there is no restriction on this initial image feature) to obtain a second code stream corresponding to the current image block.

[0130] The second bitstream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, if the initial image feature is the image feature y, the initial reconstruction feature y_hat is decoded from the second bitstream. If the initial image feature is the image quantization feature corresponding to the image feature y, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the initial reconstruction feature y_hat. For another example, if the initial image feature is the residual feature r corresponding to the image feature y, the residual feature r_hat is decoded from the second bitstream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the residual feature r, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the residual feature r_hat, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image feature y or the image feature s corresponding to the residual feature r, the image feature s' is decoded from the second bitstream, and the image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. If the initial image feature is the image quantization feature corresponding to the image feature s, the image quantization feature is decoded from the second bitstream, and the image quantization feature is dequantized to obtain the image feature s'. The image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0131] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0132] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (such as decoding the coefficient hyperparameter feature z_hat itself from the first bitstream, or decoding the hyperparameter quantization feature from the first bitstream, and dequantizing the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat). Then, perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0133] Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. For example, the initial reconstruction feature y_hat is decoded from the second code stream. Alternatively, the image quantization feature is decoded from the second code stream, and the image quantization feature is dequantized to obtain the initial reconstruction feature y_hat. For another example, the residual feature r_hat is decoded from the second code stream, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second code stream, and the image quantization feature is dequantized to obtain the residual feature r_hat, and the initial reconstruction feature y_hat is determined based on the residual feature r_hat and the predicted value mu. For another example, the image feature s' is decoded from the second code stream, and the image feature s' is feature restored to obtain the initial reconstruction feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstruction feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu. Alternatively, the image quantization feature is decoded from the second bit stream, and the image quantization feature is dequantized to obtain the image feature s', and the image feature s' is restored to obtain the initial reconstructed feature y_hat or the residual feature r_hat. If the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the predicted value mu.

[0134] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0135] Embodiment 6: Based on Embodiment 3 and Embodiment 4, a feature domain enhancement module may be added before the synthesis transformation network, and a filter enhancement module may be added after the synthesis transformation network. The filter enhancement module may include a brightness edge enhancement module and a chroma filter enhancement module. Figure 6B As shown, the input feature of the feature domain enhancement module can be the initial reconstruction feature y_hat, and the output feature of the feature domain enhancement module is the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced can be input to the synthesis transformation network, and the synthesis transformation network performs synthesis transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x. The input feature of the filter enhancement module can be the initial reconstructed image block x_hat, and the output feature of the filter enhancement module can be the target reconstructed image block x_hat_enhanced. For example, the initial reconstructed image block x_hat includes the initial brightness reconstruction image block x_Y_hat and the initial chrominance reconstruction image block x_UV_hat. The brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstruction image block x_Y_hat to obtain the target brightness reconstruction image block x_Y_hat_enhanced. The chrominance filter enhancement module performs chrominance filter enhancement on the initial chrominance reconstruction image block x_UV_hat to obtain the target chrominance reconstruction image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chrominance reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced.

[0136] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second code stream corresponding to the current image block. Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. Among them, the above process can be referred to Example 5, which will not be repeated here.

[0137] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x, wherein the initial reconstructed image block x_hat includes an initial luminance reconstructed image block x_Y_hat and an initial chrominance reconstructed image block x_UV_hat.

[0138] After obtaining the initial brightness reconstructed image block x_Y_hat corresponding to the current image block x, the brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain the target brightness reconstructed image block x_Y_hat_enhanced. After obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat_enhanced to obtain the target chroma reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0139] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, decode the second bitstream corresponding to the current image block to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5, which will not be repeated here.

[0140] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input into the synthesis transformation network, which performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x, wherein the initial reconstructed image block x_hat includes an initial luminance reconstructed image block x_Y_hat and an initial chrominance reconstructed image block x_UV_hat.

[0141] After obtaining the initial brightness reconstructed image block x_Y_hat corresponding to the current image block x, the brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain the target brightness reconstructed image block x_Y_hat_enhanced. After obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat_enhanced to obtain the target chroma reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0142] Embodiment 7: Based on Embodiment 3 and Embodiment 4, a filter enhancement module may be added after the synthesis transformation network, and the filter enhancement module may include a brightness edge enhancement module and a chroma filter enhancement module. Figure 6CAs shown, the input feature of the filtering enhancement module may be the initial reconstructed image block x_hat, and the output feature of the filtering enhancement module may be the target reconstructed image block x_hat_enhanced. For example, the initial reconstructed image block x_hat may include an initial brightness reconstructed image block x_Y_hat and an initial chrominance reconstructed image block x_UV_hat. The brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain a target brightness reconstructed image block x_Y_hat_enhanced. The chrominance filtering enhancement module performs chrominance filtering enhancement on the initial chrominance reconstructed image block x_UV_hat_enhanced to obtain a target chrominance reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chrominance reconstructed image block x_UV_hat_enhanced may be combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0143] For the encoding end: the current image block x is analyzed and transformed through the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation through the hyperparameter coding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and the coefficient hyperparameter feature z_hat is inversely transformed through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, the initial image feature corresponding to the current image block is encoded to obtain the second code stream corresponding to the current image block. Based on the probability distribution parameter sigma, the second code stream corresponding to the current image block is decoded to obtain the initial reconstruction feature y_hat corresponding to the current image block. Among them, the above process can be referred to Example 5, which will not be repeated here.

[0144] After obtaining the initial reconstruction feature y_hat, the initial reconstruction feature y_hat is input into the synthesis transformation network (different from Examples 5 and 6, the enhanced reconstruction feature y_hat_enhanced is not input into the synthesis transformation network here), and the synthesis transformation network performs a synthesis transformation on the initial reconstruction feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x, and the initial reconstructed image block x_hat includes an initial luminance reconstruction image block x_Y_hat and an initial chrominance reconstruction image block x_UV_hat.

[0145] After obtaining the initial brightness reconstructed image block x_Y_hat corresponding to the current image block x, the brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain the target brightness reconstructed image block x_Y_hat_enhanced. After obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat_enhanced to obtain the target chroma reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0146] For the decoding end: decode the first bitstream corresponding to the current image block to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, and perform a coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat through the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. Based on the probability distribution parameter sigma, decode the second bitstream corresponding to the current image block to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to Example 5, which will not be repeated here.

[0147] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input into the synthesis transformation network, and the synthesis transformation network performs a synthesis transformation on the initial reconstructed feature y_hat to obtain the initial reconstructed image block x_hat corresponding to the current image block x, and the initial reconstructed image block x_hat includes an initial brightness reconstruction image block x_Y_hat and an initial chrominance reconstruction image block x_UV_hat.

[0148] After obtaining the initial brightness reconstructed image block x_Y_hat corresponding to the current image block x, the brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain the target brightness reconstructed image block x_Y_hat_enhanced. After obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat_enhanced to obtain the target chroma reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0149] Example 8: In Example 5 and Example 6, the initial reconstructed feature y_hat is input into the feature domain enhancement module, and the feature domain enhancement module performs feature domain enhancement on the initial reconstructed feature y_hat to obtain an enhanced reconstructed feature y_hat_enhanced. For example, the initial reconstructed feature y_hat can be enhanced in the feature domain based on the feature domain enhancement parameters and probability distribution parameters sigma corresponding to the current image block to obtain the enhanced reconstructed feature y_hat_enhanced. The feature domain enhancement can include feature adaptive edge enhancement and feature adaptive stretching. The probability distribution parameter sigma is used to assist in calculation, determine the channels that require feature adaptive edge enhancement and the channels that require feature adaptive stretching, and determine the strength of the stretching. The process is described below.

[0150] The initial reconstruction feature y_hat is C L xH L xW L The three-dimensional tensor, C L is the number of channels of the initial reconstructed feature y_hat (i.e., the number of channels in the feature domain and the number of channels in the Latent domain), H L is the feature height of the initial reconstructed feature y_hat (i.e., the feature height of the feature domain and the feature height of the Latent domain), W L is the feature width of the initial reconstructed feature y_hat (i.e., the feature width of the feature domain and the feature width of the Latent domain). For each channel of the initial reconstructed feature y_hat, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of the channel ch can be H L xW L The two-dimensional tensor y_hat_ch of C is called the feature channel map. Therefore, the initial reconstruction feature y_hat can include C L Feature channel map.

[0151] The probability distribution parameter sigma is also C L xH L xW L The three-dimensional tensor, C L is the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. For each channel of the probability distribution parameter sigma, take channel ch as an example, ch can be [1, 2, ..., C L ], the shape of channel ch is H L xW LThe two-dimensional tensor sigma_ch is called the probability distribution channel map, so the probability distribution parameter sigma can include C L probability distribution channel graph, and C L The probability distribution channel diagram and C L Feature channels Figure 1 One-to-one correspondence, such as the first probability distribution channel map corresponds to the first feature channel map, the second probability distribution channel map corresponds to the second feature channel map, and so on. L The probability distribution channel diagram and the C L The feature channel map corresponds to

[0152] For C L feature channel map, C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps. There can be at least one important feature channel map and multiple unimportant feature channel maps. For example, C L At least one feature channel map is selected from the feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps. L The probability distribution channel diagram and C L Feature channels Figure 1 One-to-one correspondence, therefore, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map. For example, if the first feature channel map is an important feature channel map, the first probability distribution channel map corresponding to the first feature channel map is selected as the important probability distribution channel map.

[0153] In summary, the initial reconstructed feature y_hat may include an important feature channel map and an unimportant feature channel map, and the probability distribution parameter sigma may include an important probability distribution channel map corresponding to the important feature channel map and an unimportant probability distribution channel map corresponding to the unimportant feature channel map. For the important feature channel map, the important feature channel map may be feature-adaptively edge enhanced based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstructed feature after feature-adaptive edge enhancement. For the unimportant feature channel map, the unimportant feature channel map may be feature-adaptively stretched based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature-adaptive stretching. On this basis, an enhanced reconstructed feature y_hat_enhanced may be generated based on the first reconstructed feature and the second reconstructed feature.

[0154] For example, assuming that the feature channel Figure 1 is an important feature channel map, feature channel Figure 2 - Feature Channel Figure 4is a non-important feature channel map, probability distribution channel Figure 1 It is an important probability distribution channel diagram, probability distribution channel Figure 2 -Probability distribution channel Figure 4 For non-important probability distribution channel graphs, then: the parameters and probability distribution channels can be enhanced based on the feature domain Figure 1 For feature channels Figure 1 Perform feature adaptive edge enhancement to obtain the first reconstructed feature 1 (i.e., the reconstructed feature of the first channel) after feature adaptive edge enhancement. Figure 2 For feature channels Figure 2 The feature is adaptively stretched to obtain the second reconstructed feature 2 (i.e., the reconstructed feature of the second channel) after the feature adaptive stretching. The feature domain enhancement parameters and probability distribution channels can be Figure 3 For feature channels Figure 3 The feature is adaptively stretched to obtain the second reconstructed feature 3 after the feature adaptive stretching. The feature domain enhancement parameters and probability distribution channels can be Figure 4 For feature channels Figure 4 The feature is adaptively stretched to obtain the second reconstruction feature 4 after feature adaptive stretching. Then, the first reconstruction feature 1, the second reconstruction feature 2, the second reconstruction feature 3 and the second reconstruction feature 4 can be combined according to the channel (such as splicing according to the channel), that is, the reconstruction features of the four channels are combined to obtain the enhanced reconstruction feature y_hat_enhanced.

[0155] Embodiment 9: In embodiment 8, it is necessary to replace C L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps. For example, C is divided into important feature channel maps and unimportant feature channel maps in the following way. L The feature channel maps are divided into important feature channel maps and unimportant feature channel maps:

[0156] Mode 1: The encoder encodes the important channel identifier in the third bitstream corresponding to the current image block, and the decoder decodes the important channel identifier from the third bitstream corresponding to the current image block, so that the decoder can obtain the important channel identifier.

[0157] For example, the encoding end will C L The feature channel map is divided into important feature channel maps and unimportant feature channel maps, and there is no restriction on this process. The important channel identifier (also called important channel number, recorded as important_channel) is encoded in the third bitstream corresponding to the current image block. After receiving the third bitstream, the decoding end decodes the important channel identifier from the third bitstream and obtains the important channel identifier from C LAmong the feature channel maps, the feature channel map corresponding to the important channel identifier is selected as the important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map.

[0158] Among them, the encoding end is C L When the feature channel graphs are divided into important feature channel graphs and unimportant feature channel graphs, the feature values ​​in each feature channel graph and the probability distribution values ​​in each probability distribution channel graph can be used to obtain the feature channel graphs from C. L The important feature channel maps are selected from the feature channel maps, and the remaining feature channel maps are selected as unimportant feature channel maps.

[0159] Method 2: For both the encoding end and the decoding end, based on the feature values ​​in each feature channel map and the probability distribution values ​​in each probability distribution channel map, L Select the important feature channel maps from the feature channel maps.

[0160] For example, for each feature channel map, the number of consumed bits corresponding to the feature channel map can be determined based on the feature value in the feature channel map and the probability distribution value in the probability distribution channel map corresponding to the feature channel map. For example, the number of consumed bits corresponding to the feature channel map can be determined using the following expression, which is, of course, only an example.

[0161]

[0162] In the above expression, bits_per_ch can be used to represent the number of consumed bits corresponding to the feature channel graph ch, y_hat_ch(i,j) can be used to represent the eigenvalue of the feature point (i,j) in the feature channel graph ch, sigma_ch(i,j) can be used to represent the probability distribution value of the feature point (i,j) in the probability distribution channel graph ch, and Φ(.) can be the standard normal distribution cumulative function.

[0163] After performing the above processing on each feature channel map, the number of consumed bits corresponding to each feature channel map can be obtained. Based on the number of consumed bits corresponding to each feature channel map, it can be obtained from C L An important feature channel map is selected from the feature channel maps. For example, the feature channel map consuming the largest number of bits is selected as the important feature channel map, or K feature channel maps consuming a large number of bits are selected as the important feature channel maps, where K can be a positive integer greater than 1.

[0164] In from C LAfter selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map.

[0165] Method 3: For both the encoding and decoding ends, the bit rate consumption corresponding to each feature channel map is calculated from C L For example, based on the consumed bit rate corresponding to each feature channel map, C L The feature channel maps are sorted, and the top K feature channel maps are selected as important feature channel maps. Alternatively, based on the consumed bit rate corresponding to each feature channel map, the C feature channel maps can be sorted in order of the consumed bit rate from small to large. L The feature channel maps are sorted, and the K feature channel maps with the lowest sorting order are selected as important feature channel maps.

[0166] In from C L After selecting important feature channel maps from the feature channel maps, the remaining feature channel maps can be selected as unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map can be selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map can be selected as the unimportant probability distribution channel map.

[0167] Method 4: For both the encoding end and the decoding end, the default feature channel map (i.e., the fixed feature channel map) is used as the important feature channel map. For example, the first feature channel map is pre-agreed as the important feature channel map, or the sixth feature channel map is pre-agreed as the important feature channel map, or the tenth feature channel map is pre-agreed as the important feature channel map, and so on. Of course, any feature channel map can be used as an important feature channel map, and there is no restriction on this.

[0168] Mode 5: The encoder encodes the important channel identifier in the third bitstream corresponding to the current image block, and the decoder decodes the important channel identifier from the third bitstream corresponding to the current image block, so that the decoder can obtain the important channel identifier.

[0169] For example, the encoding end will C L The feature channel map is divided into an important feature channel map and an unimportant feature channel map, and the important channel identifier is encoded in the third bitstream corresponding to the current image block. After receiving the third bitstream, the decoding end decodes the important channel identifier from the third bitstream and obtains the important channel identifier from C LThe feature channel map corresponding to the important channel identifier is selected as the important feature channel map, and the remaining feature channel maps are selected as unimportant feature channel maps. The probability distribution channel map corresponding to the important feature channel map is selected as the important probability distribution channel map, and the probability distribution channel map corresponding to the unimportant feature channel map is selected as the unimportant probability distribution channel map. L When a feature channel map is divided into important feature channel maps and unimportant feature channel maps, based on the feature variance scale_hat, the important feature channel map and important channel identifier can be determined in the following way:

[0170] LinearToLog is a mapping table that can map the feature variance scale_hat to the logarithmic domain feature variance logarithmic_scale, for example, logarithmic_scale = LinearToLog(scale_hat).

[0171] Slice along the channel dimension to obtain the logarithmic domain feature variance logarithmic_scale_channel corresponding to a certain feature channel.

[0172] Based on logarithmic_scale_channel, calculate the logarithmic variance and sum_per_channel corresponding to each feature channel:

[0173]

[0174] The above process is repeated for a total of C channels of logarithmic_scale. L Times, that is, the logarithmic variance and sum_per_channel corresponding to each feature channel are obtained, and then the channel number corresponding to the maximum sum_per_channel is selected as the important channel identifier important_channel, and the feature channel corresponding to the important channel identifier important_channel is the important feature channel map.

[0175] Example 10: In Example 8, feature adaptive edge enhancement is performed on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain the first reconstructed feature. The feature adaptive edge enhancement process is described below.

[0176] Exemplarily, the feature domain enhancement parameters may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. For example, the encoder encodes multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block, and the decoder decodes the multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds from the third bitstream corresponding to the current image block, and the multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds are used as feature domain enhancement parameters.

[0177] The number of edge enhancement segment strength values ​​and the number of edge enhancement segment thresholds may be the same or different, taking the case where the two are the same as an example. The encoding end may also encode the number of edge enhancement segment strength values ​​(or edge enhancement segment thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of edge enhancement segment strength values ​​from the third bitstream corresponding to the current image block, taking the case where the number of edge enhancement segment strength values ​​is n as an example. The n edge enhancement segment strength values ​​may be recorded as magl-1, magl-2, ..., magl-n, and the n edge enhancement segment thresholds may be recorded as thrl-1, thrl-2, ..., thrl-n.

[0178] Exemplarily, multiple edge enhancement segmentation thresholds may form multiple edge enhancement threshold intervals, and the multiple edge enhancement threshold intervals correspond one-to-one to the multiple edge enhancement segmentation strength values. See Table 1 for an example of the corresponding relationship.

[0179] Table 1

[0180] Edge enhancement threshold range Edge enhancement segment strength value Less than thrl-1 1 (no edge increase) [thrl-1, thrl-2) magl-1 [thrl-2, thrl-3) magl-2 [thrl-3, thrl-4) magl-3 … … [thrl-(n-1),thrl-n) magl-(n-1) Greater than or equal to thrl-n magl-n

[0181] Exemplarily, the important probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the edge enhancement threshold interval corresponding to the probability distribution value is first determined, and the edge enhancement segment strength value corresponding to the probability distribution value is determined based on the edge enhancement threshold interval. For example, if the probability distribution value is within [thrl-2, thrl-3), the edge enhancement segment strength value corresponding to the probability distribution value is magl-2. If the probability distribution value is within [thrl-3, thrl-4), the edge enhancement segment strength value corresponding to the probability distribution value is magl-3, and so on. After obtaining the edge enhancement segment strength value corresponding to each probability distribution value, feature adaptive edge enhancement can be performed on the important feature channel map based on the edge enhancement segment strength value corresponding to each probability distribution value to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0182] For example, the following steps can be used to perform feature adaptive edge enhancement on important feature channel maps:

[0183] Step S11, normalizing the important feature channel map to obtain a normalized feature map.

[0184] Exemplarily, normalization can be performed based on the important feature channel map, the mean feature corresponding to the important feature channel map, and the variance feature corresponding to the important feature channel map to obtain a normalized feature map. For example, the important channel feature map can be subtracted from the mean feature and then divided by the variance feature to obtain a normalized feature map. For another example, the important channel feature map can be subtracted from the mean feature and then divided by the variance feature to obtain an intermediate feature, and then the intermediate feature is converted to obtain a normalized feature map, such as multiplying the intermediate feature by 0.1 and adding 0.5 (0.1 and 0.5 are both examples), and then limiting the feature value to be between 0 and 1. Of course, the above are just a few examples of normalizing the important feature channel map, and there is no limitation on this normalization method.

[0185] Step S12: Generate a high-frequency detail image based on the important feature channel map and the normalized feature map.

[0186] Exemplarily, a convolution operation (such as a two-dimensional convolution operation) can be performed on the normalized feature map and a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no limitation on this convolution kernel) to obtain a Gaussian blur image, and then the Gaussian blur image is subtracted from the important feature channel map to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image, and there is no limitation on this as long as the high-frequency details of the important feature channel map can be obtained.

[0187] Exemplarily, for each value in the Gaussian blur convolution kernel, the denominator of the value may be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc. The Gaussian blur convolution kernel may be an N*N convolution kernel, where N may be a positive integer. If the value of N is 3, that is, the Gaussian blur convolution kernel is a 3*3 convolution kernel, then the Gaussian blur convolution kernel may include but is not limited to:

[0188]

[0189] A may be a positive integer, B may be a positive integer, and M may be a power of 2, such as M may be 2, 4, 8, 16, 32, 64, or 128. For example, A may be 5, B may be 24, and M may be 64.

[0190] Step S13: for each eigenvalue in the high-frequency detail image, edge enhancement is performed on the eigenvalue based on the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an edge enhancement eigenvalue.

[0191] Exemplarily, the important feature channel map includes multiple eigenvalues, the important probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond one-to-one to the multiple eigenvalues. Since the high-frequency detail image includes multiple eigenvalues ​​(corresponding one-to-one to the multiple eigenvalues ​​of the important feature channel map), the multiple probability distribution values ​​of the important probability distribution channel map correspond one-to-one to the multiple eigenvalues ​​of the high-frequency detail image. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the important probability distribution channel map, and the edge enhancement threshold interval corresponding to the probability distribution value can be determined. Based on the edge enhancement threshold interval, the edge enhancement segmented strength value corresponding to the probability distribution value is determined.

[0192] After obtaining the edge enhancement segment strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the edge enhancement segment strength value to obtain an edge enhancement eigenvalue, such as multiplying the eigenvalue by the edge enhancement segment strength value (such as magl-1, magl-2, etc.) to obtain an edge enhancement eigenvalue. Obviously, if the edge enhancement threshold interval is "less than thrl-1", the eigenvalue is multiplied by 1, that is, the eigenvalue is kept unchanged and the eigenvalue is not edge enhanced. If the edge enhancement threshold interval is [thrl-1, thrl-2), the eigenvalue is multiplied by magl-1, magl-1 is a value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other edge enhancement segment strength values ​​such as magl-2 are greater than 1, and edge enhancement can be achieved.

[0193] For each eigenvalue in the high-frequency detail image, the above method can be used to obtain the edge enhancement segment strength value corresponding to the eigenvalue, and then the edge enhancement feature map can be determined based on the edge enhancement eigenvalue corresponding to each eigenvalue. For example, the edge enhancement segment strength values ​​corresponding to all eigenvalues ​​can be combined to obtain an edge enhancement feature map.

[0194] Step S14: Denormalize the edge enhancement feature map to obtain a first reconstruction feature (ie, a first reconstruction feature map). The first reconstruction feature is a reconstruction feature map after feature adaptive edge enhancement is performed on the important feature channel map.

[0195] After obtaining the edge enhancement feature map, the edge enhancement feature map can be directly denormalized to obtain the first reconstructed feature. Alternatively, after obtaining the edge enhancement feature map, the edge enhancement feature map can be added to the normalized feature map to obtain a modified edge enhancement feature map, and the modified edge enhancement feature map can be denormalized to obtain the first reconstructed feature.

[0196] Exemplarily, the edge enhanced normalized feature map can be determined based on the edge enhanced feature map, and the first reconstruction feature can be determined based on the edge enhanced normalized feature map, the mean feature corresponding to the edge enhanced normalized feature map, and the variance feature corresponding to the edge enhanced normalized feature map. The first reconstruction feature can also be called y_hat_sharp.

[0197] For example, the edge enhancement feature map can be transformed to obtain a normalized feature map after edge enhancement. For example, the eigenvalues ​​in the edge enhancement feature map are first limited to between 0 and 1, and then the eigenvalues ​​in the edge enhancement feature map are subtracted by 0.5 and divided by 10 (0.5 and 10 are both examples) to obtain the normalized feature map after edge enhancement.

[0198] The normalized feature map after edge enhancement is multiplied by the variance feature, and then the mean feature is added to obtain the edge-enhanced feature map corresponding to the important feature channel map. The edge-enhanced feature map is recorded as the first reconstructed feature y_hat_sharp.

[0199] Of course, the above is only an example of denormalizing the edge enhancement feature map, and there is no limitation on the denormalization method.

[0200] In a possible implementation manner, the encoder needs to encode multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder may adopt the following method:

[0201] The encoding end can configure multiple candidate feature domain enhancement parameters. For each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter may include multiple edge enhancement segment strength values ​​and multiple edge enhancement segment thresholds. The cost value corresponding to each candidate feature domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block can be selected from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the smallest cost value. The encoding end can encode the feature domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0202] For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature can be enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature. The feature domain enhancement process is described in the above embodiment. The target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstruction feature. The cost value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced. There is no restriction on the method for determining this cost value.

[0203] For example, for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes magl-1, magl-2, ..., magl-n, thrl-1, thrl-2, ..., thrl-n, based on the candidate feature domain enhancement parameter, the enhanced reconstruction feature can be obtained, and then the target reconstructed image block x_hat_enhanced is obtained, and the distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter corresponding to the minimum distortion index value can be selected as the feature domain enhancement parameter corresponding to the current image block, that is, the optimal feature domain enhancement parameter, and the encoding end encodes the feature domain enhancement parameter in the third bitstream.

[0204] Example 11: In Example 8, it is necessary to perform feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map (i.e., the unimportant probability distribution channel map corresponding to the unimportant feature channel map) to obtain the second reconstructed feature after feature adaptive stretching. The feature adaptive stretching process is described below.

[0205] Exemplarily, the feature domain enhancement parameter may include a stretching parameter value. For example, the encoding end may encode the stretching parameter value in the third code stream corresponding to the current image block, and the decoding end may decode the stretching parameter value from the third code stream corresponding to the current image block. The stretching parameter value is used as the feature domain enhancement parameter, and the stretching parameter value may be denoted as ρ.

[0206] Exemplarily, the non-important feature channel map may include multiple eigenvalues, and the non-important probability distribution channel map corresponding to the non-important feature channel map may include multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Based on this, for each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to the eigenvalue can be determined based on the eigenvalue, the stretching parameter value and the probability distribution value corresponding to the eigenvalue. For example, the stretched eigenvalue corresponding to the eigenvalue can be determined by using the following expression. Of course, the following expression is only an example and is not limited to this.

[0207] y_hat_scale=y_hat+ρ*clip3(sigma*y_hat,-0.5,0.5)

[0208] In the above expression, y_hat_scale represents the stretched eigenvalue, y_hat represents the eigenvalue in the unimportant feature channel map, ρ represents the stretching parameter value, sigma represents the probability distribution value in the unimportant probability distribution channel map, and the probability distribution value sigma corresponds to the eigenvalue y_hat, clip3 is a limit operation, which is used to limit sigma*y_hat between -0.5 and 0.5. -0.5 and 0.5 are configurable values ​​and are not restricted. For example, when sigma*y_hat is less than -0.5, sigma*y_hat is limited to -0.5. When sigma*y_hat is greater than 0.5, sigma*y_hat is limited to 0.5. Other values ​​remain unchanged.

[0209] After performing the above processing on each eigenvalue in the non-important feature channel map, the stretched eigenvalue corresponding to each eigenvalue can be obtained, and then the second reconstruction feature (i.e., the second reconstruction feature map) is determined based on the stretched eigenvalue corresponding to each eigenvalue. The second reconstruction feature is a reconstructed feature map after feature adaptive stretching of the non-important feature channel map. For example, the stretched eigenvalues ​​corresponding to all eigenvalues ​​are combined to obtain the second reconstruction feature. The second reconstruction feature is also called y_hat_scale.

[0210] The second reconstructed feature y_hat_scale corresponds to the unimportant feature channel map of the initial reconstructed feature y_hat, that is, the unimportant feature channel map is enhanced to obtain the second reconstructed feature y_hat_scale, the first reconstructed feature y_hat_sharp corresponds to the important feature channel map of the initial reconstructed feature y_hat, that is, the important feature channel map is enhanced to obtain the first reconstructed feature y_hat_sharp, and the second reconstructed feature y_hat_scale and the first reconstructed feature y_hat_sharp are merged to obtain the enhanced reconstructed feature y_hat_enhanced.

[0211] In a possible implementation, the encoding end needs to encode the stretching parameter value in the third code stream corresponding to the current image block. For this process, the encoding end can adopt the following method: the encoding end can configure multiple candidate stretching parameter values, and can determine the cost value corresponding to each candidate stretching parameter value. Based on the cost value corresponding to each candidate stretching parameter value, the stretching parameter value corresponding to the current image block can be selected from all candidate stretching parameter values, that is, the candidate stretching parameter value with the smallest cost value. The encoding end can encode the stretching parameter value to obtain the third code stream corresponding to the current image block.

[0212] For example, for each candidate stretching parameter value, feature adaptive stretching can be performed on the non-important feature channel map based on the candidate stretching parameter value and the non-important probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching, and then an enhanced reconstruction feature is obtained based on the second reconstruction feature, and a target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstruction feature, and a cost value corresponding to the candidate stretching parameter value is determined based on the target reconstructed image block x_hat_enhanced.

[0213] For example, for each candidate stretch parameter value, an enhanced reconstruction feature can be obtained based on the candidate stretch parameter value, and then the target reconstructed image block x_hat_enhanced is obtained, and the distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate stretch parameter value, the encoder can select the candidate stretch parameter value corresponding to the minimum distortion index value as the stretch parameter value corresponding to the current image block, that is, the optimal stretch parameter value, and the encoder can encode the stretch parameter value in the third bitstream.

[0214] In another possible implementation, the encoding end configures multiple candidate feature domain enhancement parameters, and for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds and a stretching parameter value, determines the cost value corresponding to each candidate feature domain enhancement parameter, and based on the cost value corresponding to each candidate feature domain enhancement parameter, selects the feature domain enhancement parameter corresponding to the current image block from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the smallest cost value, and the encoding end encodes the feature domain enhancement parameter (that is, multiple edge enhancement segment strength values, multiple edge enhancement segment thresholds and a stretching parameter value) to obtain a third bitstream corresponding to the current image block. For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature is enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature, determines the target reconstructed image block x_hat_enhanced based on the enhanced reconstructed feature, and determines the cost value corresponding to the candidate feature domain enhancement parameter based on the target reconstructed image block x_hat_enhanced.

[0215] Example 12: In Example 6 and Example 7, the initial reconstructed image block x_hat is input to the filtering enhancement module, and the filtering enhancement module enhances the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. For example, the initial reconstructed image block x_hat includes an initial brightness reconstructed image block x_Y_hat and an initial chrominance reconstructed image block x_UV_hat. The brightness edge enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat to obtain the target brightness reconstructed image block x_Y_hat_enhanced. The chrominance filtering enhancement module performs chrominance filtering enhancement on the initial chrominance reconstructed image block x_UV_hat_enhanced to obtain the target chrominance reconstructed image block x_UV_hat_enhanced. The target brightness reconstructed image block x_Y_hat_enhanced and the target chrominance reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced. The brightness edge enhancement process of the brightness edge enhancement module is described below.

[0216] For example, the brightness edge enhancement module can perform brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat based on the image domain enhancement parameter and probability distribution parameter sigma corresponding to the current image block to obtain the target brightness reconstructed image block x_Y_hat_enhanced corresponding to the current image block. The brightness edge enhancement process is described below.

[0217] Exemplarily, the image domain enhancement parameters may include multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds. For example, the encoding end encodes multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds in the third bitstream corresponding to the current image block, and the decoding end decodes multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds from the third bitstream corresponding to the current image block, and the multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds are used as image domain enhancement parameters.

[0218] The number of image enhancement segmentation strength values ​​and the number of image enhancement segmentation thresholds may be the same or different, taking the case where the two are the same as an example. The encoding end may also encode the number of image enhancement segmentation strength values ​​(or image enhancement segmentation thresholds) in the third bitstream corresponding to the current image block, and the decoding end decodes the number of image enhancement segmentation strength values ​​from the third bitstream corresponding to the current image block. Taking the number of image enhancement segmentation strength values ​​as m as an example, the m image enhancement segmentation strength values ​​are recorded as magy-1, magy-2, ..., magy-m, and the m image enhancement segmentation thresholds are recorded as thry-1, thry-2, ..., thry-m.

[0219] Exemplarily, multiple image enhancement segmentation thresholds may form multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation strength values. See Table 2 for an example of the correspondence.

[0220] Table 2

[0221] Image enhancement threshold range Image enhancement segment intensity value Less than thry-1 1 (no edge increase) [thry-1, thry-2) magy-1 [thry-2, thry-3) magy-2 [thry-3, thry-4) magy-3 … … [thry-(m-1), thry-m) magy-(m-1) Greater than or equal to thry-m magy-m

[0222] Exemplarily, the following steps may be used to perform brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat:

[0223] Step S21, obtaining a target probability distribution channel map based on probability distribution parameters.

[0224] Exemplarily, the initial brightness reconstruction image block x_Y_hat may be a two-dimensional tensor of H×W, where H is the image height of the initial brightness reconstruction image block x_Y_hat, and W is the image width of the initial brightness reconstruction image block x_Y_hat. The probability distribution parameter sigma may be C L xH L xW L The three-dimensional tensor, C L is the number of channels of the probability distribution parameter sigma, H L is the characteristic height of the probability distribution parameter sigma, W L is the characteristic width of the probability distribution parameter sigma. The image height H of the initial brightness reconstruction image block x_Y_hat can be greater than the characteristic height H of the probability distribution parameter sigma. L , such as the image height H is the feature height H L The image width W of the initial brightness reconstruction image block x_Y_hat may be greater than the characteristic width W of the probability distribution parameter sigma. L , such as the image width W is the feature width W L 4 times, 8 times, 16 times, etc.

[0225] In order to perform brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat using probability distribution parameters, it is necessary to obtain a target probability distribution channel map based on the probability distribution parameters. The target probability distribution channel map is a two-dimensional tensor of H×W.

[0226] For example, from C L Select a probability distribution channel graph from the probability distribution channel graphs. Since the probability distribution parameter sigma includes an important probability distribution channel graph (one) and an unimportant probability distribution channel graph (multiple), we can select a probability distribution channel graph from C L Select an important probability distribution channel diagram from the probability distribution channel diagram, or select an important probability distribution channel diagram from C LA non-important probability distribution channel graph is selected from the probability distribution channel graphs, and the following description will be given by taking the selection of an important probability distribution channel graph as an example.

[0227] Then, the important probability distribution channel map is upsampled to obtain a target probability distribution channel map. There is no restriction on the upsampling method, and the size of the target probability distribution channel map can be the same as the size of the initial brightness reconstructed image block x_Y_hat.

[0228] For example, the nearest neighbor upsampling method can be used to upsample the important probability distribution channel map to obtain the target probability distribution channel map. For the nearest neighbor upsampling method, a pixel can be selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding area on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is used as the value of the corresponding pixel on the target high-resolution image. Repeat the above process until all pixels of the target high-resolution image are assigned values.

[0229] In summary, the important probability distribution channel map can be upsampled to the same size as the initial brightness reconstructed image block x_Y_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_channel_upscale.

[0230] Exemplarily, the third bitstream corresponding to the current image block may be decoded to obtain the important channel identifier corresponding to the current image block. Of course, in addition to decoding the important channel identifier from the third bitstream, other methods may be used to obtain the important channel identifier, see the five implementation methods of Example 9. On this basis, if the probability distribution parameter includes multiple probability distribution channel maps, the probability distribution channel map corresponding to the important channel identifier is selected from the multiple probability distribution channel maps as the important probability distribution channel map; then, the important probability distribution channel map is upsampled to obtain the target probability distribution channel map.

[0231] Step S22: If the target probability distribution channel map includes multiple probability distribution values, for each probability distribution value, determine the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value.

[0232] Exemplarily, the target probability distribution channel map may include multiple probability distribution values. For each probability distribution value, the image enhancement threshold interval corresponding to the probability distribution value may be determined first, and the image enhancement segmentation intensity value corresponding to the probability distribution value may be determined based on the image enhancement threshold interval. For example, if the probability distribution value is within [thry-2, thry-3), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-2; if the probability distribution value is within [thry-3, thry-4), the image enhancement segmentation intensity value corresponding to the probability distribution value may be magy-3, and so on.

[0233] Step S23: Perform brightness edge enhancement on the initial brightness reconstructed image block x_Y_hat based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target brightness reconstructed image block x_Y_hat_enhanced corresponding to the current image block.

[0234] First, a high-frequency detail image can be generated based on the initial brightness reconstruction image block x_Y_hat. For example, a convolution operation (such as a two-dimensional convolution operation) can be performed on the initial brightness reconstruction image block x_Y_hat and a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel or a convolution kernel of other sizes, and there is no restriction on this convolution kernel) to obtain a Gaussian blur image, and then the Gaussian blur image is subtracted from the initial brightness reconstruction image block x_Y_hat to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image, as long as the high-frequency details of the initial brightness reconstruction image block x_Y_hat can be obtained.

[0235] Exemplarily, for each value in the Gaussian blur convolution kernel, the denominator of the value may be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc. The Gaussian blur convolution kernel may be an N*N convolution kernel, where N may be a positive integer. If the value of N is 3, that is, the Gaussian blur convolution kernel is a 3*3 convolution kernel, then the Gaussian blur convolution kernel may include but is not limited to:

[0236]

[0237] A may be a positive integer, B may be a positive integer, and M may be a power of 2, such as M may be 2, 4, 8, 16, 32, 64, or 128. For example, A may be 5, B may be 24, and M may be 64.

[0238] Then, for each eigenvalue in the high-frequency detail image, the eigenvalue is edge enhanced based on the image enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain the image enhancement eigenvalue. For example, the initial brightness reconstructed image block x_Y_hat includes multiple eigenvalues, the target probability distribution channel map includes multiple probability distribution values, and the multiple probability distribution values ​​correspond to the multiple eigenvalues ​​one-to-one. Since the high-frequency detail image includes multiple eigenvalues ​​(corresponding one-to-one to the multiple eigenvalues ​​of the initial brightness reconstructed image block x_Y_hat), the multiple probability distribution values ​​of the target probability distribution channel map correspond to the multiple eigenvalues ​​of the high-frequency detail image one-to-one. Based on this, for each eigenvalue in the high-frequency detail image, the probability distribution value corresponding to the eigenvalue can be determined from the target probability distribution channel map, and the image enhancement threshold interval corresponding to the probability distribution value can be determined, and then the image enhancement segmentation intensity value corresponding to the probability distribution value can be determined based on the image enhancement threshold interval.

[0239] After obtaining the image enhancement segment strength value corresponding to the probability distribution value, the eigenvalue can be edge enhanced based on the image enhancement segment strength value to obtain an image enhancement eigenvalue, such as multiplying the eigenvalue by the image enhancement segment strength value (such as magy-1, magy-2, etc.) to obtain an image enhancement eigenvalue. Obviously, if the image enhancement threshold interval is "less than thry-1", the eigenvalue is multiplied by 1, that is, the eigenvalue is kept unchanged and the eigenvalue is not edge enhanced. If the image enhancement threshold interval is [thry-1, thry-2), the eigenvalue is multiplied by magy-1, magy-1 is a value greater than 1, so that the eigenvalue is edge enhanced. Similarly, other image enhancement segment strength values ​​such as magy-2 are greater than 1, and edge enhancement can be achieved.

[0240] Then, the target brightness reconstructed image block x_Y_hat_enhanced is determined based on the image enhancement eigenvalue corresponding to each eigenvalue in the high-frequency detail image. For example, for each eigenvalue in the high-frequency detail image, the image enhancement eigenvalue corresponding to the eigenvalue can be obtained in the above manner, and then the image enhancement feature map can be determined based on the image enhancement eigenvalue corresponding to each eigenvalue. For example, the image enhancement eigenvalues ​​corresponding to all eigenvalues ​​can be combined to obtain an image enhancement feature map. Then, the image enhancement feature map is added to the initial brightness reconstructed image block x_Y_hat to obtain the final edge enhanced image, and then the edge enhanced image is limited to the image value range to obtain the target brightness reconstructed image block x_Y_hat_enhanced.

[0241] In a possible implementation manner, the encoder needs to encode multiple image enhancement segment strength values ​​and multiple image enhancement segment thresholds in the third bitstream corresponding to the current image block. For this process, the encoder may adopt the following method:

[0242] The encoding end can configure multiple candidate image domain enhancement parameters. For each candidate image domain enhancement parameter, the candidate image domain enhancement parameter may include multiple image enhancement segmentation strength values ​​and multiple image enhancement segmentation thresholds. The cost value corresponding to each candidate image domain enhancement parameter can be determined. Based on the cost value corresponding to each candidate image domain enhancement parameter, the image domain enhancement parameter corresponding to the current image block can be selected from all candidate image domain enhancement parameters, that is, the candidate image domain enhancement parameter with the smallest cost value. The encoding end can encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0243] For example, for each candidate image domain enhancement parameter, the initial brightness reconstructed image block x_Y_hat is enhanced in brightness edge based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block x_Y_hat_enhanced corresponding to the current image block. The brightness edge enhancement process is described in the above embodiment and will not be repeated here. Based on the target brightness reconstructed image block x_Y_hat_enhanced, the target reconstructed image block x_hat_enhanced can be obtained, and then the cost value corresponding to the candidate image domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced, and there is no limitation on the determination method.

[0244] For example, for each candidate image domain enhancement parameter, the candidate image domain enhancement parameter includes magy-1, magy-2, ..., magy-m, thry-1, thry-2, ..., thry-m, and the target brightness reconstructed image block x_Y_hat_enhanced can be obtained based on the candidate image domain enhancement parameter, and then the target reconstructed image block x_hat_enhanced is obtained based on the target brightness reconstructed image block x_Y_hat_enhanced, and the distortion index is used to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x. After obtaining the distortion index value corresponding to each candidate image domain enhancement parameter, the candidate image domain enhancement parameter corresponding to the minimum distortion index value can be selected as the image domain enhancement parameter corresponding to the current image block, that is, the optimal image domain enhancement parameter, so that the encoding end can encode the image domain enhancement parameter in the third bitstream.

[0245] Example 13: In Example 6 and Example 7, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process of the chroma filtering enhancement module is described below. For example, after obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module can also perform chroma filtering enhancement based on the initial luminance reconstructed image block x_Y_hat and the initial chroma reconstructed image block x_UV_hat, thereby obtaining the target chroma reconstructed image block x_UV_hat_enhanced corresponding to the current image block x. The chroma filtering enhancement process is described below.

[0246] First, adaptive upsampling is performed based on the initial chroma reconstruction image block x_UV_hat and the initial brightness reconstruction image block x_Y_hat to obtain an upsampled chroma reconstruction image block, and the upsampled chroma reconstruction image block has the same resolution as the initial brightness reconstruction image block x_Y_hat. For example, the initial chroma reconstruction image block x_UV_hat is adaptively upsampled to obtain an upsampled chroma reconstruction image block, and the upsampled chroma reconstruction image block has a required resolution size, such as the resolution of the initial brightness reconstruction image block x_Y_hat, and the adaptive upsampling process is guided by the initial brightness reconstruction image block x_Y_hat.

[0247] Then, cross-channel filtering is performed based on the upsampled chroma reconstructed image block and the initial luminance reconstructed image block x_Y_hat to obtain the cross-channel filtered chroma reconstructed image block. For example, after obtaining the upsampled chroma reconstructed image block, the upsampled chroma reconstructed image block will be subjected to cross-channel filtering to obtain the first enhanced chroma reconstructed image block (i.e., the cross-channel filtered chroma reconstructed image block), and the cross-channel filtering process is guided by the initial luminance reconstructed image block x_Y_hat.

[0248] See also Fig.6D As shown, after wavelet transform is performed on the upsampled chromaticity reconstructed image block and wavelet transform is performed on the initial luminance reconstructed image block x_Y_hat, they are connected in the channel dimension, and the features after channel dimension connection are input into the residual block network, so that the upsampled chromaticity reconstructed image block is enhanced, and finally the chromaticity reconstructed image block after cross-channel filtering is obtained through inverse wavelet transform.

[0249] Then, nonlinear chroma filtering is performed based on the chroma reconstructed image block after cross-channel filtering and the initial luminance reconstructed image block x_Y_hat to obtain the chroma reconstructed image block after nonlinear chroma filtering, and the chroma reconstructed image block after nonlinear chroma filtering is the target chroma reconstructed image block x_UV_hat_enhanced. For example, after obtaining the chroma reconstructed image block after cross-channel filtering, the chroma reconstructed image block after cross-channel filtering will be subjected to nonlinear chroma filtering to obtain the chroma reconstructed image block after the second enhancement, and the nonlinear chroma filtering process is guided by the initial luminance reconstructed image block x_Y_hat. After obtaining the chroma reconstructed image block after the second enhancement, this chroma reconstructed image block is the target chroma reconstructed image block x_UV_hat_enhanced.

[0250] See also Fig. 6E As shown, the initial luminance reconstruction image block x_Y_hat can be downsampled, and then the downsampled luminance reconstruction image block and the cross-channel filtered chrominance reconstruction image block are connected in the channel dimension, wherein the resolution of the downsampled luminance reconstruction image block is the same as the resolution of the cross-channel filtered chrominance reconstruction image block. Then, the channel-dimensional connection feature is input into the nonlinear filter, and then the target chrominance reconstruction image block x_UV_hat_enhanced is output.

[0251] In the above process, cross-channel filtering is performed first and then nonlinear chroma filtering. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, nonlinear chroma filtering may be performed first and then cross-channel filtering, or only cross-channel filtering may be performed (without nonlinear chroma filtering), or only nonlinear chroma filtering may be performed (without cross-channel filtering). There is no restriction on this, as long as x_UV_hat_enhanced can be obtained through filtering.

[0252] Example 14: In Example 6 and Example 7, the chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process of the chroma filtering enhancement module is described below. For example, after obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module can perform chroma filtering enhancement based on the target brightness reconstructed image block x_Y_hat_enhanced and the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced corresponding to the current image block x. The chroma filtering enhancement process is described below.

[0253] First, adaptive upsampling is performed based on the initial chroma reconstructed image block x_UV_hat and the target brightness reconstructed image block x_Y_hat_enhanced to obtain an upsampled chroma reconstructed image block, and the upsampled chroma reconstructed image block has the same resolution as the target brightness reconstructed image block x_Y_hat_enhanced. For example, the initial chroma reconstructed image block x_UV_hat is adaptively upsampled to obtain an upsampled chroma reconstructed image block, the resolution of which is the same as the resolution of the target brightness reconstructed image block x_Y_hat_enhanced, and the adaptive upsampling process is guided by the target brightness reconstructed image block x_Y_hat_enhanced.

[0254] Then, cross-channel filtering is performed based on the upsampled chroma reconstructed image block and the target brightness reconstructed image block x_Y_hat_enhanced to obtain the cross-channel filtered chroma reconstructed image block. For example, after obtaining the upsampled chroma reconstructed image block, the upsampled chroma reconstructed image block will be subjected to cross-channel filtering to obtain the first enhanced chroma reconstructed image block (i.e., the cross-channel filtered chroma reconstructed image block), and the cross-channel filtering process is guided by the target brightness reconstructed image block x_Y_hat_enhanced. For example, after wavelet transforming the upsampled chroma reconstructed image block and wavelet transforming the target brightness reconstructed image block x_Y_hat_enhanced, they are connected in the channel dimension, and the features after the channel dimension connection are input into the residual block network, so that the upsampled chroma reconstructed image block is enhanced, and the cross-channel filtered chroma reconstructed image block is obtained by inverse wavelet transforming.

[0255] Then, nonlinear chroma filtering is performed based on the cross-channel filtered chroma reconstructed image block and the target luminance reconstructed image block x_Y_hat_enhanced to obtain the nonlinear chroma filtered chroma reconstructed image block, and the nonlinear chroma filtered chroma reconstructed image block is the target chroma reconstructed image block x_UV_hat_enhanced. For example, after obtaining the cross-channel filtered chroma reconstructed image block, the cross-channel filtered chroma reconstructed image block will be subjected to nonlinear chroma filtering to obtain the second enhanced chroma reconstructed image block (target chroma reconstructed image block x_UV_hat_enhanced), and the nonlinear chroma filtering process is guided by the target luminance reconstructed image block x_Y_hat_enhanced. For example, the target luminance reconstructed image block x_Y_hat_enhanced is downsampled, and the downsampled luminance reconstructed image block is connected with the cross-channel filtered chroma reconstructed image block in the channel dimension, and the resolution of the downsampled luminance reconstructed image block is the same as the resolution of the cross-channel filtered chroma reconstructed image block. The channel dimension connected features are input into the nonlinear filter, and then the target chromaticity reconstructed image block x_UV_hat_enhanced is output.

[0256] In the above process, cross-channel filtering is performed first and then nonlinear chroma filtering. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, nonlinear chroma filtering may be performed first and then cross-channel filtering, or only cross-channel filtering may be performed (without nonlinear chroma filtering), or only nonlinear chroma filtering may be performed (without cross-channel filtering). There is no restriction on this, as long as x_UV_hat_enhanced can be obtained through filtering.

[0257] Example 15: For Example 5-Example 14, if a feature domain enhancement module is added before the synthetic transformation network, then the feature domain enhancement module can perform feature domain enhancement on the brightness component, can also perform feature domain enhancement on the chrominance component, and can also perform feature domain enhancement on both the brightness component and the chrominance component at the same time. In this embodiment, feature domain enhancement of the brightness component is taken as an example. In order to perform feature domain enhancement on the brightness component, the initial reconstructed feature y_hat corresponding to the brightness component of the current image block can be feature domain enhanced based on the feature domain enhancement parameters and the probability distribution parameters to obtain the enhanced reconstructed feature y_hat_enhanced corresponding to the brightness component. The feature domain enhancement process can be referred to Example 5-Example 14, which will not be repeated here.

[0258] For Example 5 to Example 14, if a filtering enhancement module is added after the synthetic transformation network, then the filtering enhancement module can perform brightness edge enhancement on the brightness component, can also perform chroma filtering enhancement on the chroma component, and can also perform edge enhancement on the brightness component and the chroma component at the same time. In this embodiment, the edge enhancement of the brightness component and the chroma component at the same time is taken as an example. For example, in order to perform brightness edge enhancement on the brightness component, the initial brightness reconstructed image block x_Y_hat corresponding to the brightness component of the current image block can be enhanced based on the image domain enhancement parameters and the probability distribution parameters to obtain the target brightness reconstructed image block x_Y_hat_enhanced. The brightness edge enhancement process can refer to Example 5 to Example 14, which will not be repeated here. In addition, in order to perform chroma filtering enhancement on the chroma component, the initial chroma reconstructed image block x_UV_hat can be enhanced by chroma filtering to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process can refer to Example 5 to Example 14, which will not be repeated here.

[0259] When the initial chroma reconstruction image block x_UV_hat is subjected to chroma filtering enhancement, the target probability distribution channel map of the chroma component can also be obtained based on the probability distribution parameters of the luminance component. For example, an important probability distribution channel map is selected from all probability distribution channel maps of the probability distribution parameters of the luminance component, and the important probability distribution channel map is upsampled to obtain the target probability distribution channel map of the chroma component. The size of the target probability distribution channel map is the same as the size of the initial chroma reconstruction image block x_UV_hat of the chroma component. For example, the important probability distribution channel map is upsampled using the nearest neighbor upsampling method to obtain the target probability distribution channel map of the chroma component. For the nearest neighbor upsampling method, a pixel is selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding area on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is used as the value of the corresponding pixel on the target high-resolution image. Repeat the above process until all pixels of the target high-resolution image are assigned values.

[0260] After obtaining the target probability distribution channel map of the chroma component, for each probability distribution value in the target probability distribution channel map, the image enhancement segment intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value, and the initial chroma reconstructed image block x_UV_hat corresponding to the chroma component is subjected to chroma filtering enhancement based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process can refer to Example 12, and the relevant operations can be performed on the chroma component.

[0261] Embodiment 16: This embodiment proposes a decoding method, which may include the following steps:

[0262] Step S31, decode feature domain enhancement parameters, image domain enhancement parameters and important channel identifiers (i.e., important channel number important_channel) from Bitstream#3 (i.e., the third bitstream, which can also be called feature enhancement header information bitstream) corresponding to the current image block. Among them, the feature domain enhancement parameters may include multiple edge enhancement segment strength values ​​(denoted as magl-1, magl-2, ..., magl-n), multiple edge enhancement segment thresholds (denoted as thrl-1, thrl-2, ..., thrl-n), and stretching parameter value ρ, and the image domain enhancement parameters may include multiple image enhancement segment strength values ​​(denoted as magy-1, magy-2, ..., magy-m), and multiple image enhancement segment thresholds (denoted as thry-1, thry-2, ..., thry-m). The feature domain enhancement parameters may also include the number n of edge enhancement segment strength values, and the image domain enhancement parameters may also include the number m of image enhancement segment strength values.

[0263] Step S32: The input features of the feature domain enhancement module are the initial reconstruction feature y_hat and the probability distribution parameter sigma. The initial reconstruction feature y_hat is a feature with a size of C L xH L xW L The three-dimensional tensor, C L is the number of channels in the feature domain, i.e., the Latent domain, H L is the characteristic height of the Latent domain, W L is the characteristic width of the Latent domain. The probability distribution parameter sigma is the size of C L xH L xW L When the channel of the initial reconstruction feature y_hat is [1, 2, …, C L ], this shape is H L The two-dimensional tensor y_hat_ch of xW is called the feature channel map. When the channel of the probability distribution parameter sigma is [1, 2, ..., C L ], this shape is H L xW L The two-dimensional tensor sigma_ch is called the probability distribution channel map.

[0264] Step S33: Initial reconstruction feature y_hat has C L feature channel graphs, and the probability distribution parameter sigma has C L probability distribution channel diagram, C L The probability distribution channel diagram and C L Feature channels Figure 1One-to-one correspondence, the probability distribution channel map is represented as sigma_ch. The feature channel map where the channel ch is an important channel identifier important_channel is subjected to feature adaptive edge enhancement, and the feature channel map where the channel ch is a non-important channel identifier is subjected to feature adaptive stretching.

[0265] Step S34, for the feature adaptive edge enhancement process, the input data are the important feature channel map y_hat_ch, the important probability distribution channel map sigma_ch, multiple edge enhancement segmentation strength values ​​(magl-1, magl-2, ..., magl-n), and multiple edge enhancement segmentation thresholds (thrl-1, thrl-2, ..., thrl-n). Based on the above input data, the important feature channel map y_hat_ch can be feature-adaptively enhanced to obtain a reconstructed feature map after feature adaptive edge enhancement. The reconstructed feature map is called the first reconstructed feature y_hat_sharp. The feature adaptive edge enhancement process refers to Example 10, which will not be repeated here.

[0266] Step S35: for the feature adaptive stretching process, the input data is the non-important feature channel map (the remaining C L -1 non-important feature channel map), non-important probability distribution channel map (the remaining C L -1 non-important probability distribution channel map), stretching parameter value ρ, based on the above input data, feature adaptive stretching can be performed on each non-important feature channel map to obtain a reconstructed feature map after feature adaptive stretching, and the reconstructed feature map is called the second reconstructed feature y_hat_scale. The feature adaptive stretching process is shown in Example 11 and will not be repeated here. Among them, the size of the non-important feature channel map subjected to feature adaptive stretching is (C L -1)xH L xW L , and the size of the corresponding probability distribution parameter sigma is also (C L -1)xH L xW L Each element of this three-dimensional tensor can be subjected to the following adaptive stretching algorithm to obtain the second reconstructed feature after stretching: y_hat_scale=y_hat+ρ*clip3(sigma*y_hat,-0.5,0.5), where clip3 is a limit operation. The second reconstructed feature after the feature adaptive stretching algorithm is y_hat_scale.

[0267] Step S36, the second reconstruction feature y_hat_scale corresponds to the non-important channel enhancement of the initial reconstruction feature y_hat, the first reconstruction feature y_hat_sharp corresponds to the important channel enhancement of the initial reconstruction feature y_hat, and the first reconstruction feature y_hat_sharp and the second reconstruction feature y_hat_scale are merged to obtain the enhanced reconstruction feature y_hat_enhanced after image enhancement.

[0268] Step S37, input the enhanced reconstruction feature y_hat_enhanced after image enhancement to the synthesis transformation network to obtain the initial reconstructed image block x_hat, the initial reconstructed image block x_hat is a two-dimensional tensor with a size of H×W, and the important probability distribution channel map sigma_channel of the probability distribution parameter sigma is upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is recorded as the target probability distribution channel map sigma_channel_upscale.

[0269] Step S38, for the filtering enhancement process, the input data of the filtering enhancement module are the initial reconstructed image block x_hat, the target probability distribution channel map sigma_channel_upscale, multiple image enhancement segmentation intensity values ​​(magy-1, magy-2, ..., magy-m), and multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). Based on the above input data, the filtering enhancement module can perform filtering enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. For the filtering enhancement process, refer to Example 12-Example 14, which will not be repeated here.

[0270] Embodiment 17: This embodiment proposes a coding method, which may include the following steps:

[0271] Step S41: Based on C L The feature channel map y_hat and C L A probability distribution channel graph sigma is used to determine the important channel identifier corresponding to the important feature channel graph (the important channel identifier is also called the important channel number important_channel).

[0272] For example, slice y_hat and sigma along the channel dimension, and get the current y_hat_channel and sigma_channel after slicing. Calculate the bits_per_ch of each feature channel map through these two tensors, as shown in the following expression, and repeat the above process C LThe bits_per_ch of each feature channel map is obtained, and the channel number corresponding to the maximum bits_per_ch is selected as the important channel identifier.

[0273]

[0274] Step S42, determining the stretching parameter value ρ corresponding to the feature adaptive stretching process. For example, select N1 candidate stretching parameter values ​​ρ, obtain the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate stretching parameter value ρ, use the distortion index to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x, and select the candidate stretching parameter value ρ corresponding to the minimum distortion index value as the optimal stretching parameter value ρ corresponding to the feature adaptive stretching process.

[0275] Step S43, determine the feature domain enhancement parameters corresponding to the feature adaptive edge enhancement process, the feature domain enhancement parameters include multiple edge enhancement segmentation strength values ​​(magl-1, magl-2, ..., magl-n), multiple edge enhancement segmentation thresholds (thrl-1, thrl-2, ..., thrl-n). For example, select N2 candidate feature domain enhancement parameters, obtain the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate feature domain enhancement parameter, use the distortion index to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x, and select the candidate feature domain enhancement parameter corresponding to the minimum distortion index value as the optimal feature domain enhancement parameter corresponding to the feature adaptive edge enhancement process.

[0276] Step S44, determine the image domain enhancement parameters corresponding to the image domain enhancement process, the image domain enhancement parameters may include multiple image enhancement segmentation strength values ​​(magy-1, magy-2, ..., magy-m), multiple image enhancement segmentation thresholds (thry-1, thry-2, ..., thry-m). For example, select N3 candidate image domain enhancement parameters, obtain the enhanced reconstruction features and target reconstructed image block x_hat_enhanced corresponding to each candidate image domain enhancement parameter, use the distortion index to calculate the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x, and select the candidate image domain enhancement parameter corresponding to the minimum distortion index value as the optimal image domain enhancement parameter corresponding to the image domain enhancement process.

[0277] Step S45, encode the important channel identifier corresponding to the important feature channel map, the optimal stretching parameter value ρ, the optimal feature domain enhancement parameter, and the optimal image domain enhancement parameter into the header information bitstream (the third bitstream #Bitstreams3 corresponding to the current image block). It should be noted that the feature domain enhancement module and the image domain enhancement module will not change #Bitstreams1 and #Bitstreams2, but the encoding end and the decoding end will pass through the enhancement module when reconstructing the image, so the reconstructed image will be consistent at the encoding and decoding ends.

[0278] Embodiment 18: The method for image adaptive edge enhancement (i.e., the process in which the filtering enhancement module performs filtering enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced) may be a non-edge enhancement mask edge enhancement algorithm, i.e., a USM edge enhancement algorithm (Unsharp Masking edge enhancement), which includes:

[0279] Step S51, performing a two-dimensional convolution operation on the original reconstructed image and the Gaussian blur convolution kernel to obtain a Gaussian blur image.

[0280] Step S52: subtract the Gaussian blurred image from the original reconstructed image to obtain a high-frequency detail image.

[0281] Step S53: multiply the high-frequency detail image by an edge enhancement coefficient (ie, the image enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue), and add it to the original reconstructed image to obtain a final edge enhanced image.

[0282] Step S54: Limit the edge-enhanced image to a value range of the image.

[0283] Exemplarily, the original reconstructed image is the initial reconstructed image block x_hat. After limiting the edge enhanced image to the value range of the image, the target reconstructed image block x_hat_enhanced can be obtained. The process can refer to the above embodiment.

[0284] Embodiment 19: The method of feature adaptive edge enhancement (i.e., the process in which the feature domain enhancement module performs feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain the first reconstructed feature) can be a non-edge enhancement mask edge enhancement algorithm, i.e., a USM edge enhancement algorithm, which includes:

[0285] Step S61: subtract the mean from the important channel feature map y_hat_channel and divide it by the variance to obtain a normalized feature map.

[0286] Step S62: multiply the normalized feature map by 0.1 and add 0.5, and limit the feature value to between 0 and 1.

[0287] Step S63: Perform a two-dimensional convolution operation on the normalized feature map and the Gaussian blur convolution kernel to obtain a Gaussian blurred image.

[0288] Step S64: subtract the Gaussian blurred image from the original reconstructed image to obtain a high-frequency detail image.

[0289] Step S65: multiply the high-frequency detail image by an edge enhancement coefficient (ie, the edge enhancement segment strength value corresponding to the probability distribution value corresponding to the eigenvalue), and add the coefficient to the normalized feature map to obtain an edge enhancement feature map.

[0290] Step S66: limit the feature value of the edge enhancement feature map to between 0 and 1.

[0291] Step S67: subtract 0.5 from the edge enhancement feature map and divide the map by 10 to obtain a normalized feature map after edge enhancement.

[0292] Step S68, multiply the normalized feature map after edge enhancement by the variance and add the mean to obtain the important channel feature edge enhanced feature map y_hat_shap corresponding to the important channel feature map y_hat_channel, that is, the first reconstructed feature.

[0293] Exemplarily, the original reconstructed image is the important feature channel map y_hat_channel. The process can be referred to Example 10.

[0294] Embodiment 20: During the decoding process, the initial chrominance reconstructed image block x_UV_hat is adaptively upsampled, and the chrominance reconstructed image becomes the required resolution size, and this process is guided by the initial luminance reconstructed image block x_Y_hat. The chrominance reconstructed image after adaptive upsampling will be subjected to cross-channel filtering to obtain the chrominance reconstructed image after the first enhancement, and this process is guided by the initial luminance reconstructed image block x_Y_hat. The chrominance reconstructed image after the first enhancement will be subjected to nonlinear chrominance filtering to obtain the second chrominance image enhancement and output the final chrominance enhanced image x_UV_hat_enhanced. In addition, the initial luminance reconstructed image block x_Y_hat is subjected to luminance edge enhancement to output the target luminance reconstructed image block x_Y_hat_enhanced.

[0295] For the enhancement process of cross-channel filtering, the chromaticity reconstructed image and the initial brightness reconstructed image block x_Y_hat are connected in the channel dimension after wavelet transformation, and then after passing through the residual block network, the chromaticity reconstructed image is enhanced, and finally a new chromaticity reconstructed image is obtained through inverse wavelet transformation. And / or, for the enhancement process of nonlinear chromaticity filtering, after downsampling the initial brightness reconstructed image block x_Y_hat, it becomes the same resolution size as the chromaticity reconstructed image, and then connected in the channel dimension, after passing through a nonlinear filter, the target chromaticity enhanced image x_UV_hat_enhanced can be obtained.

[0296] Embodiment 21: During the decoding process, the filter enhancement submodule can extract important channel identifiers from the bitstream.

[0297] Embodiment 22: During the decoding process, for the Gaussian blur convolution kernel involved in each embodiment, the denominator of each weight in the Gaussian blur convolution kernel is designed to be a power of 2, such as an available weight design as shown below:

[0298]

[0299] Embodiment 23: During the encoding process, the important channel identifier can be determined according to the feature variance scale_hat, and the method is as follows:

[0300] LinearToLog is a mapping table that can map the feature variance scale_hat to the logarithmic domain feature variance logarithmic_scale, for example, logarithmic_scale = LinearToLog(scale_hat).

[0301] Slice along the channel dimension to obtain the logarithmic domain feature variance logarithmic_scale_channel corresponding to a certain feature channel.

[0302] Based on logarithmic_scale_channel, calculate the logarithmic variance and sum_per_channel corresponding to each feature channel:

[0303]

[0304] The above process is repeated for a total of C channels of logarithmic_scale. Ltimes, that is, the logarithmic variance and sum_per_channel corresponding to each feature channel are obtained, and then the channel number corresponding to the maximum sum_per_channel is selected as the important channel identifier important_channel, and the important channel identifier important_channel is also called the important channel number.

[0305] It can be seen from the above technical solutions that in the embodiment of the present application, an end-to-end video image compression method is proposed, which can realize the encoding and decoding of video images based on neural networks, and achieve the purpose of improving the encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. Combined with the network structure design and the header information code stream (such as the third code stream), the neural network can effectively guarantee the quality of the reconstructed image block while maintaining low complexity, so as to achieve the purpose of improving the encoding performance and decoding performance, and reduce the complexity. The image quality of the features is enhanced by using image domain enhancement parameters and probability distribution parameters. The encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream. The decoding end enhances the reconstructed image block through the image domain enhancement parameters, improves the encoding performance, and improves the reconstructed image quality. Channel specificity can be considered, and for the finer-grained adaptive scaling of the probability distribution parameters, edge enhancement, which has a greater improvement on indicators such as VMAF, can be introduced. The characteristics of the bottleneck layer tensor in the end-to-end image coding framework are utilized. The importance of the channel is judged by the bit rate consumed by each channel in the feature domain, and different methods of feature domain enhancement are performed respectively. The probability distribution parameters of important channels in the feature domain are used to adaptively obtain edge enhancement, thereby improving the quality of the encoded and reconstructed image, and the subjective image quality evaluation indicators are improved.

[0306] Illustratively, the above-mentioned embodiments can be implemented individually or in combination. For example, each embodiment in Embodiment 1 to Embodiment 23 can be implemented individually, and at least two embodiments in Embodiment 1 to Embodiment 23 can be implemented in combination.

[0307] Illustratively, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way.

[0308] Based on the same application concept as the above method, a decoding device is also proposed in the embodiment of the present application. The device is applied to the decoding end, and the device includes: a memory, which is configured to store video data; a decoder, which is configured to implement the decoding method in the above embodiments 1 to 23, that is, the processing flow of the decoding end.

[0309] Based on the same application concept as the above method, an encoding device is also proposed in the embodiment of the present application. The device is applied to the encoding end, and the device includes: a memory, which is configured to store video data; an encoder, which is configured to implement the encoding method in the above embodiments 1 to 23, that is, the processing flow of the encoding end.

[0310] Based on the same application concept as the above method, the decoding end device (also referred to as a video decoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig. 7A As shown. It includes: a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 stores machine-executable instructions that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the decoding method of the above-mentioned embodiments 1-23 of the present application.

[0311] For example, in one possible implementation, the processor 711 executes the machine executable instructions to implement:

[0312] Decoding a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block;

[0313] Determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the second bitstream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block;

[0314] Determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0315] Brightness edge enhancement is performed on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block.

[0316] Based on the same application concept as the above method, the encoding end device (also referred to as a video encoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Figure 7B As shown. It includes: a processor 721 and a machine-readable storage medium 722, the machine-readable storage medium 722 stores machine-executable instructions that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the encoding method of the above-mentioned embodiments 1-23 of the present application.

[0317] For example, in one possible implementation, the processor 721 executes the machine executable instructions to implement:

[0318] Encoding the coefficient hyper-parameter features corresponding to the current image block to obtain a first bitstream corresponding to the current image block;

[0319] Determine probability distribution parameters based on the coefficient hyperparameter features, and encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block;

[0320] Determining an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature corresponding to the initial image feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block;

[0321] For each candidate image domain enhancement parameter, based on the candidate image domain enhancement parameter and the probability distribution parameter, an initial brightness reconstructed image block is enhanced in brightness to obtain a target brightness reconstructed image block; and based on the target brightness reconstructed image block, a cost value corresponding to the candidate image domain enhancement parameter is determined;

[0322] An image domain enhancement parameter corresponding to the current image block is selected from all candidate image domain enhancement parameters based on a cost value corresponding to each candidate image domain enhancement parameter, and the image domain enhancement parameter is encoded to obtain a third code stream corresponding to the current image block.

[0323] Based on the same application concept as the above method, an embodiment of the present application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the decoding method or encoding method of the above embodiments 1-23 of the present application.

[0324] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented, such as the decoding method or encoding method in the above embodiments.

[0325] Based on the same application concept as the above method, an embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the decoding method or encoding method disclosed in the above example of the present application.

[0326] Based on the same application concept as the above method, an embodiment of the present application also proposes a decoding device, which can be applied to a decoding end (also called a video decoder), and the decoding device includes: a decoding module, which is used to decode a first code stream corresponding to a current image block to obtain coefficient hyper-parameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyper-parameter features, and decode the second code stream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; a determination module, which is used to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction features; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block; an enhancement module, which is used to perform brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block.

[0327] Exemplarily, the enhancement module is further used to perform chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block; the decoding device also includes: a generation module, used to generate a target reconstructed image block based on the target brightness reconstructed image block and the target chroma reconstructed image block.

[0328] Exemplarily, the enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block, and is specifically used to: decode a third code stream corresponding to a current image block to obtain image domain enhancement parameters corresponding to the current image block; perform brightness edge enhancement on the initial brightness reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters to obtain a target brightness reconstructed image block corresponding to the current image block.

[0329] Exemplarily, the image domain enhancement parameters include multiple image enhancement segmentation intensity values ​​and multiple image enhancement segmentation thresholds, the multiple image enhancement segmentation thresholds constitute multiple image enhancement threshold intervals, and the multiple image enhancement threshold intervals correspond one-to-one to the multiple image enhancement segmentation intensity values; the enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block based on the image domain enhancement parameters and the probability distribution parameters to obtain a target brightness reconstructed image block corresponding to the current image block, and is specifically used to: obtain a target probability distribution channel map based on the probability distribution parameters; if the target probability distribution channel map includes multiple probability distribution values, then for each probability distribution value, determine the image enhancement segmentation intensity value corresponding to the probability distribution value based on the image enhancement threshold interval corresponding to the probability distribution value; perform brightness edge enhancement on the initial brightness reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain a target brightness reconstructed image block.

[0330] Exemplarily, the enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block, and is specifically used to: decode the third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameters include multiple probability distribution channel maps, select the probability distribution channel map corresponding to the important channel identifier from the multiple probability distribution channel maps included in the probability distribution parameters as the important probability distribution channel map; upsample the important probability distribution channel map to obtain the target probability distribution channel map; and perform brightness edge enhancement on the initial brightness reconstructed image block according to the target probability distribution channel map to obtain a target brightness reconstructed image block corresponding to the current image block.

[0331] Exemplarily, the enhancement module performs brightness edge enhancement on the initial brightness reconstructed image block based on the image enhancement segmented intensity value corresponding to each probability distribution value to obtain the target brightness reconstructed image block corresponding to the current image block, and is specifically used to: generate a high-frequency detail image based on the initial brightness reconstructed image block; for each eigenvalue in the high-frequency detail image, perform brightness edge enhancement on the eigenvalue based on the image enhancement segmented intensity value corresponding to the probability distribution value corresponding to the eigenvalue to obtain an image enhancement eigenvalue; and determine the target brightness reconstructed image block based on the image enhancement eigenvalue corresponding to each eigenvalue.

[0332] Exemplarily, when the enhancement module generates a high-frequency detail image based on the initial brightness reconstructed image block, it is specifically used to: perform a convolution operation on the initial brightness reconstructed image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; subtract the Gaussian blur image from the initial brightness reconstructed image block to obtain the high-frequency detail image; wherein, for each value in the Gaussian blur convolution kernel, the denominator of the value is a power of 2.

[0333] Exemplarily, the enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block, and is specifically used to: perform adaptive upsampling based on the initial chroma reconstructed image block and the initial luminance reconstructed image block to obtain an upsampled chroma reconstructed image block, the upsampled chroma reconstructed image block has the same resolution as the initial luminance reconstructed image block; perform a filtering operation on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filtering operation includes cross-channel filtering and / or nonlinear chroma filtering.

[0334] Exemplarily, when the enhancement module performs a filtering operation on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block, it is specifically used to: perform cross-channel filtering on the upsampled chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the target chroma reconstructed image block; or, perform nonlinear chroma filtering on the upsampled chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the target chroma reconstructed image block; or, perform cross-channel filtering on the upsampled chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the cross-channel filtered chroma reconstructed image block, and perform nonlinear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the target chroma reconstructed image block; or, perform nonlinear chroma filtering on the upsampled chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the nonlinear chroma filtered chroma reconstructed image block, and perform cross-channel filtering on the nonlinear chroma filtered chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the target chroma reconstructed image block.

[0335] Exemplarily, the enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block, and is specifically used to: perform adaptive upsampling based on the initial chroma reconstructed image block and the target luminance reconstructed image block to obtain an upsampled chroma reconstructed image block, the upsampled chroma reconstructed image block has the same resolution as the target luminance reconstructed image block; perform a filtering operation on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filtering operation includes cross-channel filtering and / or nonlinear chroma filtering.

[0336] Exemplarily, when the enhancement module performs a filtering operation on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block, it is specifically used to: perform cross-channel filtering on the upsampled chroma reconstructed image block based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, perform nonlinear chroma filtering on the upsampled chroma reconstructed image block based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, perform cross-channel filtering on the upsampled chroma reconstructed image block based on the target brightness reconstructed image block to obtain the cross-channel filtered chroma reconstructed image block, and perform nonlinear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block; or, perform nonlinear chroma filtering on the upsampled chroma reconstructed image block based on the target brightness reconstructed image block to obtain the nonlinear chroma filtered chroma reconstructed image block, and perform cross-channel filtering on the nonlinear chroma filtered chroma reconstructed image block based on the target brightness reconstructed image block to obtain the target chroma reconstructed image block.

[0337] Exemplarily, when the determination module determines the initial reconstructed image block corresponding to the current image block based on the initial reconstruction features, it is specifically used to: input the initial reconstruction features to a synthesis transformation network to obtain the initial reconstructed image block; or, decode a third code stream corresponding to the current image block to obtain feature domain enhancement parameters corresponding to the current image block; perform feature domain enhancement on the initial reconstruction features based on the feature domain enhancement parameters and the probability distribution parameters to obtain enhanced reconstruction features; and input the enhanced reconstruction features to a synthesis transformation network to obtain the initial reconstructed image block.

[0338] Exemplarily, the decoding module is further used to decode the third code stream corresponding to the current image block to obtain the important channel identifier corresponding to the current image block; wherein the initial reconstruction feature includes C feature channel maps, the probability distribution parameters include C probability distribution channel maps, and the C probability distribution channel maps and the C feature channels Figure 1 One by one, the feature channel map corresponding to the important channel identifier is used as the important feature channel map, the remaining feature channel maps are used as the unimportant feature channel maps, the probability distribution channel map corresponding to the important feature channel map is used as the important probability distribution channel map, and the remaining probability distribution channel map is used as the unimportant probability distribution channel map; the determination module performs feature domain enhancement on the initial reconstructed feature based on the feature domain enhancement parameters and the probability distribution parameters, and when obtaining the enhanced reconstructed feature, it is specifically used to: perform feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameters and the important probability distribution channel map to obtain a first reconstructed feature after feature adaptive edge enhancement; perform feature adaptive stretching on the unimportant feature channel map based on the feature domain enhancement parameters and the unimportant probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching; generate the enhanced reconstructed feature based on the first reconstructed feature and the second reconstructed feature.

[0339] Based on the same application concept as the above method, an embodiment of the present application further proposes an encoding device, which is applied to an encoding end (also called a video encoder), and the device includes: an encoding module, which is used to encode the coefficient hyperparameter features corresponding to the current image block to obtain a first code stream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode the initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; a determination module, which is used to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction features corresponding to the initial image features; the initial reconstructed image block includes an initial bright The method comprises the following steps: the first step is to perform brightness edge enhancement on the initial brightness reconstructed image block and the initial chroma reconstructed image block; the second step is to perform brightness edge enhancement on the initial brightness reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter for each candidate image domain enhancement parameter to obtain a target brightness reconstructed image block; the second step is to determine a cost value corresponding to the candidate image domain enhancement parameter based on the target brightness reconstructed image block; the second step is to select an image domain enhancement parameter corresponding to the current image block from all the candidate image domain enhancement parameters based on the cost value corresponding to each candidate image domain enhancement parameter; the third step is to encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0340] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The above is only an embodiment of the present application and is not intended to limit the present application.

[0341] For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A decoding method, characterized in that: Applied to a decoding end, the method comprises: Decoding a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; Determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the second bitstream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; Determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block; Brightness edge enhancement is performed on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block.

2. The method according to claim 1, characterized in that After determining the initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature, the method further includes: Performing chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block; A target reconstructed image block is generated based on the target luminance reconstructed image block and the target chrominance reconstructed image block.

3. The method according to claim 1, characterized in that The step of performing brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block includes: Decoding a third bitstream corresponding to the current image block to obtain an image domain enhancement parameter corresponding to the current image block; Brightness edge enhancement is performed on the initial brightness reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block corresponding to the current image block.

4. The method according to claim 3, characterized in that The image domain enhancement parameters include a plurality of image enhancement segmentation strength values ​​and a plurality of image enhancement segmentation thresholds, wherein the plurality of image enhancement segmentation thresholds constitute a plurality of image enhancement threshold intervals, and the plurality of image enhancement threshold intervals correspond one-to-one to the plurality of image enhancement segmentation strength values; The performing brightness edge enhancement on the initial brightness reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block corresponding to the current image block includes: Acquire a target probability distribution channel map based on the probability distribution parameters; If the target probability distribution channel map includes multiple probability distribution values, then for each probability distribution value, an image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value; The initial brightness reconstructed image block is subjected to brightness edge enhancement based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target brightness reconstructed image block corresponding to the current image block.

5. The method according to claim 1 or 4, characterized in that: The step of performing brightness edge enhancement on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block includes: Decoding a third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; If the probability distribution parameters include a plurality of probability distribution channel graphs, then selecting the probability distribution channel graph corresponding to the important channel identifier as the important probability distribution channel graph from the plurality of probability distribution channel graphs included in the probability distribution parameters; Upsampling the important probability distribution channel map to obtain the target probability distribution channel map; According to the target probability distribution channel map, brightness edge enhancement is performed on the initial brightness reconstructed image block to obtain a target brightness reconstructed image block corresponding to the current image block.

6. The method according to claim 4, characterized in that The step of performing brightness edge enhancement on the initial brightness reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain a target brightness reconstructed image block corresponding to the current image block includes: Generate a high-frequency detail image based on the initial brightness reconstruction image block; For each eigenvalue in the high-frequency detail image, performing brightness edge enhancement on the eigenvalue based on the image enhancement segment intensity value corresponding to the probability distribution value corresponding to the eigenvalue, so as to obtain an image enhancement eigenvalue; The target brightness reconstructed image block is determined based on the image enhancement eigenvalue corresponding to each eigenvalue.

7. The method according to claim 6, characterized in that The generating a high-frequency detail image based on the initial brightness reconstruction image block comprises: Performing a convolution operation on the initial brightness reconstruction image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; Subtract the Gaussian blurred image from the initial brightness reconstructed image block to obtain the high-frequency detail image; Among them, for each value in the Gaussian blur convolution kernel, the denominator of the value is a power of 2.

8. The method according to claim 7, characterized in that The Gaussian blur convolution kernel is an N*N convolution kernel. If the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel includes: Among them, A is a positive integer, B is a positive integer, M is a power of 2, and M is 2, 4, 8, 16, 32, 64, or 128.

9. The method according to claim 2, characterized in that: The step of performing chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block includes: Adaptively upsampling the initial chroma reconstructed image block and the initial luminance reconstructed image block to obtain an upsampled chroma reconstructed image block, wherein the upsampled chroma reconstructed image block has the same resolution as the initial luminance reconstructed image block; A filtering operation is performed on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filtering operation includes cross-channel filtering and / or nonlinear chroma filtering.

10. The method according to claim 9, characterized in that The filtering operation is performed on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block, including: Perform cross-channel filtering on the upsampled chrominance reconstructed image block based on the initial luminance reconstructed image block to obtain the target chrominance reconstructed image block; or, Based on the initial luminance reconstructed image block, nonlinear chrominance filtering is performed on the upsampled chrominance reconstructed image block to obtain the target chrominance reconstructed image block; or, performing cross-channel filtering on the upsampled chroma reconstructed image block based on the initial luminance reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and performing nonlinear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the initial luminance reconstructed image block to obtain the target chroma reconstructed image block; or, Based on the initial luminance reconstructed image block, nonlinear chrominance filtering is performed on the upsampled chrominance reconstructed image block to obtain a nonlinear chrominance filtered chrominance reconstructed image block, and based on the initial luminance reconstructed image block, cross-channel filtering is performed on the nonlinear chrominance filtered chrominance reconstructed image block to obtain the target chrominance reconstructed image block.

11. The method according to claim 2, characterized in that The step of performing chroma filtering enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block includes: Adaptively upsampling the initial chroma reconstructed image block and the target brightness reconstructed image block to obtain an upsampled chroma reconstructed image block, wherein the upsampled chroma reconstructed image block has the same resolution as the target brightness reconstructed image block; A filtering operation is performed on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filtering operation includes cross-channel filtering and / or nonlinear chroma filtering.

12. The method according to claim 11, characterized in that The filtering operation is performed on the upsampled chroma reconstructed image block to obtain the target chroma reconstructed image block, including: Perform cross-channel filtering on the upsampled chrominance reconstructed image block based on the target luminance reconstructed image block to obtain the target chrominance reconstructed image block; or, Based on the target luminance reconstructed image block, nonlinear chrominance filtering is performed on the upsampled chrominance reconstructed image block to obtain the target chrominance reconstructed image block; or, performing cross-channel filtering on the upsampled chroma reconstructed image block based on the target luminance reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and performing nonlinear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the target luminance reconstructed image block to obtain the target chroma reconstructed image block; or, Based on the target brightness reconstructed image block, nonlinear chroma filtering is performed on the upsampled chroma reconstructed image block to obtain a nonlinear chroma filtered chroma reconstructed image block, and based on the target brightness reconstructed image block, cross-channel filtering is performed on the nonlinear chroma filtered chroma reconstructed image block to obtain the target chroma reconstructed image block.

13. The method according to any one of claims 1 to 12, characterized in that: The determining, based on the initial reconstruction feature, an initial reconstructed image block corresponding to the current image block comprises: Inputting the initial reconstruction feature into a synthetic transformation network to obtain the initial reconstructed image block; or, The third code stream corresponding to the current image block is decoded to obtain the feature domain enhancement parameters corresponding to the current image block; the initial reconstructed features are enhanced in the feature domain based on the feature domain enhancement parameters and the probability distribution parameters to obtain enhanced reconstructed features; the enhanced reconstructed features are input into the synthesis transformation network to obtain the initial reconstructed image block.

14. The method according to claim 13, characterized in that The method further includes: decoding a third bitstream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; wherein the initial reconstruction feature includes C feature channel maps, the probability distribution parameters include C probability distribution channel maps, the C probability distribution channel maps correspond to the C feature channel maps one-to-one, the feature channel map corresponding to the important channel identifier is used as an important feature channel map, the remaining feature channel maps are used as non-important feature channel maps, the probability distribution channel map corresponding to the important feature channel map is used as an important probability distribution channel map, and the remaining probability distribution channel map is used as a non-important probability distribution channel map; the feature domain enhancement of the initial reconstruction feature based on the feature domain enhancement parameter and the probability distribution parameter to obtain the enhanced reconstruction feature includes: Performing feature-adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map to obtain a first reconstructed feature after feature-adaptive edge enhancement; Based on the feature domain enhancement parameter and the non-important probability distribution channel map, the non-important feature channel map is subjected to feature adaptive stretching to obtain a second reconstructed feature after feature adaptive stretching; The enhanced reconstruction feature is generated based on the first reconstruction feature and the second reconstruction feature.

15. A coding method, characterized in that: Applied to the encoding end, the method includes: Encoding the coefficient hyper-parameter features corresponding to the current image block to obtain a first bitstream corresponding to the current image block; Determine probability distribution parameters based on the coefficient hyperparameter features, and encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second code stream corresponding to the current image block; Determining an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature corresponding to the initial image feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block; For each candidate image domain enhancement parameter, based on the candidate image domain enhancement parameter and the probability distribution parameter, an initial brightness reconstructed image block is enhanced in brightness to obtain a target brightness reconstructed image block; and based on the target brightness reconstructed image block, a cost value corresponding to the candidate image domain enhancement parameter is determined; An image domain enhancement parameter corresponding to the current image block is selected from all candidate image domain enhancement parameters based on a cost value corresponding to each candidate image domain enhancement parameter, and the image domain enhancement parameter is encoded to obtain a third code stream corresponding to the current image block.

16. A decoding device, characterized in that: Applied to a decoding end, the device comprises: A decoding module, configured to decode a first bitstream corresponding to a current image block to obtain coefficient hyperparameter features corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and decode a second bitstream corresponding to the current image block based on the probability distribution parameters to obtain initial reconstruction features corresponding to the current image block; A determination module, configured to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block; The enhancement module is used to perform brightness edge enhancement on the initial brightness reconstruction image block to obtain a target brightness reconstruction image block.

17. A coding device, characterized in that: Applied to the encoding end, the device includes: An encoding module, configured to encode coefficient hyperparameter features corresponding to a current image block to obtain a first bitstream corresponding to the current image block; determine probability distribution parameters based on the coefficient hyperparameter features, and encode initial image features corresponding to the current image block based on the probability distribution parameters to obtain a second bitstream corresponding to the current image block; A determination module, configured to determine an initial reconstructed image block corresponding to the current image block based on an initial reconstruction feature corresponding to the initial image feature; the initial reconstructed image block includes an initial brightness reconstructed image block and an initial chrominance reconstructed image block; An enhancement module, configured to perform brightness edge enhancement on an initial brightness reconstructed image block based on each candidate image domain enhancement parameter and the probability distribution parameter to obtain a target brightness reconstructed image block; The determination module is further used to determine a cost value corresponding to the candidate image domain enhancement parameter based on the target brightness reconstructed image block; and select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the cost value corresponding to each candidate image domain enhancement parameter; The encoding module is further used to encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

18. A decoding end device, characterized in that: The decoding end device comprises: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement the method according to any one of claims 1 to 14.

19. A coding terminal device, characterized in that: The encoding end device includes: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine executable instructions to implement the method of claim 15 .

20. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by the processor, the method described in any one of claims 1 to 14 is implemented; or, when the computer instructions are executed by the processor, the method described in claim 15 is implemented.

Images