A decoding, encoding method, apparatus and device thereof

Abstract

The application provides a decoding method, an encoding method, a device and equipment thereof. The decoding method comprises: decoding a first code stream corresponding to a current image block to obtain a coefficient hyperparameter feature; determining a probability distribution parameter based on the coefficient hyperparameter feature, decoding a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature; determining an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature, wherein the initial reconstruction image block comprises an initial luminance reconstruction image block and an initial chroma reconstruction image block; and performing luminance edge enhancement on the initial luminance reconstruction image block to obtain a target luminance reconstruction image block. Through the technical scheme, the encoding performance and the decoding performance can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of coding and decoding, in particular to a decoding method and device, an encoding method and device, and equipment thereof. BACKGROUND

[0002] In order to save space, video images are transmitted after being encoded. Complete video encoding can include prediction, transformation, quantization, entropy encoding, filtering and the like. For the prediction process, the prediction process can include intra prediction and inter prediction. Inter prediction refers to using the correlation in the time domain of a video to predict the current pixel using the pixels of the adjacent coded image, so as to effectively remove the temporal redundancy of the video. Intra prediction refers to using the correlation in the spatial domain of a video to predict the current pixel using the pixels of the coded block of the current frame image, so as to remove the spatial redundancy of the video.

[0003] With the rapid development of deep learning, deep learning has achieved success in many high-level computer vision problems, such as image classification, object detection, etc. Deep learning has also gradually begun to be applied in the field of coding and decoding, i.e., a neural network can be used to encode and decode images. Although the coding and decoding method based on the neural network shows great performance potential, the coding and decoding method based on the neural network still has problems such as poor coding performance, poor decoding performance, and high complexity. SUMMARY

[0004] Therefore, the present application provides a decoding method and device, an encoding method and device, and equipment thereof, to improve the coding performance and decoding performance.

[0005] The present application provides a decoding method, applied to a decoding end, the method comprising:

[0006] Decoding a first code stream corresponding to a current image block to obtain a coefficient hyperparameter feature corresponding to the current image block;

[0007] Determining a probability distribution parameter based on the coefficient hyperparameter feature, and decoding a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block;

[0008] Determining an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstruction image block includes an initial luminance reconstruction image block and an initial chrominance reconstruction image block;

[0009] Performing luminance edge enhancement on the initial luminance reconstruction image block to obtain a target luminance reconstruction image block.

[0010] The present application provides an encoding method, applied to an encoding end, the method comprising:

[0011] encode the coefficient hyperparameter feature corresponding to the current image block to obtain a first code stream corresponding to the current image block;

[0012] determine a probability distribution parameter based on the coefficient hyperparameter feature, and encode the initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block;

[0013] determine an initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature; wherein the initial reconstructed image block includes an initial luminance reconstructed image block and an initial chrominance reconstructed image block;

[0014] for each candidate image domain enhancement parameter, perform luminance edge enhancement on the initial luminance reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstructed image block, and determine an objective value corresponding to the candidate image domain enhancement parameter based on the target luminance reconstructed image block;

[0015] select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the objective value corresponding to each candidate image domain enhancement parameter, and encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0016] The application provides a decoding device applied to a decoding end, and the device comprises:

[0017] a decoding module configured to decode a first code stream corresponding to a current image block to obtain a coefficient hyperparameter feature corresponding to the current image block, determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstructed feature corresponding to the current image block;

[0018] a determination module configured to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature; wherein the initial reconstructed image block includes an initial luminance reconstructed image block and an initial chrominance reconstructed image block;

[0019] an enhancement module configured to perform luminance edge enhancement on the initial luminance reconstructed image block to obtain a target luminance reconstructed image block.

[0020] The application provides an encoding device applied to an encoding end, and the device comprises:

[0021] an encoding module configured to encode a coefficient hyperparameter feature corresponding to a current image block to obtain a first code stream corresponding to the current image block, determine a probability distribution parameter based on the coefficient hyperparameter feature, and encode an initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block;

[0022] determining, by a determining module, an initial reconstructed image block corresponding to the current image block based on the initial image feature corresponding initial reconstructed feature; the initial reconstructed image block includes an initial luma reconstructed image block and an initial chroma reconstructed image block;

[0023] enhancing, by an enhancing module, the initial luma reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luma reconstructed image block;

[0024] The determining module is further configured to determine a value of the candidate image domain enhancement parameter based on the target luma reconstructed image block; and select the image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the value of each candidate image domain enhancement parameter.

[0025] The encoding module is further configured to encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0026] The present application provides a decoding end device, the decoding end device comprises a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions which can be executed by the processor;

[0027] The processor is used for executing the machine executable instructions to realize the decoding method.

[0028] The present application provides an encoding end device, the decoding end device comprises a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions which can be executed by the processor;

[0029] The processor is used for executing the machine executable instructions to realize the encoding method.

[0030] The present application provides an electronic device, comprising: a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions which can be executed by the processor; the processor is used for executing the machine executable instructions to realize the decoding method; or, the processor is used for executing the machine executable instructions to realize the encoding method.

[0031] The present application provides a machine readable storage medium, the machine readable storage medium stores a plurality of computer instructions, the computer instructions are executed by the processor to realize the decoding method; or, realize the encoding method.

[0032] As can be seen from the above technical solutions, this application proposes an end-to-end video image compression method that can encode and decode video images based on neural networks. By combining image domain enhancement parameters and probability distribution parameters, it aims to improve encoding and decoding efficiency. By combining network structure design and header information stream (such as a third stream), the neural network effectively ensures the quality of reconstructed image blocks while maintaining low complexity, thereby improving encoding and decoding performance and reducing complexity. Image domain enhancement parameters and probability distribution parameters are used to enhance the image quality of features. The encoding end does not directly change the feature information but encodes the image domain enhancement parameters into the header information stream. The decoding end enhances the reconstructed image blocks using the image domain enhancement parameters, improving encoding performance and reconstructed image quality. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a three-dimensional feature matrix in one embodiment of this application;

[0034] Figure 2 This is a flowchart illustrating a decoding method in one embodiment of this application;

[0035] Figure 3 This is a flowchart illustrating an encoding method in one embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the processing procedure at the encoding end in one embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the processing procedure at the decoding end in one embodiment of this application;

[0038] Figure 6A This is a schematic diagram showing the location of the feature domain enhancement module in one embodiment of this application;

[0039] Figure 6B This is a schematic diagram showing the positions of the feature domain enhancement module and the image domain enhancement module in one embodiment of this application;

[0040] Figure 6C This is a schematic diagram showing the location of the image domain enhancement module in one embodiment of this application;

[0041] Figure 6D This is a schematic diagram of cross-channel filtering in one embodiment of this application;

[0042] Figure 6E This is a schematic diagram of nonlinear chromaticity filtering in one embodiment of this application;

[0043] Figure 7A This is a hardware structure diagram of the decoding end device in one embodiment of this application;

[0044] Figure 7B is a hardware structure diagram of an encoding end device in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The terminology used in the present application merely for the purpose of describing particular embodiments and is not intended to limit the present application. The singular forms "a," "an," and "the" used in the present application and claims are intended to include plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used herein, refer to any or all possible combinations of one or more of the associated listed items. It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal sequence. Rather, these terms are used only as labels to distinguish different sets of information from each other. For example, a first information can be termed a second information, and similarly, a second information can also be termed a first information depending on the context, without departing from the scope of the present application. Furthermore, the word "if' can be interpreted as meaning "when" or "upon" or "in response to determining."

[0046] The present application provides a decoding method and an encoding method, which can involve the following concepts:

[0047] Entropy Encoding: Entropy encoding is an encoding process that does not lose any information according to the entropy principle. The information entropy is the average amount of information of a source (a measure of uncertainty). The encoding method of entropy encoding can include, but is not limited to, Shannon encoding, Huffman encoding and arithmetic coding.

[0048] Neural Network (NN): Neural network refers to artificial neural network. The neural network is an operation model composed of a large number of nodes (or called neurons) connected with each other. In the neural network, the neuron processing unit can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in the neural network can be divided into three categories: input units, output units and hidden units. The input unit accepts the signals and data of the external world; the output unit realizes the output of the processing result; the hidden unit is the unit between the input and output units, which cannot be observed from the outside of the system. The connection weight between neurons reflects the connection strength between units, and the information representation and processing are embodied in the connection relationship of processing units. Neural network is a non-programmed, brain-like information processing method. The essence of neural network is to obtain a parallel distributed information processing function through the transformation and dynamics of neural network, and to imitate the information processing function of the human brain neural system at different levels and levels. In the field of video processing, the commonly used neural network can include but is not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0049] Convolutional Neural Network (CNN): Convolutional neural network is a kind of feedforward neural network, which is one of the most representative network structures in deep learning technology. The artificial neuron of convolutional neural network can respond to a part of the surrounding units in the coverage range, and has excellent performance for large image processing. The basic structure of convolutional neural network includes two layers. One is the feature extraction layer (also called convolution layer), the input of each neuron is connected with the local receptive field of the previous layer, and the local feature is extracted. Once the local feature is extracted, the positional relationship between the local feature and other features is also determined. The second is the feature mapping layer (also called activation layer). Each calculation layer of neural network is composed of multiple feature mappings. Each feature mapping is a plane, and all the weights of the neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of convolutional network. In addition, since the neurons on a mapping plane share weights, the number of free parameters of the network is reduced.

[0050] For example, one of the advantages of the convolutional neural network over the image processing algorithm is that the complex pre-processing process of the image (extracting artificial features, etc.) is avoided, and the original image can be directly inputted for end-to-end learning. One of the advantages of the convolutional neural network over the general neural network is that the general neural network adopts full connection, i.e., all the neurons from the input layer to the hidden layer are connected, which will result in a large number of parameters, and the network training is time-consuming and even difficult to train. The convolutional neural network avoids this difficulty through local connection and weight sharing.

[0051] Deconvolution: The deconvolution layer is also called the transpose convolution layer. The working process of the deconvolution layer is similar to that of the convolution layer, and the main difference is that the deconvolution layer will be padded to make the output larger than the input (of course, it can also be kept the same). If the stride is 1, the output size is equal to the input size. If the stride is N, the width of the output feature is N times the width of the input feature, and the height of the output feature is N times the height of the input feature.

[0052] Generalization ability: The generalization ability can refer to the adaptation ability of the machine learning algorithm to new samples. The purpose of learning is to learn the rule hidden in the data pair. The network trained for the data outside the learning set with the same rule can also give a suitable output. This ability can be called the generalization ability.

[0053] Feature: The feature involved in the present application is a three-dimensional feature matrix or tensor of C*W*H. Referring to FIG. 1, which is a schematic diagram of a three-dimensional feature matrix, in the three-dimensional feature matrix, C represents the number of channels, H represents the feature height, and W represents the feature width. The three-dimensional feature matrix can be the input of the neural network or the output of the neural network. Figure 1

[0054] ​Rate-Distortion Optimized: There are two indicators to evaluate the coding efficiency: code rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. In mode selection, the decision formula is essentially a comprehensive evaluation of the two. For example, the cost of the mode: J (mode) = D + λ * R, where D represents Distortion (distortion), which can usually be measured using the SSE indicator. SSE refers to the sum of the squares of the differences between the reconstructed image block and the source image. In order to achieve cost consideration, the SAD indicator can also be used. SAD is the sum of the absolute values of the differences between the reconstructed image block and the source image. λ is the Lagrange multiplier, and R is the actual number of bits required for image block coding under this mode, including the total number of bits required for coding mode information, motion information, and residual error. In mode selection, if the rate-distortion principle is used to compare and decide the coding mode, the coding performance can usually be guaranteed to be optimal.

[0055] A large number of encoding tools are proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the encoding tool that can obtain the optimal coding performance is often different. Therefore, in the encoding process, RDO (Rate-Distortion Opitimize) is usually used to compare the coding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the marker information in the bit stream. This method, although it brings higher encoding complexity, can adaptively select the optimal mode combination for different content and obtain the optimal coding performance. The decoding end can obtain the relevant mode information by directly parsing the flag information, and the complexity is less affected.

[0056] In the general framework of end-to-end image coding, it mainly includes feature main information part and hyper-prior side information part. The feature main information part includes analysis network, quantization, normal entropy coding, normal entropy decoding and synthesis network. The hyper-prior side information includes hyper-prior analysis network, quantization, factorized entropy coding, factorized entropy decoding and hyper-prior synthesis network. The image component x is compressed and coded by the analysis network and synthesis network of the feature main information part respectively, and is reconstructed and recovered; the hyper-prior side information part is mainly used to model the probability of the feature main information, and guide the entropy coding and decoding of the feature main information. In the general framework of end-to-end image coding, there is a problem that the hyper-prior side information is not fully utilized, and the image reconstruction quality is not fully improved.

[0057] According to the above finding, the embodiment utilizes the characteristics of the end-to-end image coding framework, and the decoding end uses the probability distribution parameters to perform quality enhancement on the reconstructed image, thereby improving the quality of the reconstructed image. The encoding end does not directly change the main information, but encodes the image domain enhancement parameters into the header information code stream (third code stream), and the decoding end performs enhancement on the reconstructed image through the image domain enhancement parameters.

[0058] The decoding method and the encoding method in the embodiments of the present application will be described in detail below in combination with several specific embodiments.

[0059] Embodiment 1: A decoding method is proposed in the embodiments of the present application, as shown in Figure 2 The method can be applied to the decoding end (also referred to as a video decoder), and the method can include the following steps.

[0060] Step 201: Decoding the first code stream corresponding to the current image block to obtain the coefficient hyperparameter feature corresponding to the current image block.

[0061] Step 202: Determining the probability distribution parameters based on the coefficient hyperparameter feature, and decoding the second code stream corresponding to the current image block based on the probability distribution parameters to obtain the initial reconstructed feature corresponding to the current image block.

[0062] Step 203: Determining the initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature, wherein the initial reconstructed image block can include an initial luminance reconstructed image block and an initial chrominance reconstructed image block.

[0063] Step 204: Performing luminance edge enhancement on the initial luminance reconstructed image block to obtain a target luminance reconstructed image block.

[0064] For example, after determining the initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature, the initial chrominance reconstructed image block can also be subjected to chrominance filter enhancement to obtain a target chrominance reconstructed image block. For example, the chrominance filter enhancement and the luminance edge enhancement are synchronous and belong to a parallel relationship, and the chrominance filter enhancement and the luminance edge enhancement can be performed simultaneously, or the luminance edge enhancement can be performed first and then the chrominance filter enhancement, or the chrominance filter enhancement can be performed first and then the luminance edge enhancement, and the order of the luminance and chrominance enhancement is not limited.

[0065] For example, after obtaining the target luminance reconstructed image block and the target chrominance reconstructed image block, the target reconstructed image block can also be generated based on the target luminance reconstructed image block and the target chrominance reconstructed image block.

[0066] Exemplarily, to obtain the target chroma reconstructed image block, chroma filter enhancement can be performed based on the initial luminance reconstructed image block and the initial chroma reconstructed image block to obtain the target chroma reconstructed image block, or chroma filter enhancement can be performed based on the target luminance reconstructed image block and the initial chroma reconstructed image block to obtain the target chroma reconstructed image block.

[0067] Exemplarily, the luminance edge enhancement on the initial luminance reconstructed image block to obtain the target luminance reconstructed image block can include but is not limited to: decoding a third code stream corresponding to the current image block (the third code stream is also referred to as a header information code stream corresponding to the current image block) to obtain an image domain enhancement parameter corresponding to the current image block; and performing luminance edge enhancement on the initial luminance reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter to obtain the target luminance reconstructed image block corresponding to the current image block.

[0068] Exemplarily, the image domain enhancement parameter can include a plurality of image enhancement segment intensity values and a plurality of image enhancement segment threshold values, the plurality of image enhancement segment threshold values form a plurality of image enhancement threshold intervals, and the plurality of image enhancement threshold intervals correspond to the plurality of image enhancement segment intensity values one by one. Based on this, the luminance edge enhancement on the initial luminance reconstructed image block based on the image domain enhancement parameter and the probability distribution parameter to obtain the target luminance reconstructed image block corresponding to the current image block can include but is not limited to: obtaining a target probability distribution channel map based on the probability distribution parameter; if the target probability distribution channel map includes a plurality of probability distribution values, for each probability distribution value, determining an image enhancement segment intensity value corresponding to the probability distribution value based on an image enhancement threshold interval corresponding to the probability distribution value; and performing luminance edge enhancement on the initial luminance reconstructed image block based on the image enhancement segment intensity value corresponding to each probability distribution value to obtain the target luminance reconstructed image block corresponding to the current image block.

[0069] Exemplarily, obtaining the target probability distribution channel map based on the probability distribution parameter can include but is not limited to: decoding the third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameter includes a plurality of probability distribution channel maps, selecting a probability distribution channel map corresponding to the important channel identifier from the plurality of probability distribution channel maps as an important probability distribution channel map; and performing up-sampling on the important probability distribution channel map to obtain the target probability distribution channel map.

[0070] Exemplarily, the luminance edge enhancement on the initial luminance reconstructed image block to obtain the target luminance reconstructed image block can include but is not limited to: decoding the third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameter includes a plurality of probability distribution channel maps, selecting a probability distribution channel map corresponding to the important channel identifier from the plurality of probability distribution channel maps included in the probability distribution parameter as an important probability distribution channel map; upsampling the important probability distribution channel map to obtain a target probability distribution channel map; and performing luminance edge enhancement on the initial luminance reconstructed image block according to the target probability distribution channel map to obtain the target luminance reconstructed image block corresponding to the current image block.

[0071] Exemplarily, the luminance edge enhancement on the initial luminance reconstructed image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain the target luminance reconstructed image block corresponding to the current image block can include but is not limited to: generating a high-frequency detail image based on the initial luminance reconstructed image block; for each feature value in the high-frequency detail image, performing luminance edge enhancement on the feature value based on the image enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the feature value to obtain an image enhancement feature value; and determining the target luminance reconstructed image block based on the image enhancement feature value corresponding to each feature value.

[0072] Exemplarily, the generation of the high-frequency detail image based on the initial luminance reconstructed image block can include but is not limited to: performing convolution operation on the initial luminance reconstructed image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; and subtracting the Gaussian blur image from the initial luminance reconstructed image block to obtain the high-frequency detail image. Wherein, for each value in the Gaussian blur convolution kernel, the denominator of the value can be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc.

[0073] Exemplarily, the Gaussian blur convolution kernel can be an N*N convolution kernel, N can be a positive integer, and if the value of N is 3, i.e. the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel can include but is not limited to:

[0074]

[0075] Wherein, A can be a positive integer, B can be a positive integer, and M can be a power of 2, such as M can be 2, 4, 8, 16, 32, 64, 128. For example, A can be 5, B can be 24, and M can be 64.

[0076] Exemplarily, for step 204, the chroma filter enhancement on the initial chroma reconstructed image block to obtain the target chroma reconstructed image block can include but is not limited to: the initial chroma reconstructed image block can be subjected to chroma filter enhancement based on the target luminance reconstructed image block or the initial luminance reconstructed image block to obtain the target chroma reconstructed image block.

[0077] For example, the adaptive up-sampling can be performed based on the initial chroma reconstructed image block and the initial luma reconstructed image block to obtain an up-sampled chroma reconstructed image block, which can have the same resolution as the initial luma reconstructed image block. A filtering operation can be performed on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block. The filtering operation can include, but is not limited to, cross-channel filtering and / or non-linear chroma filtering. For example, the cross-channel filtering can be performed on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the non-linear chroma filtering can be performed on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the cross-channel filtering can be performed on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and the non-linear chroma filtering can be performed on the cross-channel filtered chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the non-linear chroma filtering can be performed on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain a non-linear chroma filtered chroma reconstructed image block, and the cross-channel filtering can be performed on the non-linear chroma filtered chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block.

[0078] For example, the adaptive up-sampling can be performed based on the initial chroma reconstructed image block and the target luma reconstructed image block to obtain an up-sampled chroma reconstructed image block, which can have the same resolution as the target luma reconstructed image block. A filtering operation can be performed on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block. The filtering operation can include, but is not limited to, cross-channel filtering and / or non-linear chroma filtering. For example, the cross-channel filtering can be performed on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the non-linear chroma filtering can be performed on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the cross-channel filtering can be performed on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and the non-linear chroma filtering can be performed on the cross-channel filtered chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block. Alternatively, the non-linear chroma filtering can be performed on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain a non-linear chroma filtered chroma reconstructed image block, and the cross-channel filtering can be performed on the non-linear chroma filtered chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block.

[0079] For example, determining the initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature can include, but is not limited to: inputting the initial reconstructed feature into a synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block. Alternatively, decoding the third code stream corresponding to the current image block to obtain a feature domain enhancement parameter corresponding to the current image block; performing feature domain enhancement on the initial reconstructed feature based on the feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature; and inputting the enhanced reconstructed feature into the synthesis transformation network to obtain the initial reconstructed image block corresponding to the current image block.

[0080] For example, the third code stream corresponding to the current image block can also be decoded to obtain an important channel identifier corresponding to the current image block; wherein the initial reconstructed feature includes C feature channel maps, the probability distribution parameter includes C probability distribution channel maps, and the C probability distribution channel maps correspond to the C feature channel maps in a one-to-one manner. Figure 1 For example, the important channel identifier corresponds to a feature channel map as an important feature channel map, and the remaining feature channel maps are non-important feature channel maps; the important channel identifier corresponds to a probability distribution channel map as an important probability distribution channel map, and the remaining probability distribution channel maps are non-important probability distribution channel maps.

[0081] For example, performing feature domain enhancement on the initial reconstructed feature based on the feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature can include, but is not limited to: performing feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map to obtain a first reconstructed feature after feature adaptive edge enhancement; performing feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching; and generating the enhanced reconstructed feature based on the first reconstructed feature and the second reconstructed feature.

[0082] For example, the feature domain enhancement parameter can include a plurality of edge enhancement segmentation intensity values and a plurality of edge enhancement segmentation threshold values, the plurality of edge enhancement segmentation threshold values can form a plurality of edge enhancement threshold intervals, and the plurality of edge enhancement threshold intervals correspond to the plurality of edge enhancement segmentation intensity values in a one-to-one manner. Based on this, performing feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map to obtain a first reconstructed feature after feature adaptive edge enhancement can include, but is not limited to: if the important probability distribution channel map includes a plurality of probability distribution values, for each probability distribution value, the edge enhancement segmentation intensity value corresponding to the probability distribution value can be determined based on the edge enhancement threshold interval corresponding to the probability distribution value; and then, the important feature channel map is subjected to feature adaptive edge enhancement based on the edge enhancement segmentation intensity value corresponding to each probability distribution value to obtain the first reconstructed feature after feature adaptive edge enhancement.

[0083] For example, performing feature adaptive edge enhancement on the important feature channel map based on the edge enhancement segment intensity value corresponding to each probability distribution value to obtain the first reconstructed feature after feature adaptive edge enhancement can include but is not limited to: performing normalization on the important feature channel map to obtain a normalized feature map; generating a high-frequency detail image based on the important feature channel map and the normalized feature map; performing edge enhancement on each feature value in the high-frequency detail image based on the edge enhancement segment intensity value corresponding to the probability distribution value corresponding to the feature value to obtain an edge enhanced feature value; determining an edge enhanced feature map based on the edge enhanced feature value corresponding to each feature value; and performing inverse normalization on the edge enhanced feature map to obtain the first reconstructed feature.

[0084] For example, the feature domain enhancement parameter can include a stretching parameter value, and performing feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain the second reconstructed feature after feature adaptive stretching can include but is not limited to: if the non-important feature channel map can include a plurality of feature values, and the non-important probability distribution channel map can include a plurality of probability distribution values, then for each feature value in the non-important feature channel map, a stretched feature value corresponding to the feature value can be determined based on the feature value, the stretching parameter value, and the probability distribution value corresponding to the feature value; and then the second reconstructed feature is determined based on the stretched feature value corresponding to each feature value in the non-important feature channel map.

[0085] For example, the above execution order is only an example given for the convenience of description, and in actual application, the execution order between steps can also be changed, and the execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in the specification, and the steps included in the method can be more or less than described in the specification. In addition, a single step described in the specification can be divided into multiple steps for description in other embodiments; multiple steps described in the specification can also be combined into a single step for description in other embodiments.

[0086] It can be seen from the above technical solutions that, in the embodiments of the present application, after obtaining the initial luminance reconstruction image block and the initial chrominance reconstruction image block corresponding to the current image block, the initial luminance reconstruction image block can be subjected to luminance edge enhancement to obtain a target luminance reconstruction image block, the initial chrominance reconstruction image block can be subjected to chrominance filter enhancement to obtain a target chrominance reconstruction image block, and a target reconstruction image block can be generated based on the target luminance reconstruction image block and the target chrominance reconstruction image block, thereby proposing an end-to-end video image compression method, which can realize encoding and decoding of video images based on a neural network, and can achieve the purpose of improving coding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. In combination with network structure design and header information code stream (such as a third code stream), the neural network can effectively ensure the quality of the reconstruction image block while maintaining low complexity, thereby achieving the purpose of improving coding performance and decoding performance and reducing complexity. The image quality of the features is enhanced by using the image domain enhancement parameters and the probability distribution parameters, the feature information is not directly changed at the encoding end, but the image domain enhancement parameters are encoded into the header information code stream, the reconstruction image block is enhanced by the image domain enhancement parameters at the decoding end, the coding performance is improved, and the reconstruction image quality is improved.

[0087] Embodiment 2: In the embodiments of the present application, an encoding method is proposed, as shown in Figure 3 The method can be applied to an encoding end (also referred to as a video encoder), and the method can include the following steps:

[0088] Step 301: Encode the coefficient hyperparameter feature corresponding to the current image block to obtain a first code stream corresponding to the current image block.

[0089] Step 302: Determine a probability distribution parameter based on the coefficient hyperparameter feature, and encode an initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block.

[0090] Step 303: Determine an initial reconstruction image block corresponding to the current image block based on an initial reconstruction feature corresponding to the initial image feature, wherein the initial reconstruction image block can include an initial luminance reconstruction image block and an initial chrominance reconstruction image block.

[0091] Step 304: For each candidate image domain enhancement parameter, perform luminance edge enhancement on the initial luminance reconstruction image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstruction image block, and determine a value of the candidate image domain enhancement parameter based on the target luminance reconstruction image block. For example, the initial chrominance reconstruction image block can be subjected to chrominance filter enhancement to obtain a target chrominance reconstruction image block, a target reconstruction image block can be generated based on the target luminance reconstruction image block and the target chrominance reconstruction image block, and the value of the candidate image domain enhancement parameter can be determined based on the target reconstruction image block.

[0092] Step 305, selecting the image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the generation cost value corresponding to each candidate image domain enhancement parameter, encoding the image domain enhancement parameter, and obtaining a third code stream corresponding to the current image block (the third code stream can also be referred to as a header information code stream corresponding to the current image block).

[0093] For example, the processing process at the encoding end is similar to the processing process at the decoding end, and the same parts will not be repeated. The processing process at the decoding end can be applied to the encoding end, that is, the same processing mode is used at the encoding end.

[0094] For example, the above execution order is only an example given for convenience of description, and the execution order between steps can be changed in actual application, and the execution order is not limited. Moreover, in other embodiments, the steps of the method can be more or less than described in the specification. In addition, a single step described in the specification can be divided into multiple steps in other embodiments; multiple steps described in the specification can be combined into a single step in other embodiments.

[0095] From the above technical solutions, in the embodiments of the present application, after obtaining the initial luminance reconstructed image block and the initial chrominance reconstructed image block corresponding to the current image block, the initial luminance reconstructed image block can be subjected to luminance edge enhancement to obtain a target luminance reconstructed image block, the initial chrominance reconstructed image block can be subjected to chrominance filter enhancement to obtain a target chrominance reconstructed image block, and the target reconstructed image block can be generated based on the target luminance reconstructed image block and the target chrominance reconstructed image block, thereby proposing an end-to-end video image compression method, which can realize encoding and decoding of video images based on a neural network, and achieve the purpose of improving encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. By combining network structure design and header information code stream (such as the third code stream), the neural network can effectively ensure the quality of the reconstructed image block while maintaining low complexity, achieve the purpose of improving encoding performance and decoding performance, and reduce complexity. The image quality of the features is enhanced by using the image domain enhancement parameters and the probability distribution parameters, the encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream, and the decoding end enhances the reconstructed image block by using the image domain enhancement parameters, improves the encoding performance, and improves the reconstructed image quality.

[0096] Embodiment 3: For embodiments 1 and 2, the processing process at the encoding end can be seen from Figure 4 of course, Figure 4 is only an example of the processing process at the encoding end, and the processing process at the encoding end is not limited.

[0097] After obtaining the current image block x (the current image block x can be the original image block x, i.e., the input image block), the encoding end can analyze and transform the current image block x through an analysis transformation network (i.e., a neural network) to obtain the image feature y corresponding to the current image block x. Wherein, the feature transformation of the current image block x through the analysis transformation network refers to: transforming the current image block x into the image feature y in the latent domain, so as to facilitate the operation of all subsequent processes in the latent domain.

[0098] Wherein, the image can be divided into 1 image block, or can be divided into multiple image blocks, if the image is divided into 1 image block, the current image block x can also be the image, that is, the encoding and decoding process of the image block can also be directly used for the image.

[0099] After obtaining the image feature y, the encoding end performs coefficient hyperparameter feature transformation on the image feature y to obtain the coefficient hyperparameter feature z. For example, the image feature y can be input to a hyperparameter encoding network (i.e., a neural network) to perform coefficient hyperparameter feature transformation on the image feature y to obtain the coefficient hyperparameter feature z. Wherein, the hyperparameter encoding network can be a trained neural network, and the training process of the hyperparameter encoding network is not limited, as long as it can perform coefficient hyperparameter feature transformation on the image feature y. Wherein, the image feature y in the latent domain obtains the hyper-prior latent information z after passing through the hyperparameter encoding network.

[0100] After obtaining the coefficient hyperparameter feature z, the encoding end can quantize the coefficient hyperparameter feature z to obtain the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, i.e. Figure 4 The Q operation in the above formula is a quantization process. After obtaining the hyperparameter quantization feature corresponding to the coefficient hyperparameter feature z, the hyperparameter quantization feature is encoded to obtain the Bitstream#1 (i.e., the first code stream) corresponding to the current image block, i.e. Figure 4 The AE operation in the above formula represents an encoding process, such as an entropy encoding process. Alternatively, the encoding end can also directly encode the coefficient hyperparameter feature z to obtain the Bitstream#1 corresponding to the current image block. Wherein, the hyperparameter quantization feature or the coefficient hyperparameter feature z carried in the Bitstream#1 is mainly used to obtain the parameters of the mean and the probability distribution model.

[0101] After obtaining the Bitstream#1 corresponding to the current image block, the encoding end can send the Bitstream#1 corresponding to the current image block to the decoding end. For the processing process of the decoding end for the Bitstream#1 corresponding to the current image block, see the subsequent embodiments.

[0102] The encoding end can also decode Bitstream#1 to obtain the hyperparameter quantization feature after obtaining Bitstream#1 corresponding to the current image block, that is, Figure 4 AD in the formula (3) represents a decoding process, and then the hyperparameter quantization feature is dequantized to obtain the coefficient hyperparameter feature z_hat, which can be the same as or different from the coefficient hyperparameter feature z, Figure 4 The IQ operation in the formula (4) is a dequantization process. Alternatively, the encoding end can also decode Bitstream#1 to obtain the coefficient hyperparameter feature z_hat after obtaining Bitstream#1 corresponding to the current image block, without involving the dequantization process of the coefficient hyperparameter feature z_hat.

[0103] For the encoding process of Bitstream#1, a fixed probability density model encoding method can be used, and for the decoding process of Bitstream#1, a fixed probability density model decoding method can be used, and the encoding and decoding processes are not limited.

[0104] After the encoding end obtains the coefficient hyperparameter feature z_hat, context-based prediction can be performed based on the coefficient hyperparameter feature z_hat of the current image block and the reconstructed feature y_hat of the previous image block (the determination process of the reconstructed feature y_hat is described in subsequent embodiments) to obtain the prediction value mu (i.e., the mean mu) corresponding to the current image block. For example, the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat are input into the mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyperparameter feature z_hat and the reconstructed feature y_hat. The prediction process is not limited. For the context-based prediction process, the input includes the coefficient hyperparameter feature z_hat and the decoded reconstructed feature y_hat, which are jointly input to obtain a more accurate prediction value mu, and the prediction value mu is used to obtain the residual r_hat by subtracting the original feature and to obtain the reconstructed feature y_hat by adding the decoded residual.

[0105] It should be noted that the mean prediction network is an optional neural network, that is, there can be no mean prediction network, that is, the prediction value mu does not need to be determined by the mean prediction network, Figure 4 The dashed box in the formula (5) indicates that the mean prediction network is optional.

[0106] After obtaining the image feature y, the encoding end can determine the residual feature r based on the image feature y and the prediction value mu, such as taking the difference between the image feature y and the prediction value mu as the residual feature r. Then, the residual feature r is processed to obtain the image feature s, and the feature processing process is not limited and can be any feature processing mode. In this case, a mean prediction network needs to be deployed to provide the prediction value mu. Alternatively, after obtaining the image feature y, the encoding end can process the image feature y to obtain the image feature s, and the feature processing process is not limited and can be any feature processing mode. In this case, the mean prediction network does not need to be deployed, and the residual process is optional, as indicated by the dashed box.

[0107] After obtaining the image feature s, the encoding end can quantize the image feature s to obtain the image quantization feature corresponding to the image feature s, that is, Figure 4 The Q operation in the above formula is a quantization process. After obtaining the image quantization feature corresponding to the image feature s, the encoding end can encode the image quantization feature to obtain the Bitstream#2 (i.e., the second code stream) corresponding to the current image block, that is, Figure 4 The AE operation in the above formula represents an encoding process, such as an entropy encoding process. Alternatively, the encoding end can also directly encode the image feature s to obtain the Bitstream#2 corresponding to the current image block, without involving the quantization process of the image feature s.

[0108] After obtaining the Bitstream#2 corresponding to the current image block, the encoding end can send the Bitstream#2 corresponding to the current image block to the decoding end. For the processing process of the decoding end for the Bitstream#2 corresponding to the current image block, see the subsequent embodiments.

[0109] After obtaining the Bitstream#2 corresponding to the current image block, the encoding end can also decode the Bitstream#2 to obtain the image quantization feature, that is, Figure 4 The AD in the above formula represents a decoding process, and then the encoding end can dequantize the image quantization feature to obtain the image feature s', which can be the same as or different from the image feature s, Figure 4 The IQ operation in the above formula is a dequantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the encoding end can also decode the Bitstream#2 to obtain the image feature s', without involving the dequantization process of the image quantization feature.

[0110] After obtaining image features s', the encoder can perform feature recovery (the inverse process of feature processing) on ​​s'. This feature recovery process is not restricted and can be any method, resulting in residual features r_hat. Residual features r_hat and r can be the same or different. After obtaining residual features r_hat, the encoder determines image features y_hat (i.e., reconstructed features) based on residual features r_hat and predicted values ​​mu. Image features y_hat and y can be the same or different; for example, the sum of residual features r_hat and predicted values ​​mu can be used as image features y_hat. In this case, a mean prediction network needs to be deployed to provide the predicted values ​​mu. Alternatively, after obtaining image features s', the encoder can perform feature recovery (the inverse process of feature processing) on ​​s' to obtain image features y_hat. Image features y_hat and y can be the same or different. In this case, a mean prediction network is not needed, and the residual process is indicated by a dashed box as optional.

[0111] After obtaining the image feature y_hat, the encoder can perform a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current image block x. For example, the image feature y_hat can be input into the synthetic transformation network, which will perform a synthetic transformation on the image feature y_hat to obtain the reconstructed image block x_hat. Thus, the image reconstruction process is completed.

[0112] In one possible implementation, when the encoding end encodes the image quantization features or image features s to obtain Bitstream#2 corresponding to the current image block, the encoding end needs to first determine the probability distribution model, and then encode the image quantization features or image features s based on the probability distribution model. Furthermore, when the encoding end decodes Bitstream#2, it also needs to first determine the probability distribution model, and then decode Bitstream#2 based on the probability distribution model.

[0113] To obtain the probability distribution model, please refer to [link / reference]. Figure 4 As shown, after obtaining the hyperparameter feature z_hat, the encoder can perform an inverse hyperparameter feature transformation on z_hat to obtain the probability distribution parameters. For example, the hyperparameter feature z_hat can be input into a probabilistic hyperparameter decoding network, which will then perform an inverse hyperparameter feature transformation on z_hat to obtain the probability distribution parameter sigma. After obtaining the probability distribution parameter, a probability distribution model can be generated based on it. The probabilistic hyperparameter decoding network can be a trained neural network; the training process of this network is not restricted, as long as it can perform the inverse hyperparameter feature transformation on z_hat.

[0114] In a possible implementation, the encoding end processing process described above can be performed by a deep learning model or a neural network model, so as to realize an end-to-end image compression and encoding process, and the encoding process is not limited.

[0115] For the decoding end processing process of embodiments 1 and 2, please refer to Figure 5 As shown in the figure, of course, Figure 5 which is only an example of the decoding end processing process, and the decoding end processing process is not limited.

[0116] After obtaining the Bitstream#1 corresponding to the current image block, the decoding end can also decode the Bitstream#1 to obtain the hyper-quantized feature, that is, Figure 5 AD in the figure represents the decoding process, and then the hyper-quantized feature is dequantized to obtain the coefficient hyper-parameter feature z_hat, which can be the same as or different from the coefficient hyper-parameter feature z, Figure 5 the IQ operation in the figure is the dequantization process. Alternatively, after obtaining the Bitstream#1 corresponding to the current image block, the decoding end can also decode the Bitstream#1 to obtain the coefficient hyper-parameter feature z_hat, without involving the dequantization process of the coefficient hyper-parameter feature z_hat.

[0117] For the decoding process of the Bitstream#1, a decoding method of a fixed probability density model can be used, and the decoding method is not limited.

[0118] The image can be divided into one image block or multiple image blocks. If the image is divided into one image block, the current image block x can also be the image, that is, the decoding process of the image block can also be directly used for the image.

[0119] After obtaining the coefficient hyper-parameter feature z_hat, the decoding end can perform context-based prediction based on the coefficient hyper-parameter feature z_hat of the current image block and the reconstructed feature y_hat of the previous image block (the determination process of the reconstructed feature y_hat is described in subsequent embodiments) to obtain the prediction value mu (i.e., the mean mu) corresponding to the current image block. For example, the coefficient hyper-parameter feature z_hat and the reconstructed feature y_hat are input into a mean prediction network, and the mean prediction network determines the prediction value mu based on the coefficient hyper-parameter feature z_hat and the reconstructed feature y_hat, and the prediction process is not limited. For the context-based prediction process, the input includes the coefficient hyper-parameter feature z_hat and the decoded reconstructed feature y_hat, and the two are jointly input to obtain a more accurate prediction value mu.

[0120] It should be noted that the mean prediction network is an optional neural network, i.e., there can be no mean prediction network, i.e., the predicted value mu does not need to be determined by the mean prediction network, Figure 5 The dashed box in indicates that the mean prediction network is optional.

[0121] After obtaining the Bitstream#2 corresponding to the current image block, the decoding end can also decode the Bitstream#2 to obtain the image quantized feature, i.e., Figure 5 AD in indicates the decoding process, and then the decoding end can dequantize the image quantized feature to obtain the image feature s', which can be the same as or different from the image feature s, Figure 5 The IQ operation in is a dequantization process. Alternatively, after obtaining the Bitstream#2 corresponding to the current image block, the decoding end can also decode the Bitstream#2 to obtain the image feature s' without involving the dequantization process of the image quantized feature.

[0122] After obtaining the image feature s', the decoding end can perform feature restoration (i.e., the inverse process of feature processing) on the image feature s' to obtain the residual feature r_hat, which can be the same as or different from the residual feature r. After obtaining the residual feature r_hat, the decoding end determines the image feature y_hat (i.e., the reconstructed feature) based on the residual feature r_hat and the predicted value mu, which can be the same as or different from the image feature y, such as taking the sum of the residual feature r_hat and the predicted value mu as the image feature y_hat. In this case, the mean prediction network needs to be deployed to provide the predicted value mu. Alternatively, after obtaining the image feature s', the decoding end performs feature restoration on the image feature s' to obtain the image feature y_hat, which can be the same as or different from the image feature y. In this case, the mean prediction network does not need to be deployed, and the residual process is indicated by the dashed box as an optional process.

[0123] After obtaining the image feature y_hat, the decoding end can perform synthesis transformation on the image feature y_hat to obtain the reconstructed image block x_hat corresponding to the current image block x, such as inputting the image feature y_hat to the synthesis transformation network to perform synthesis transformation on the image feature y_hat by the synthesis transformation network to obtain the reconstructed image block x_hat, thereby completing the image reconstruction process.

[0124] In one possible implementation, when decoding the Bitstream#2, the decoding end needs to first determine the probability distribution model, and then decode the Bitstream#2 based on the probability distribution model. In order to obtain the probability distribution model, continue to refer to Figure 5As shown, after obtaining the coefficient hyperparameter feature z_hat, the decoding end can perform coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter sigma. For example, the coefficient hyperparameter feature z_hat is input into the probability hyperparameter decoding network, the coefficient hyperparameter feature z_hat is subjected to coefficient hyperparameter feature inverse transformation by the probability hyperparameter decoding network to obtain the probability distribution parameter, and after obtaining the probability distribution parameter, the probability distribution model can be generated based on the probability distribution parameter. The probability hyperparameter decoding network can be a trained neural network, and the training process of the probability hyperparameter decoding network is not limited. The probability hyperparameter decoding network can only perform coefficient hyperparameter feature inverse transformation on the coefficient hyperparameter feature z_hat to obtain the probability distribution parameter.

[0125] In a possible implementation, the processing process of the decoding end described above can be performed by a deep learning model or a neural network model, so as to realize an end-to-end image compression and encoding process, and the decoding process is not limited.

[0126] Embodiment 5: On the basis of Embodiment 3 and Embodiment 4, a feature domain enhancement module can be added before the synthesis transformation network. See Figure 6A As shown, the input feature of the feature domain enhancement module can be the image feature y_hat (hereinafter referred to as the initial reconstruction feature y_hat), and the output feature of the feature domain enhancement module can be the enhanced reconstruction feature y_hat_enhanced. The enhanced reconstruction feature y_hat_enhanced is input into the synthesis transformation network, and the synthesis transformation network is subjected to synthesis transformation on the enhanced reconstruction feature y_hat_enhanced to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0127] For the encoding end: after obtaining the current image block x, the current image block x is subjected to analysis transformation by the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation by the hyperparameter encoding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature (which can be the coefficient hyperparameter feature z itself or the hyperparameter quantization feature of the coefficient hyperparameter feature z) corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block.

[0128] The first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat (for example, the coefficient hyperparameter feature z_hat itself is decoded from the first code stream, or the hyperparameter quantization feature is decoded from the first code stream, and the coefficient hyperparameter feature z_hat is obtained by inverse quantization on the hyperparameter quantization feature). Then, the coefficient hyperparameter feature z_hat is subjected to coefficient hyperparameter feature inverse transformation by the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0129] encode the initial image feature corresponding to the current image block based on the probability distribution parameter sigma, to obtain a second code stream corresponding to the current image block.

[0130] decode the second code stream corresponding to the current image block based on the probability distribution parameter sigma, to obtain an initial reconstructed feature y_hat corresponding to the current image block. For example, if the initial image feature is the image feature y, the initial reconstructed feature y_hat is decoded from the second code stream. If the initial image feature is the image quantized feature corresponding to the image feature y, the image quantized feature is decoded from the second code stream, and the initial reconstructed feature y_hat is obtained by dequantizing the image quantized feature. For another example, if the initial image feature is the residual feature r corresponding to the image feature y, the residual feature r_hat is decoded from the second code stream, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the prediction value mu. If the initial image feature is the image quantized feature corresponding to the residual feature r, the image quantized feature is decoded from the second code stream, and the residual feature r_hat is obtained by dequantizing the image quantized feature, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the prediction value mu. If the initial image feature is the image feature s corresponding to the image feature y or the residual feature r, the image feature s' is decoded from the second code stream, and the initial reconstructed feature y_hat or the residual feature r_hat is obtained by feature restoration on the image feature s', and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the prediction value mu. If the initial image feature is the image quantized feature corresponding to the image feature s, the image quantized feature is decoded from the second code stream, and the image feature s' is obtained by dequantizing the image quantized feature, and the initial reconstructed feature y_hat or the residual feature r_hat is obtained by feature restoration on the image feature s', and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the prediction value mu.

[0131] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input to the feature domain enhancement module, the feature domain enhancement module performs feature domain enhancement on the initial reconstructed feature y_hat to obtain an enhanced reconstructed feature y_hat_enhanced, and the enhanced reconstructed feature y_hat_enhanced is input to the synthesis transformation network, and the synthesis transformation network performs synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0132] For the decoding end: the first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block (such as decoding the coefficient hyperparameter feature z_hat itself from the first code stream, or decoding the hyperparameter quantization feature from the first code stream, and performing inverse quantization on the hyperparameter quantization feature to obtain the coefficient hyperparameter feature z_hat). Then, the coefficient hyperparameter feature z_hat is inversely transformed by the probability hyperparameter decoding network to obtain the probability distribution parameter sigma.

[0133] The second code stream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstructed feature y_hat corresponding to the current image block. For example, the initial reconstructed feature y_hat is decoded from the second code stream. Or, the image quantization feature is decoded from the second code stream, and the initial reconstructed feature y_hat is obtained by inverse quantization on the image quantization feature. For another example, the residual feature r_hat is decoded from the second code stream, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the prediction value mu. Or, the image quantization feature is decoded from the second code stream, and the residual feature r_hat is obtained by inverse quantization on the image quantization feature, and the initial reconstructed feature y_hat is determined based on the residual feature r_hat and the prediction value mu. For another example, the image feature s' is decoded from the second code stream, the initial reconstructed feature y_hat or the residual feature r_hat is obtained by feature recovery on the image feature s', and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the prediction value mu. Or, the image quantization feature is decoded from the second code stream, and the image feature s' is obtained by inverse quantization on the image quantization feature, the initial reconstructed feature y_hat or the residual feature r_hat is obtained by feature recovery on the image feature s', and if the residual feature r_hat is obtained, the initial reconstructed feature y_hat can also be determined based on the residual feature r_hat and the prediction value mu.

[0134] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat can be input into the feature domain enhancement module, which performs feature domain enhancement on the initial reconstructed feature y_hat to obtain the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced is then input into the synthesis transformation network, which performs a synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0135] Example 6: Based on Examples 3 and 4, a feature domain enhancement module can be added before the synthesis transform network, and a filtering enhancement module can be added after the synthesis transform network. This filtering enhancement module may include a luminance edge enhancement module and a chrominance filtering enhancement module. See also Figure 6B As shown, the input feature of the feature domain enhancement module can be the initial reconstructed feature y_hat, and the output feature of the feature domain enhancement module is the enhanced reconstructed feature y_hat_enhanced. The enhanced reconstructed feature y_hat_enhanced can be input to the synthesis transformation network, which performs a synthesis transformation on the enhanced reconstructed feature y_hat_enhanced to obtain the initial reconstructed image block x_hat corresponding to the current image block x. The input feature of the filtering enhancement module can be the initial reconstructed image block x_hat, and the output feature of the filtering enhancement module can be the target reconstructed image block x_hat_enhanced. For example, the initial reconstructed image block x_hat includes the initial luminance reconstructed image block x_Y_hat and the initial chrominance reconstructed image block x_UV_hat. The luminance edge enhancement module performs luminance edge enhancement on the initial luminance reconstructed image block x_Y_hat to obtain the target luminance reconstructed image block x_Y_hat_enhanced. The chrominance filtering enhancement module performs chrominance filtering enhancement on the initial chrominance reconstructed image block x_UV_hat to obtain the target chrominance reconstructed image block x_UV_hat_enhanced. Then, the target brightness reconstructed image block x_Y_hat_enhanced and the target chromaticity reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced.

[0136] For the encoding end: the current image block x is analyzed and transformed by the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation by the hyperparameter encoding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The coefficient hyperparameter feature z_hat corresponding to the current image block is obtained by decoding the first code stream corresponding to the current image block. The coefficient hyperparameter feature z_hat is subjected to coefficient hyperparameter feature inverse transformation by the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. The initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma to obtain the second code stream corresponding to the current image block. The initial reconstruction feature y_hat corresponding to the current image block is obtained by decoding the second code stream corresponding to the current image block based on the probability distribution parameter sigma. The above process can be referred to in Embodiment 5 and will not be repeated here.

[0137] After obtaining the initial reconstruction feature y_hat, the initial reconstruction feature y_hat can be input to the feature domain enhancement module, the initial reconstruction feature y_hat is subjected to feature domain enhancement by the feature domain enhancement module to obtain the enhanced reconstruction feature y_hat_enhanced, and the enhanced reconstruction feature y_hat_enhanced is input to the synthesis transformation network, the enhanced reconstruction feature y_hat_enhanced is subjected to synthesis transformation by the synthesis transformation network to obtain the initial reconstruction image block x_hat corresponding to the current image block x, and the initial reconstruction image block x_hat includes the initial luminance reconstruction image block x_Y_hat and the initial chrominance reconstruction image block x_UV_hat.

[0138] After obtaining the initial luminance reconstruction image block x_Y_hat corresponding to the current image block x, the initial luminance reconstruction image block x_Y_hat is subjected to luminance edge enhancement by the luminance edge enhancement module to obtain the target luminance reconstruction image block x_Y_hat_enhanced. After obtaining the initial chrominance reconstruction image block x_UV_hat corresponding to the current image block x, the initial chrominance reconstruction image block x_UV_hat is subjected to chrominance filter enhancement by the chrominance filter enhancement module to obtain the target chrominance reconstruction image block x_UV_hat_enhanced. Then, the target luminance reconstruction image block x_Y_hat_enhanced and the target chrominance reconstruction image block x_UV_hat_enhanced are combined to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0139] For the decoding end, the first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, the coefficient hyperparameter feature z_hat is inversely transformed by the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. The second code stream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstruction feature y_hat corresponding to the current image block. The above process can be referred to in Embodiment 5 and will not be repeated here.

[0140] After obtaining the initial reconstruction feature y_hat, the initial reconstruction feature y_hat can be input to the feature domain enhancement module, and the initial reconstruction feature y_hat is enhanced by the feature domain enhancement module to obtain the enhanced reconstruction feature y_hat_enhanced, and the enhanced reconstruction feature y_hat_enhanced is input to the synthesis transformation network, and the enhanced reconstruction feature y_hat_enhanced is transformed by the synthesis transformation network to obtain the initial reconstruction image block x_hat corresponding to the current image block x, and the initial reconstruction image block x_hat includes the initial luminance reconstruction image block x_Y_hat and the initial chrominance reconstruction image block x_UV_hat.

[0141] After obtaining the initial luminance reconstruction image block x_Y_hat corresponding to the current image block x, the luminance edge enhancement module enhances the initial luminance reconstruction image block x_Y_hat to obtain the target luminance reconstruction image block x_Y_hat_enhanced. After obtaining the initial chrominance reconstruction image block x_UV_hat corresponding to the current image block x, the chrominance filter enhancement module enhances the initial chrominance reconstruction image block x_UV_hat to obtain the target chrominance reconstruction image block x_UV_hat_enhanced. Then, the target luminance reconstruction image block x_Y_hat_enhanced and the target chrominance reconstruction image block x_UV_hat_enhanced are combined to obtain the target reconstruction image block x_hat_enhanced corresponding to the current image block x.

[0142] Embodiment 7: On the basis of Embodiment 3 and Embodiment 4, a filter enhancement module can be added after the synthesis transformation network, which can include a luminance edge enhancement module and a chrominance filter enhancement module. See Figure 6CAs shown, the input feature of the filter enhancement module can be the initial reconstructed image block x_hat, and the output feature of the filter enhancement module can be the target reconstructed image block x_hat_enhanced. For example, the initial reconstructed image block x_hat can include an initial luma reconstructed image block x_Y_hat and an initial chroma reconstructed image block x_UV_hat. The luma edge enhancement module performs luma edge enhancement on the initial luma reconstructed image block x_Y_hat to obtain a target luma reconstructed image block x_Y_hat_enhanced. The chroma filter enhancement module performs chroma filter enhancement on the initial chroma reconstructed image block x_UV_hat to obtain a target chroma reconstructed image block x_UV_hat_enhanced. Then, the target luma reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced can be combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0143] For the encoding end: the current image block x is analyzed and transformed by the analysis transformation network to obtain the image feature y corresponding to the current image block x. The image feature y is subjected to coefficient hyperparameter feature transformation by the hyperparameter encoding network to obtain the coefficient hyperparameter feature z. The coefficient hyperparameter feature corresponding to the current image block is encoded to obtain the first code stream corresponding to the current image block. The coefficient hyperparameter feature z_hat corresponding to the current image block is obtained by decoding the first code stream corresponding to the current image block, and the probability distribution parameter sigma is obtained by subjecting the coefficient hyperparameter feature z_hat to coefficient hyperparameter feature inverse transformation by the probability hyperparameter decoding network. The initial image feature corresponding to the current image block is encoded based on the probability distribution parameter sigma to obtain the second code stream corresponding to the current image block. The initial reconstructed feature y_hat corresponding to the current image block is obtained by decoding the second code stream corresponding to the current image block based on the probability distribution parameter sigma. The above process can be referred to in Embodiment 5 and will not be repeated here.

[0144] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input to the synthesis transformation network (different from Embodiments 5 and 6, the enhanced reconstructed feature y_hat_enhanced is not input to the synthesis transformation network here), and the initial reconstructed feature y_hat is subjected to synthesis transformation by the synthesis transformation network to obtain the initial reconstructed image block x_hat corresponding to the current image block x, which includes the initial luma reconstructed image block x_Y_hat and the initial chroma reconstructed image block x_UV_hat.

[0145] The luminance edge enhancement module performs luminance edge enhancement on the initial luminance reconstructed image block x_Y_hat corresponding to the current image block x to obtain a target luminance reconstructed image block x_Y_hat_enhanced. The chroma filter enhancement module performs chroma filter enhancement on the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x to obtain a target chroma reconstructed image block x_UV_hat_enhanced. Then, the target luminance reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0146] For the decoding end: the first code stream corresponding to the current image block is decoded to obtain the coefficient hyperparameter feature z_hat corresponding to the current image block, the coefficient hyperparameter feature z_hat is inversely transformed by the probability hyperparameter decoding network to obtain the probability distribution parameter sigma. The second code stream corresponding to the current image block is decoded based on the probability distribution parameter sigma to obtain the initial reconstructed feature y_hat corresponding to the current image block. The above process can be referred to in Embodiment 5 and will not be repeated here.

[0147] After obtaining the initial reconstructed feature y_hat, the initial reconstructed feature y_hat is input into the synthesis transformation network, and the initial reconstructed feature y_hat is synthesized and transformed by the synthesis transformation network to obtain the initial reconstructed image block x_hat corresponding to the current image block x, the initial reconstructed image block x_hat includes the initial luminance reconstructed image block x_Y_hat and the initial chroma reconstructed image block x_UV_hat.

[0148] The luminance edge enhancement module performs luminance edge enhancement on the initial luminance reconstructed image block x_Y_hat corresponding to the current image block x to obtain a target luminance reconstructed image block x_Y_hat_enhanced. The chroma filter enhancement module performs chroma filter enhancement on the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x to obtain a target chroma reconstructed image block x_UV_hat_enhanced. Then, the target luminance reconstructed image block x_Y_hat_enhanced and the target chroma reconstructed image block x_UV_hat_enhanced are combined to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x.

[0149] In Embodiment 8, in Embodiment 5 and Embodiment 6, the initial reconstruction feature y_hat is input to the feature domain enhancement module, the feature domain enhancement module performs feature domain enhancement on the initial reconstruction feature y_hat to obtain an enhanced reconstruction feature y_hat_enhanced. For example, the initial reconstruction feature y_hat can be enhanced based on the feature domain enhancement parameter and the probability distribution parameter sigma corresponding to the current image block to obtain the enhanced reconstruction feature y_hat_enhanced. The feature domain enhancement can include feature adaptive edge enhancement and feature adaptive stretching. The probability distribution parameter sigma is used to assist in calculation to determine the channels that need to be subjected to feature adaptive edge enhancement and feature adaptive stretching, and to determine the stretching strength. The process is described below.

[0150] The initial reconstruction feature y_hat is a three-dimensional tensor of C L x H L x W L , C L is the number of channels of the initial reconstruction feature y_hat (i.e., the number of channels of the feature domain, the number of channels of the Latent domain), H L is the feature height of the initial reconstruction feature y_hat (i.e., the feature height of the feature domain, the feature height of the Latent domain), and W L is the feature width of the initial reconstruction feature y_hat (i.e., the feature width of the feature domain, the feature width of the Latent domain). For each channel of the initial reconstruction feature y_hat, take channel ch as an example for description. ch can be any value in [1, 2, …, C L ], and the shape corresponding to the channel ch can be a two-dimensional tensor of H L x W L , which is referred to as a feature channel map. Therefore, the initial reconstruction feature y_hat can include C L feature channel maps.

[0151] The probability distribution parameter sigma is also a three-dimensional tensor of C L x H L x W L , C L is the number of channels of the probability distribution parameter sigma, H L is the feature height of the probability distribution parameter sigma, and W L is the feature width of the probability distribution parameter sigma. For each channel of the probability distribution parameter sigma, take channel ch as an example for description. ch can be any value in [1, 2, …, C L ], and the shape corresponding to the channel ch is H L x W LThe two-dimensional tensor sigma_ch of the probability distribution parameter sigma is called a probability distribution channel map. Therefore, the probability distribution parameter sigma can include C L probability distribution channel maps, and the C L probability distribution channel maps correspond to the C L feature channels Figure 1 One-to-one correspondence, such as the first probability distribution channel map corresponds to the first feature channel map, the second probability distribution channel map corresponds to the second feature channel map, and so on, and the C L probability distribution channel map corresponds to the C L feature channel map.

[0152] For the C L feature channel maps, the C L feature channel maps can be divided into important feature channel maps and non-important feature channel maps. The important feature channel maps can be at least one, and the non-important feature channel maps can be multiple. For example, at least one feature channel map can be selected from the C L feature channel maps as an important feature channel map, and the remaining feature channel maps are selected as non-important feature channel maps. Since the C L probability distribution channel maps correspond to the C L feature channels Figure 1 , the important probability distribution channel maps corresponding to the important feature channel maps can be selected, and the non-important probability distribution channel maps corresponding to the non-important feature channel maps can be selected. For example, if the first feature channel map is an important feature channel map, the first probability distribution channel map corresponding to the first feature channel map is selected as an important probability distribution channel map.

[0153] In summary, the initial reconstructed feature y_hat can include important feature channel maps and non-important feature channel maps, and the probability distribution parameter sigma can include important probability distribution channel maps corresponding to the important feature channel maps and non-important probability distribution channel maps corresponding to the non-important feature channel maps. For the important feature channel maps, the feature adaptive edge enhancement can be performed on the important feature channel maps based on the feature domain enhancement parameter and the important probability distribution channel maps to obtain the first reconstructed feature after the feature adaptive edge enhancement. For the non-important feature channel maps, the feature adaptive stretching can be performed on the non-important feature channel maps based on the feature domain enhancement parameter and the non-important probability distribution channel maps to obtain the second reconstructed feature after the feature adaptive stretching. On this basis, the enhanced reconstructed feature y_hat_enhanced can be generated based on the first reconstructed feature and the second reconstructed feature.

[0154] For example, it is assumed that the feature channel Figure 1 is an important feature channel map, and the feature channel Figure 2 is a non-important feature channel map. The feature channel Figure 4The probability distribution channel is a non-important feature channel graph Figure 1 The probability distribution channel is an important probability distribution channel graph Figure 2 The probability distribution channel is an important probability distribution channel graph Figure 4 The probability distribution channel is a non-important probability distribution channel graph, and the feature adaptive edge enhancement can be performed on the feature channel based on the feature domain enhancement parameter and the probability distribution channel Figure 1 The feature adaptive edge enhancement is performed on the feature channel Figure 1 to obtain the first reconstructed feature 1 (i.e., the reconstructed feature of the first channel) after the feature adaptive edge enhancement. The feature adaptive edge enhancement can be performed on the feature channel based on the feature domain enhancement parameter and the probability distribution channel Figure 2 The feature adaptive edge enhancement is performed on the feature channel Figure 2 to obtain the second reconstructed feature 2 (i.e., the reconstructed feature of the second channel) after the feature adaptive edge enhancement. The feature adaptive edge enhancement can be performed on the feature channel based on the feature domain enhancement parameter and the probability distribution channel Figure 3 The feature adaptive edge enhancement is performed on the feature channel Figure 3 to obtain the second reconstructed feature 3 after the feature adaptive edge enhancement. The feature adaptive edge enhancement can be performed on the feature channel based on the feature domain enhancement parameter and the probability distribution channel Figure 4 The feature adaptive edge enhancement is performed on the feature channel Figure 4 to obtain the second reconstructed feature 4 after the feature adaptive edge enhancement. Then, the first reconstructed feature 1, the second reconstructed feature 2, the second reconstructed feature 3, and the second reconstructed feature 4 can be combined (e.g., spliced) according to the channel, i.e., the reconstructed features of the four channels are combined to obtain the enhanced reconstructed feature y_hat_enhanced.

[0155] In embodiment 8, C L feature channel graphs need to be divided into important feature channel graphs and non-important feature channel graphs, for example, the C L feature channel graphs are divided into important feature channel graphs and non-important feature channel graphs in the following manner:

[0156] Manner 1: The encoding end encodes the important channel identifier in the third code stream corresponding to the current image block, and the decoding end decodes the important channel identifier from the third code stream corresponding to the current image block, so that the decoding end can obtain the important channel identifier.

[0157] For example, the encoding end divides the C L feature channel graphs into important feature channel graphs and non-important feature channel graphs, which is not limited, and encodes the important channel identifier (also known as important channel number, denoted as important_channel) in the third code stream corresponding to the current image block. After receiving the third code stream, the decoding end decodes the important channel identifier from the third code stream, and obtains the important channel identifier from the C LThe important feature channel graph is selected from the C feature channel graphs based on the feature values in each feature channel graph and the probability distribution values in each probability distribution channel graph, and the remaining feature channel graphs are selected as non-important feature channel graphs.

[0158] wherein the encoding end selects the C L When the C feature channel graphs are divided into important feature channel graphs and non-important feature channel graphs, the important feature channel graph can be selected from the C L feature channel graphs based on the feature values in each feature channel graph and the probability distribution values in each probability distribution channel graph, and the remaining feature channel graphs are selected as non-important feature channel graphs.

[0159] In mode 2, the important feature channel graph is selected from the C L feature channel graphs based on the feature values in each feature channel graph and the probability distribution values in each probability distribution channel graph.

[0160] For example, for each feature channel graph, the number of consumed bits corresponding to the feature channel graph can be determined based on the feature values in the feature channel graph and the probability distribution values in the probability distribution channel graph corresponding to the feature channel graph. For example, the number of consumed bits corresponding to the feature channel graph can be determined by using the following expression, of course, the following expression is only an example.

[0161]

[0162] In the above expression, bits_per_ch can be used to represent the number of consumed bits corresponding to the feature channel graph ch, y_hat_ch(i, j) can be used to represent the feature value of the feature point (i, j) in the feature channel graph ch, sigma_ch(i, j) can be used to represent the probability distribution value of the feature point (i, j) in the probability distribution channel graph ch, and Φ(.) can be a standard normal distribution cumulative function.

[0163] After the above processing is performed on each feature channel graph, the number of consumed bits corresponding to each feature channel graph can be obtained, and based on the number of consumed bits corresponding to each feature channel graph, the important feature channel graph can be selected from the C L feature channel graphs, for example, the feature channel graph with the largest number of consumed bits is selected as the important feature channel graph, or the K feature channel graphs with large numbers of consumed bits are selected as the important feature channel graphs, and K can be a positive integer greater than 1.

[0164] After the important feature channel graph is selected from the C LAfter selecting the important feature channel graph from the C feature channel graphs, the remaining feature channel graphs can be selected as non-important feature channel graphs, the probability distribution channel graph corresponding to the important feature channel graph can be selected as an important probability distribution channel graph, and the probability distribution channel graph corresponding to the non-important feature channel graph can be selected as a non-important probability distribution channel graph.

[0165] Method 3, for the encoding end and the decoding end, based on the consumed code rate corresponding to each feature channel graph, from C L feature channel graphs, select important feature channel graphs. For example, based on the consumed code rate corresponding to each feature channel graph, the C L feature channel graphs can be sorted in descending order of consumed code rate, and the top K feature channel graphs can be selected as important feature channel graphs. Alternatively, based on the consumed code rate corresponding to each feature channel graph, the C L feature channel graphs can be sorted in ascending order of consumed code rate, and the last K feature channel graphs can be selected as important feature channel graphs.

[0166] After selecting the important feature channel graph from the C L feature channel graphs, the remaining feature channel graphs can be selected as non-important feature channel graphs, the probability distribution channel graph corresponding to the important feature channel graph can be selected as an important probability distribution channel graph, and the probability distribution channel graph corresponding to the non-important feature channel graph can be selected as a non-important probability distribution channel graph.

[0167] Method 4, for the encoding end and the decoding end, the default feature channel graph (i.e. fixed feature channel graph) is taken as the important feature channel graph, for example, the first feature channel graph is pre-agreed as the important feature channel graph, or the sixth feature channel graph is pre-agreed as the important feature channel graph, or the tenth feature channel graph is pre-agreed as the important feature channel graph, and so on. Of course, any feature channel graph can be taken as the important feature channel graph, which is not limited.

[0168] Method 5, the encoding end encodes the important channel identifier in the third code stream corresponding to the current image block, and the decoding end decodes the important channel identifier from the third code stream corresponding to the current image block, so that the decoding end can obtain the important channel identifier.

[0169] For example, the encoding end divides the C L feature channel graphs into important feature channel graphs and non-important feature channel graphs, and encodes the important channel identifier in the third code stream corresponding to the current image block. After receiving the third code stream, the decoding end decodes the important channel identifier from the third code stream, and selects the important feature channel graph from the C LThe feature channel graph is divided into an important feature channel graph and a non-important feature channel graph. The probability distribution channel graph corresponding to the important feature channel graph is selected as an important probability distribution channel graph, and the probability distribution channel graph corresponding to the non-important feature channel graph is selected as a non-important probability distribution channel graph. The encoding end divides the C L When the C feature channel graph is divided into an important feature channel graph and a non-important feature channel graph, the important feature channel graph and the important channel identifier can be determined based on the feature variance scale_hat in the following manner:

[0170] LinearToLog is a mapping table that can map the feature variance scale_hat to the logarithmic domain feature variance logarithmic_scale. For example, logarithmic_scale = LinearToLog(scale_hat).

[0171] The logarithmic domain feature variance logarithmic_scale_channel corresponding to a certain feature channel is obtained by slicing along the channel dimension.

[0172] Based on logarithmic_scale_channel, the logarithmic variance sum_per_channel corresponding to each feature channel is calculated.

[0173]

[0174] The above process is repeated C times, i.e., the logarithmic variance sum_per_channel corresponding to each feature channel is obtained, and then the channel number corresponding to the maximum sum_per_channel is selected as the important channel identifier important_channel, and the feature channel corresponding to the important channel identifier important_channel is the important feature channel graph. L

[0175] In embodiment 8, the feature adaptive edge enhancement is performed on the important feature channel graph based on the feature domain enhancement parameter and the important probability distribution channel graph to obtain the first reconstructed feature. The feature adaptive edge enhancement process is described as follows.

[0176] ​For example, the feature field enhancement parameter can include a plurality of edge enhancement segment intensity values and a plurality of edge enhancement segment threshold values. For example, the encoder encodes the plurality of edge enhancement segment intensity values and the plurality of edge enhancement segment threshold values in the third code stream corresponding to the current image block, the decoder decodes the plurality of edge enhancement segment intensity values and the plurality of edge enhancement segment threshold values from the third code stream corresponding to the current image block, and the plurality of edge enhancement segment intensity values and the plurality of edge enhancement segment threshold values are used as the feature field enhancement parameter.

[0177] The number of edge enhancement segment intensity values can be the same as or different from the number of edge enhancement segment threshold values. For example, the number of edge enhancement segment intensity values (or edge enhancement segment threshold values) is n. The n edge enhancement segment intensity values can be denoted as magl-1, magl-2, …, magl-n, and the n edge enhancement segment threshold values can be denoted as thrl-1, thrl-2, …, thrl-n.

[0178] For example, the plurality of edge enhancement segment threshold values can form a plurality of edge enhancement threshold intervals, and the plurality of edge enhancement threshold intervals correspond to the plurality of edge enhancement segment intensity values in a one-to-one manner. For example, the correspondence relationship is shown in Table 1.

[0179] Table 1

[0180] Edge enhancement threshold interval Edge enhancement segment intensity value Less than thrl-1 1 (i.e. no edge increase) [thrl-1, thrl-2) magl-1 [thrl-2, thrl-3) magl-2 [thrl-3, thrl-4) magl-3 ... ... [thrl-(n-1), thrl-n) magl-(n-1) Greater than or equal to thrl-n magl-n

[0181] For example, the important probability distribution channel map can include a plurality of probability distribution values. For each probability distribution value, the edge enhancement threshold interval corresponding to the probability distribution value is determined, and the edge enhancement segment intensity value corresponding to the probability distribution value is determined based on the edge enhancement threshold interval. For example, if the probability distribution value is located in [thrl-2, thrl-3), the edge enhancement segment intensity value corresponding to the probability distribution value is magl-2, if the probability distribution value is located in [thrl-3, thrl-4), the edge enhancement segment intensity value corresponding to the probability distribution value is magl-3, and so on. After obtaining the edge enhancement segment intensity value corresponding to each probability distribution value, the important feature channel map is subjected to feature adaptive edge enhancement based on the edge enhancement segment intensity value corresponding to each probability distribution value, and a first reconstructed feature subjected to feature adaptive edge enhancement is obtained.

[0182] For example, the important feature channel map can be subjected to feature adaptive edge enhancement by the following steps:

[0183] In step S11, the important feature channel map is normalized to obtain a normalized feature map.

[0184] Exemplarily, the normalized feature map can be obtained based on the important feature channel map, the mean feature corresponding to the important feature channel map, and the variance feature corresponding to the important feature channel map. For example, the important feature channel map can be subtracted by the mean feature, and then divided by the variance feature to obtain the normalized feature map. For another example, the important feature channel map can be subtracted by the mean feature, and then divided by the variance feature to obtain an intermediate feature, and then the intermediate feature is converted to obtain the normalized feature map, such as multiplying the intermediate feature by 0.1 and adding 0.5 (0.1 and 0.5 are examples), and then limiting the feature value to be between 0 and 1. Of course, the above is only several examples of normalizing the important feature channel map, and the normalization manner is not limited.

[0185] In step S12, the high-frequency detail image is generated based on the important feature channel map and the normalized feature map.

[0186] Exemplarily, the normalized feature map and the Gaussian blur convolution kernel (which can be a 3*3 convolution kernel, or a convolution kernel of other sizes, and the convolution kernel is not limited) can be convolved (such as two-dimensional convolution operation) to obtain a Gaussian blur image, and then the important feature channel map is subtracted by the Gaussian blur image to obtain the high-frequency detail image. Of course, the above is only an example of generating the high-frequency detail image, as long as the high-frequency details of the important feature channel map can be obtained, and the limitation is not made.

[0187] Exemplarily, for each value in the Gaussian blur convolution kernel, the denominator of the value can be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc. The Gaussian blur convolution kernel can be an N*N convolution kernel, N can be a positive integer, if the value of N is 3, that is, the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel can include but not limited to:

[0188]

[0189] Wherein, A can be a positive integer, B can be a positive integer, and M can be a power of 2, such as M can be 2, 4, 8, 16, 32, 64, 128. For example, A can be 5, B can be 24, and M can be 64.

[0190] In step S13, for each feature value in the high-frequency detail image, the feature value is edge enhanced based on the edge enhancement segmentation intensity value corresponding to the probability distribution value corresponding to the feature value to obtain an edge enhanced feature value.

[0191] For example, the important feature channel map includes a plurality of feature values, the important probability distribution channel map includes a plurality of probability distribution values, and the plurality of probability distribution values correspond to the plurality of feature values one by one. Since the high-frequency detail image includes a plurality of feature values (corresponding to the plurality of feature values of the important feature channel map one by one), the plurality of probability distribution values of the important probability distribution channel map correspond to the plurality of feature values of the high-frequency detail image one by one. Based on this, for each feature value in the high-frequency detail image, the probability distribution value corresponding to the feature value can be determined from the important probability distribution channel map, and the edge enhancement threshold interval corresponding to the probability distribution value can be determined. Based on the edge enhancement threshold interval, the edge enhancement segment intensity value corresponding to the probability distribution value is determined.

[0192] After obtaining the edge enhancement segment intensity value corresponding to the probability distribution value, the feature value can be edge enhanced based on the edge enhancement segment intensity value to obtain an edge enhanced feature value, such as multiplying the feature value by the edge enhancement segment intensity value (such as magl-1, magl-2, etc.) to obtain an edge enhanced feature value. Obviously, if the edge enhancement threshold interval is “less than thrl-1”, the feature value is multiplied by 1, that is, the feature value is kept unchanged and is not edge enhanced. If the edge enhancement threshold interval is [thrl-1, thrl-2), the feature value is multiplied by magl-1, and magl-1 is a value greater than 1, so that the feature value is edge enhanced. Similarly, magl-2 and other edge enhancement segment intensity values are greater than 1, which can achieve edge enhancement.

[0193] For each feature value in the high-frequency detail image, the edge enhancement segment intensity value corresponding to the feature value can be obtained in the above manner, and then the edge enhanced feature map is determined based on the edge enhanced feature value corresponding to each feature value. For example, the edge enhancement segment intensity values corresponding to all feature values can be combined to obtain an edge enhanced feature map.

[0194] In step S14, the edge enhanced feature map is de-normalized to obtain a first reconstructed feature (i.e., a first reconstructed feature map), which is a reconstructed feature map after feature adaptive edge enhancement of the important feature channel map.

[0195] After obtaining the edge enhanced feature map, the edge enhanced feature map can be directly de-normalized to obtain the first reconstructed feature. Alternatively, after obtaining the edge enhanced feature map, the edge enhanced feature map can be added to the normalized feature map to obtain a corrected edge enhanced feature map, and the corrected edge enhanced feature map is de-normalized to obtain the first reconstructed feature.

[0196] For example, the edge-enhanced feature map can be converted to obtain the edge-enhanced normalized feature map. For example, the feature values in the edge-enhanced feature map are first limited to be between 0 and 1, and then the feature values in the edge-enhanced feature map are subtracted by 0.5 and divided by 10 (0.5 and 10 are examples) to obtain the edge-enhanced normalized feature map.

[0197] For example, the edge-enhanced feature map can be converted to obtain the edge-enhanced normalized feature map. For example, the feature values in the edge-enhanced feature map are first limited to be between 0 and 1, and then the feature values in the edge-enhanced feature map are subtracted by 0.5 and divided by 10 (0.5 and 10 are examples) to obtain the edge-enhanced normalized feature map.

[0198] The edge-enhanced normalized feature map is multiplied by the variance feature and then added to the mean feature to obtain the edge-enhanced feature map corresponding to the important feature channel map, and the edge-enhanced feature map is denoted as the first reconstructed feature y_hat_sharp.

[0199] Of course, the above is only an example of the de-normalization of the edge-enhanced feature map, and the de-normalization manner is not limited.

[0200] In a possible implementation, the encoding end needs to encode multiple edge-enhanced segmentation intensity values and multiple edge-enhanced segmentation thresholds in the third code stream corresponding to the current image block. For this process, the encoding end can use the following manner:

[0201] The encoding end can configure multiple candidate feature domain enhancement parameters. For each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter can include multiple edge-enhanced segmentation intensity values and multiple edge-enhanced segmentation thresholds. The encoding end can determine the generation value corresponding to each candidate feature domain enhancement parameter. Based on the generation value corresponding to each candidate feature domain enhancement parameter, the feature domain enhancement parameter corresponding to the current image block can be selected from all candidate feature domain enhancement parameters, that is, the candidate feature domain enhancement parameter with the minimum generation value. The encoding end can encode the feature domain enhancement parameter to obtain the third code stream corresponding to the current image block.

[0202] For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature can be feature domain enhanced based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature. The feature domain enhancement process is described in the above embodiment. The target reconstructed image block x_hat_enhanced is determined based on the enhanced reconstructed feature. The generation value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced. The determination manner of the generation value is not limited.

[0203] For example, for each candidate feature field enhancement parameter, the candidate feature field enhancement parameter includes magl-1, magl-2, …, magl-n, thrl-1, thrl-2, …, thrl-n, based on the candidate feature field enhancement parameter, an enhanced reconstructed feature can be obtained, and then a target reconstructed image block x_hat_enhanced is obtained, and a distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index. After obtaining the distortion index value corresponding to each candidate feature field enhancement parameter, the candidate feature field enhancement parameter corresponding to the smallest distortion index value can be selected as the feature field enhancement parameter corresponding to the current image block, that is, the optimal feature field enhancement parameter, and the encoding end encodes the feature field enhancement parameter in the third code stream.

[0204] In embodiment 8, the feature-adaptive stretching needs to be performed on the non-important feature channel map based on the feature field enhancement parameter and the non-important probability distribution channel map (that is, the non-important probability distribution channel map corresponding to the non-important feature channel map), to obtain the second reconstructed feature after feature-adaptive stretching. The feature-adaptive stretching process is described below.

[0205] For example, the feature field enhancement parameter can include a stretching parameter value. For example, the encoding end can encode the stretching parameter value in the third code stream corresponding to the current image block, and the decoding end can decode the stretching parameter value from the third code stream corresponding to the current image block, and the stretching parameter value is used as the feature field enhancement parameter. The stretching parameter value can be denoted as p.

[0206] For example, the non-important feature channel map can include a plurality of feature values, and the non-important probability distribution channel map corresponding to the non-important feature channel map can include a plurality of probability distribution values, and the plurality of probability distribution values correspond to the plurality of feature values one by one. Based on this, for each feature value in the non-important feature channel map, the stretched feature value corresponding to the feature value can be determined based on the feature value, the stretching parameter value, and the probability distribution value corresponding to the feature value. For example, the stretched feature value corresponding to the feature value can be determined using the following expression, and of course, the following expression is only an example, which is not limited.

[0207] y_hat_scale = y_hat + p * clip3(sigma * y_hat, -0.5, 0.5)

[0208] In the above expression, y_hat_scale represents the stretched eigenvalue, y_hat represents the eigenvalue in the non-important feature channel graph, p represents the stretching parameter value, sigma represents the probability distribution value in the non-important probability distribution channel graph, and the probability distribution value sigma corresponds to the eigenvalue y_hat, and clip3 is a limiting operation for limiting sigma*y_hat to between -0.5 and 0.5, -0.5 and 0.5 are configurable values, and there is no limitation on this, for example, when sigma*y_hat is less than -0.5, sigma*y_hat is limited to -0.5, when sigma*y_hat is greater than 0.5, sigma*y_hat is limited to 0.5, and other values remain unchanged.

[0209] After the above processing is performed on each eigenvalue in the non-important feature channel graph, the stretched eigenvalue corresponding to each eigenvalue can be obtained, and then the second reconstruction feature (i.e., the second reconstruction feature graph) is determined based on the stretched eigenvalue corresponding to each eigenvalue, and the second reconstruction feature is the reconstruction feature graph after the feature adaptive stretching of the non-important feature channel graph, for example, the second reconstruction feature is obtained by combining the stretched eigenvalues corresponding to all eigenvalues, and the second reconstruction feature is also referred to as y_hat scale.

[0210] The second reconstruction feature y_hat scale corresponds to the non-important feature channel graph of the initial reconstruction feature y_hat, that is, the second reconstruction feature y_hat_scale is obtained by enhancing the non-important feature channel graph, the first reconstruction feature y_hat_sharp corresponds to the important feature channel graph of the initial reconstruction feature y_hat, that is, the first reconstruction feature y_hat_sharp is obtained by enhancing the important feature channel graph, and the second reconstruction feature y_hat_scale and the first reconstruction feature y_hat_sharp are combined to obtain the enhanced reconstruction feature y_hat_enhanced.

[0211] In a possible implementation, the encoding end needs to encode the stretching parameter value in the third code stream corresponding to the current image block, and for this process, the encoding end can use the following manner: the encoding end can configure multiple candidate stretching parameter values, can determine the generation value corresponding to each candidate stretching parameter value, and based on the generation value corresponding to each candidate stretching parameter value, the stretching parameter value corresponding to the current image block can be selected from all candidate stretching parameter values, that is, the candidate stretching parameter value with the minimum generation value, and the encoding end can encode the stretching parameter value to obtain the third code stream corresponding to the current image block.

[0212] For example, for each candidate stretching parameter value, the non-important feature channel map can be adaptively stretched based on the candidate stretching parameter value and the non-important probability distribution channel map to obtain a second reconstructed feature after feature adaptive stretching, and then an enhanced reconstructed feature is obtained based on the second reconstructed feature, the target reconstructed image block xhat enhanced is determined based on the enhanced reconstructed feature, and the objective function value corresponding to the candidate stretching parameter value is determined based on the target reconstructed image block xhat enhanced.

[0213] For example, for each candidate stretching parameter value, the enhanced reconstructed feature can be obtained based on the candidate stretching parameter value, and then the target reconstructed image block xhat enhanced is obtained, and the distortion index value of the target reconstructed image block xhat enhanced and the current image block x is calculated using the distortion index. After obtaining the distortion index value corresponding to each candidate stretching parameter value, the encoding end can select the candidate stretching parameter value corresponding to the smallest distortion index value as the stretching parameter value corresponding to the current image block, that is, the optimal stretching parameter value, and the encoding end can encode the stretching parameter value in the third code stream.

[0214] In another possible implementation, the encoding end configures multiple candidate feature domain enhancement parameters, for each candidate feature domain enhancement parameter, the candidate feature domain enhancement parameter includes multiple edge enhancement segment intensity values, multiple edge enhancement segment threshold values and a stretching parameter value, determines the objective function value corresponding to each candidate feature domain enhancement parameter, selects the feature domain enhancement parameter corresponding to the current image block from all candidate feature domain enhancement parameters based on the objective function value corresponding to each candidate feature domain enhancement parameter, that is, the candidate feature domain enhancement parameter with the smallest objective function value, and the encoding end encodes the feature domain enhancement parameter (that is, multiple edge enhancement segment intensity values, multiple edge enhancement segment threshold values and a stretching parameter value) to obtain the third code stream corresponding to the current image block. For example, for each candidate feature domain enhancement parameter, the initial reconstructed feature is enhanced in the feature domain based on the candidate feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstructed feature, the target reconstructed image block xhat enhanced is determined based on the enhanced reconstructed feature, and the objective function value corresponding to the candidate feature domain enhancement parameter is determined based on the target reconstructed image block xhat enhanced.

[0215] In Embodiment 12, in Embodiment 6 and Embodiment 7, the initial reconstructed image block x hat is input to the filter enhancement module, the initial reconstructed image block x hat is enhanced by the filter enhancement module, and the target reconstructed image block x hat enhanced corresponding to the current image block x is obtained. For example, the initial reconstructed image block x hat includes an initial luma reconstructed image block x Y hat and an initial chroma reconstructed image block x UV hat. The luma edge enhancement module performs luma edge enhancement on the initial luma reconstructed image block x Y hat to obtain a target luma reconstructed image block x Y hat enhanced. The chroma filter enhancement module performs chroma filter enhancement on the initial chroma reconstructed image block x UV hat to obtain a target chroma reconstructed image block x UV hat enhanced. The target luma reconstructed image block x Y hat enhanced and the target chroma reconstructed image block x UV hat enhanced are combined to obtain the target reconstructed image block x hat enhanced. The luma edge enhancement process of the luma edge enhancement module is described below.

[0216] For example, the luma edge enhancement module can perform luma edge enhancement on the initial luma reconstructed image block x Y hat based on the image domain enhancement parameter and the probability distribution parameter sigma corresponding to the current image block to obtain the target luma reconstructed image block x Y hat enhanced corresponding to the current image block. The luma edge enhancement process is described below.

[0217] For example, the image domain enhancement parameter can include a plurality of image enhancement segmentation intensity values and a plurality of image enhancement segmentation threshold values. For example, the encoder encodes the plurality of image enhancement segmentation intensity values and the plurality of image enhancement segmentation threshold values in the third code stream corresponding to the current image block, the decoder decodes the plurality of image enhancement segmentation intensity values and the plurality of image enhancement segmentation threshold values from the third code stream corresponding to the current image block, and the plurality of image enhancement segmentation intensity values and the plurality of image enhancement segmentation threshold values are used as the image domain enhancement parameter.

[0218] The number of image enhancement segmentation intensity values can be the same as or different from the number of image enhancement segmentation threshold values. For example, the number of image enhancement segmentation intensity values (or image enhancement segmentation threshold values) is encoded by the encoder in the third code stream corresponding to the current image block, and the number of image enhancement segmentation intensity values is decoded by the decoder from the third code stream corresponding to the current image block. For example, the number of image enhancement segmentation intensity values is m, the m image enhancement segmentation intensity values are denoted as magy-1, magy-2, …, magy-m, and the m image enhancement segmentation threshold values are denoted as thry-1, thry-2, …, thry-m.

[0219] Exemplarily, the plurality of image enhancement segmentation threshold values can constitute a plurality of image enhancement threshold value intervals, and the plurality of image enhancement threshold value intervals correspond to the plurality of image enhancement segmentation intensity values one by one. For an example of the correspondence, see Table 2.

[0220] Table 2

[0221] Image enhancement threshold interval Image enhancement segment intensity value Less than thry-1 1 (i.e. no edge increase) [thry-1, thry-2) magy-1 [thry-2, thry-3) magy-2 [thry-3, thry-4) magy-3 .. .. [thry-(m-1), thry-m) magy-(m-1) Greater than or equal to thry-m magy-m

[0222] Exemplarily, the initial luminance reconstructed image block x Y hat can be subjected to luminance edge enhancement by the following steps:

[0223] Step S21, obtaining a target probability distribution channel map based on the probability distribution parameter.

[0224] Exemplarily, the initial luminance reconstructed image block x Y hat can be a two-dimensional tensor of HxW, H is the image height of the initial luminance reconstructed image block x Y hat, and W is the image width of the initial luminance reconstructed image block x Y hat. The probability distribution parameter sigma can be a three-dimensional tensor of C L xH L xW L , C L is the number of channels of the probability distribution parameter sigma, H L is the feature height of the probability distribution parameter sigma, and W L is the feature width of the probability distribution parameter sigma. The image height H of the initial luminance reconstructed image block x Y hat can be greater than the feature height H L of the probability distribution parameter sigma, such as the image height H being 4 times, 8 times, 16 times, etc. of the feature height H L . The image width W of the initial luminance reconstructed image block x Y hat can be greater than the feature width W L of the probability distribution parameter sigma, such as the image width W being 4 times, 8 times, 16 times, etc. of the feature width W

[0225] In order to perform luminance edge enhancement on the initial luminance reconstructed image block x Y hat by using the probability distribution parameter, a target probability distribution channel map, which is a two-dimensional tensor of HxW, needs to be obtained based on the probability distribution parameter.

[0226] For example, one probability distribution channel map can be selected from the C L probability distribution channel maps. Since the probability distribution parameter sigma includes one important probability distribution channel map and multiple non-important probability distribution channel maps, one important probability distribution channel map can be selected from the C L probability distribution channel maps, or one non-important probability distribution channel map can be selected from the C LAn unimportant probability distribution channel graph is selected from the probability distribution channel graphs, and a subsequent important probability distribution channel graph is selected as an example for description.

[0227] The important probability distribution channel graph is then up-sampled to obtain a target probability distribution channel graph, and the up-sampling method is not limited, and the size of the target probability distribution channel graph is the same as that of the initial luminance reconstruction image block x Y hat.

[0228] For example, the nearest neighbor up-sampling method can be used to up-sample the important probability distribution channel graph to obtain the target probability distribution channel graph. For the nearest neighbor up-sampling method, a pixel in the original low-resolution image (i.e., the important probability distribution channel graph) can be selected as the center point of the corresponding region in the target high-resolution image (i.e., the target probability distribution channel graph). The value of the pixel closest to the center point in the original low-resolution image is assigned to the corresponding pixel in the target high-resolution image. The above process is repeated until all pixels in the target high-resolution image are assigned values.

[0229] As described above, the important probability distribution channel graph can be up-sampled to the same size as the initial luminance reconstruction image block x Y hat, and the up-sampled probability distribution channel graph is denoted as the target probability distribution channel graph sigma_channel_upscale.

[0230] For example, the third code stream corresponding to the current image block can be decoded to obtain the important channel identifier corresponding to the current image block. Of course, in addition to decoding the important channel identifier from the third code stream, other methods can also be used to obtain the important channel identifier, as described in the five implementation manners of embodiment 9. On this basis, if the probability distribution parameters include multiple probability distribution channel graphs, the probability distribution channel graph corresponding to the important channel identifier is selected from the multiple probability distribution channel graphs as the important probability distribution channel graph. Then, the important probability distribution channel graph is up-sampled to obtain the target probability distribution channel graph.

[0231] In step S22, if the target probability distribution channel graph includes multiple probability distribution values, for each probability distribution value, the image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value.

[0232] Exemplarily, the target probability distribution channel map can include a plurality of probability distribution values, for each probability distribution value, an image enhancement threshold interval corresponding to the probability distribution value can be determined first, and an image enhancement segmented intensity value corresponding to the probability distribution value can be determined based on the image enhancement threshold interval. For example, if the probability distribution value is located in [thry-2, thry-3), the image enhancement segmented intensity value corresponding to the probability distribution value can be magy-2, if the probability distribution value is located in [thry-3, thry-4), the image enhancement segmented intensity value corresponding to the probability distribution value can be magy-3, and so on.

[0233] In step S23, the initial luminance reconstructed image block x Y hat is subjected to luminance edge enhancement based on the image enhancement segmented intensity value corresponding to each probability distribution value, to obtain a target luminance reconstructed image block x Y hatenhanced corresponding to the current image block.

[0234] Firstly, a high-frequency detail image can be generated based on the initial luminance reconstructed image block x Y hat. For example, a convolution operation (such as a two-dimensional convolution operation) can be performed on the initial luminance reconstructed image block x Y hat and a Gaussian blur convolution kernel (which can be a 3*3 convolution kernel, or a convolution kernel of other sizes, and the convolution kernel is not limited), to obtain a Gaussian blur image, and then the initial luminance reconstructed image block x Y hat is subtracted from the Gaussian blur image to obtain a high-frequency detail image. Of course, the above is only an example of generating a high-frequency detail image, as long as the high-frequency details of the initial luminance reconstructed image block x Y hat can be obtained.

[0235] Exemplarily, for each value in the Gaussian blur convolution kernel, the denominator of the value can be a power of 2, such as 2, 4, 8, 16, 32, 64, 128, etc. The Gaussian blur convolution kernel can be an N*N convolution kernel, N can be a positive integer, if the value of N is 3, that is, the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel can include but is not limited to:

[0236]

[0237] Wherein, A can be a positive integer, B can be a positive integer, and M can be a power of 2, such as M can be 2, 4, 8, 16, 32, 64, 128. For example, A can be 5, B can be 24, and M can be 64.

[0238] Then, for each feature value in the high-frequency detail image, the feature value is edge enhanced based on the image enhancement segmented intensity value corresponding to the probability distribution value corresponding to the feature value, to obtain an image enhancement feature value. For example, the initial luminance reconstruction image block x_Y_hat includes a plurality of feature values, the target probability distribution channel image includes a plurality of probability distribution values, and the plurality of probability distribution values correspond one-to-one to the plurality of feature values. Since the high-frequency detail image includes a plurality of feature values (corresponding one-to-one to the plurality of feature values of the initial luminance reconstruction image block x_Y_hat), the plurality of probability distribution values of the target probability distribution channel image correspond one-to-one to the plurality of feature values of the high-frequency detail image. Based on this, for each feature value in the high-frequency detail image, the probability distribution value corresponding to the feature value can be determined from the target probability distribution channel image, and the image enhancement threshold interval corresponding to the probability distribution value is determined, and then the image enhancement segmented intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval.

[0239] After obtaining the image enhancement segmented intensity value corresponding to the probability distribution value, the feature value can be edge enhanced based on the image enhancement segmented intensity value to obtain an image enhancement feature value, such as multiplying the feature value by the image enhancement segmented intensity value (such as magy-1, magy-2, etc.), to obtain an image enhancement feature value. Obviously, if the image enhancement threshold interval is “less than thry-1”, the feature value is multiplied by 1, that is, the feature value is kept unchanged, and the feature value is not edge enhanced. If the image enhancement threshold interval is [thry-1, thry-2), the feature value is multiplied by magy-1, magy-1 is a value greater than 1, so that the feature value is edge enhanced. Similarly, other image enhancement segmented intensity values such as magy-2 are greater than 1, which can achieve edge enhancement.

[0240] Then, the target luminance reconstruction image block x_Y_hat_enhanced is determined based on the image enhancement feature value corresponding to each feature value in the high-frequency detail image. For example, for each feature value in the high-frequency detail image, the image enhancement feature value corresponding to the feature value can be obtained in the above manner, and then the image enhancement feature map is determined based on the image enhancement feature value corresponding to each feature value. For example, the image enhancement feature values corresponding to all feature values can be combined to obtain an image enhancement feature map. Then, the image enhancement feature map is added to the initial luminance reconstruction image block x_Y_hat to obtain a final edge enhanced image, and the edge enhanced image is limited to the image value domain, thereby obtaining the target luminance reconstruction image block x_Y_hat_enhanced.

[0241] In a possible implementation, the encoding end needs to encode a plurality of image enhancement segmented intensity values and a plurality of image enhancement segmented threshold values in the third code stream corresponding to the current image block. For this process, the encoding end can use the following manner:

[0242] The encoding end can configure multiple candidate image domain enhancement parameters. For each candidate image domain enhancement parameter, the candidate image domain enhancement parameter can include multiple image enhancement segment intensity values and multiple image enhancement segment threshold values. The encoding end can determine a generation value corresponding to each candidate image domain enhancement parameter. Based on the generation value corresponding to each candidate image domain enhancement parameter, the encoding end can select the image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters, that is, the candidate image domain enhancement parameter with the minimum generation value. The encoding end can encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0243] For example, for each candidate image domain enhancement parameter, the initial luminance reconstructed image block x_Y_hat is subjected to luminance edge enhancement based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstructed image block x_Y_hat_enhanced corresponding to the current image block. The luminance edge enhancement process is described above and will not be repeated here. The target reconstructed image block x_hat_enhanced can be obtained based on the target luminance reconstructed image block x_Y_hat_enhanced. Then, the generation value corresponding to the candidate image domain enhancement parameter is determined based on the target reconstructed image block x_hat_enhanced. The determination method is not limited.

[0244] For example, for each candidate image domain enhancement parameter, the candidate image domain enhancement parameter includes magy-1, magy-2, …, magy-m, thry-1, thry-2, …, thry-m. The target luminance reconstructed image block x_Y_hat_enhanced can be obtained based on the candidate image domain enhancement parameter. Then, the target reconstructed image block x_hat_enhanced is obtained based on the target luminance reconstructed image block x_Y_hat_enhanced. The distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index. After obtaining the distortion index value corresponding to each candidate image domain enhancement parameter, the candidate image domain enhancement parameter corresponding to the minimum distortion index value can be selected as the image domain enhancement parameter corresponding to the current image block, that is, the best image domain enhancement parameter. In this way, the encoding end can encode the image domain enhancement parameter in the third code stream.

[0245] Example 13: In Examples 6 and 7, a chroma filtering enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process of the chroma filtering enhancement module is described below. For example, after obtaining the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement module can also perform chroma filtering enhancement based on the initial luminance reconstructed image block x_Y_hat and the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced corresponding to the current image block x. This chroma filtering enhancement process is described below.

[0246] First, adaptive upsampling is performed on the initial chroma reconstructed image block x_UV_hat and the initial luminance reconstructed image block x_Y_hat to obtain an upsampled chroma reconstructed image block. The resolution of the upsampled chroma reconstructed image block is the same as that of the initial luminance reconstructed image block x_Y_hat. For example, the initial chroma reconstructed image block x_UV_hat is adaptively upsampled to obtain the upsampled chroma reconstructed image block, which has the required resolution size, such as the resolution of the initial luminance reconstructed image block x_Y_hat. Furthermore, the adaptive upsampling process is guided by the initial luminance reconstructed image block x_Y_hat.

[0247] Then, cross-channel filtering is performed on the upsampled chroma reconstructed image block and the initial luminance reconstructed image block x_Y_hat to obtain the cross-channel filtered chroma reconstructed image block. For example, after obtaining the upsampled chroma reconstructed image block, it will undergo cross-channel filtering to obtain the first enhanced chroma reconstructed image block (i.e., the cross-channel filtered chroma reconstructed image block), and the cross-channel filtering process is guided by the initial luminance reconstructed image block x_Y_hat.

[0248] See Figure 6D As shown, after performing wavelet transform on the upsampled chroma reconstructed image block and the initial luminance reconstructed image block x_Y_hat, the features of the channel-dimensional concatenation are input into the residual block network, which enhances the upsampled chroma reconstructed image block. Finally, the cross-channel filtered chroma reconstructed image block is obtained through inverse wavelet transform.

[0249] Then, the non-linear chroma filtering is performed based on the cross-channel filtered chroma reconstructed image block and the initial luma reconstructed image block x_Y_hat, to obtain a non-linear chroma filtered chroma reconstructed image block, and the non-linear chroma filtered chroma reconstructed image block is the target chroma reconstructed image block x_UV_hat_enhanced. For example, after the cross-channel filtered chroma reconstructed image block is obtained, the cross-channel filtered chroma reconstructed image block will be subjected to the non-linear chroma filtering to obtain a second enhanced chroma reconstructed image block, and the non-linear chroma filtering process is guided by the initial luma reconstructed image block x_Y_hat. After the second enhanced chroma reconstructed image block is obtained, this chroma reconstructed image block is the target chroma reconstructed image block x_UV_hat_enhanced.

[0250] Referring to Figure 6E As shown in the figure, the initial luma reconstructed image block x_Y_hat can be down-sampled, and then the down-sampled luma reconstructed image block is connected with the cross-channel filtered chroma reconstructed image block in the channel dimension, wherein the resolution of the down-sampled luma reconstructed image block is the same as that of the cross-channel filtered chroma reconstructed image block. Then, the feature after the channel dimension connection is input into the non-linear filter, and then the target chroma reconstructed image block x_UV_hat_enhanced is output.

[0251] In the above process, the cross-channel filtering is performed first and then the non-linear chroma filtering is performed. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, the non-linear chroma filtering can also be performed first and then the cross-channel filtering is performed, or only the cross-channel filtering (without non-linear chroma filtering) can be performed, or only the non-linear chroma filtering (without cross-channel filtering) can be performed, and no limitation is made in this regard, as long as x_UV_hat_enhanced can be obtained through filtering.

[0252] In embodiments 6 and 7, the chroma filtering enhancement module is used to perform chroma filtering enhancement on the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced. The chroma filtering enhancement process of the chroma filtering enhancement module is described below. For example, after the chroma filtering enhancement module obtains the initial chroma reconstructed image block x_UV_hat corresponding to the current image block x, the chroma filtering enhancement can be performed based on the target luma reconstructed image block x_Y_hat_enhanced and the initial chroma reconstructed image block x_UV_hat to obtain the target chroma reconstructed image block x_UV_hat_enhanced corresponding to the current image block x, and the chroma filtering enhancement process is described below.

[0253] First, adaptive up-sampling is performed based on the initial chroma reconstructed image block x_UV_hat and the target luma reconstructed image block x_Y_hat_enhanced, to obtain an up-sampled chroma reconstructed image block. The up-sampled chroma reconstructed image block has the same resolution as the target luma reconstructed image block x_Y_hat_enhanced. For example, the initial chroma reconstructed image block x_UV_hat is up-sampled to obtain an up-sampled chroma reconstructed image block, which has the same resolution as the target luma reconstructed image block x_Y_hat_enhanced, and the adaptive up-sampling process is guided by the target luma reconstructed image block x_Y_hat_enhanced.

[0254] Then, cross-channel filtering is performed based on the up-sampled chroma reconstructed image block and the target luma reconstructed image block x_Y_hat_enhanced, to obtain a cross-channel filtered chroma reconstructed image block. For example, after obtaining the up-sampled chroma reconstructed image block, the up-sampled chroma reconstructed image block is filtered by cross-channel filtering to obtain a first enhanced chroma reconstructed image block (i.e., a cross-channel filtered chroma reconstructed image block), and the cross-channel filtering process is guided by the target luma reconstructed image block x_Y_hat_enhanced. For example, after wavelet transformation is performed on the up-sampled chroma reconstructed image block and wavelet transformation is performed on the target luma reconstructed image block x_Y_hat_enhanced, the channel dimension is connected, and the connected features in the channel dimension are input into a residual block network, so that the up-sampled chroma reconstructed image block is enhanced, and the cross-channel filtered chroma reconstructed image block is obtained through inverse wavelet transformation.

[0255] Then, non-linear chroma filtering is performed based on the cross-channel filtered chroma reconstructed image block and the target luma reconstructed image block x_Y_hat_enhanced, to obtain a non-linear chroma filtered chroma reconstructed image block, which is the target chroma reconstructed image block x_UV_hat_enhanced. For example, after obtaining the cross-channel filtered chroma reconstructed image block, the cross-channel filtered chroma reconstructed image block is filtered by non-linear chroma filtering to obtain a second enhanced chroma reconstructed image block (the target chroma reconstructed image block x_UV_hat_enhanced), and the non-linear chroma filtering process is guided by the target luma reconstructed image block x_Y_hat_enhanced. For example, the target luma reconstructed image block x_Y_hat_enhanced is down-sampled, and the down-sampled luma reconstructed image block is connected with the cross-channel filtered chroma reconstructed image block in the channel dimension. The down-sampled luma reconstructed image block has the same resolution as the cross-channel filtered chroma reconstructed image block. The connected features in the channel dimension are input into a non-linear filter, and then the target chroma reconstructed image block x_UV_hat_enhanced is output.

[0256] In the above process, the cross-channel filtering is performed first and then the non-linear chroma filtering is performed. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, the non-linear chroma filtering can also be performed first and then the cross-channel filtering is performed, or only the cross-channel filtering (without non-linear chroma filtering) can be performed, or only the non-linear chroma filtering (without cross-channel filtering) can be performed, and no limitation is made in this regard as long as x_UV_hat_enhanced can be obtained through filtering.

[0257] In the above process, the cross-channel filtering is performed first and then the non-linear chroma filtering is performed. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, the non-linear chroma filtering can also be performed first and then the cross-channel filtering is performed, or only the cross-channel filtering (without non-linear chroma filtering) can be performed, or only the non-linear chroma filtering (without cross-channel filtering) can be performed, and no limitation is made in this regard as long as x_UV_hat_enhanced can be obtained through filtering.

[0258] In the above process, the cross-channel filtering is performed first and then the non-linear chroma filtering is performed. In order to obtain the target chroma reconstructed image block x_UV_hat_enhanced, the non-linear chroma filtering can also be performed first and then the cross-channel filtering is performed, or only the cross-channel filtering (without non-linear chroma filtering) can be performed, or only the non-linear chroma filtering (without cross-channel filtering) can be performed, and no limitation is made in this regard as long as x_UV_hat_enhanced can be obtained through filtering.

[0259] In the chroma filter enhancement of the initial chroma reconstructed image block x UV hat, the target probability distribution channel map of the chroma component can also be obtained based on the probability distribution parameters of the luminance component, such as selecting an important probability distribution channel map from all the probability distribution channel maps of the probability distribution parameters of the luminance component, and upsampling the important probability distribution channel map to obtain the target probability distribution channel map of the chroma component, which has the same size as the initial chroma reconstructed image block x UV hat of the chroma component. For example, the nearest neighbor upsampling method is used to upsample the important probability distribution channel map to obtain the target probability distribution channel map of the chroma component. For the nearest neighbor upsampling method, a pixel is selected on the original low-resolution image (i.e., the important probability distribution channel map) as the center point of the corresponding region on the target high-resolution image (i.e., the target probability distribution channel map). The value of the pixel closest to the pixel on the original low-resolution image is taken as the value of the corresponding pixel on the target high-resolution image. The above process is repeated until all the pixels of the target high-resolution image are assigned a value.

[0260] After obtaining the target probability distribution channel map of the chroma component, for each probability distribution value in the target probability distribution channel map, the image enhancement segmentation intensity value corresponding to the probability distribution value is determined based on the image enhancement threshold interval corresponding to the probability distribution value, and the initial chroma reconstructed image block x UV hat corresponding to the chroma component is subjected to chroma filter enhancement based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain a target chroma reconstructed image block x UV hat_enhanced. The chroma filter enhancement process can refer to Embodiment 12, and the relevant operations are performed for the chroma component.

[0261] Embodiment 16: A decoding method is proposed in this embodiment, which can include the following steps:

[0262] Step S31: Decode the feature domain enhancement parameters, image domain enhancement parameters, and important channel identifier (i.e., important_channel number) from Bitstream#3 (the third bitstream, which can also be called the feature enhancement header information bitstream) corresponding to the current image block. The feature domain enhancement parameters may include multiple edge enhancement segment intensity values ​​(denoted as magl-1, magl-2, ..., magl-n), multiple edge enhancement segment thresholds (denoted as thrl-1, thrl-2, ..., thrl-n), and a stretching parameter value ρ. The image domain enhancement parameters may include multiple image enhancement segment intensity values ​​(denoted as magy-1, magy-2, ..., magy-m), and multiple image enhancement segment thresholds (denoted as thry-1, thry-2, ..., thry-m). The feature domain enhancement parameters may also include the number of edge enhancement segment intensity values ​​n, and the image domain enhancement parameters may also include the number of image enhancement segment intensity values ​​m.

[0263] Step S32: The input features of the feature domain enhancement module are the initial reconstructed feature y_hat and the probability distribution parameter sigma. The initial reconstructed feature y_hat is a feature of size C. L xH L xW L The three-dimensional tensor, C L H represents the number of channels in the feature domain, i.e., the latent domain. L W represents the feature height of the Latent domain. L Let be the feature width of the Latent domain. The probability distribution parameter sigma is a parameter of size C. L xH L xW L The three-dimensional tensor. When the channels of the initially reconstructed feature y_hat are [1, 2, ..., C] L For any value in ], the shape will be H. L xW L The two-dimensional tensor y_hat_ch is called the feature channel map. When the channels of the probability distribution parameter sigma are [1, 2, ..., C], ... L For any value in ], the shape will be H. L xW L The two-dimensional tensor sigma_ch is called the probability distribution channel map.

[0264] Step S33: The initial reconstructed feature y_hat has a total of C L There are 1 feature channel map, and the probability distribution parameter sigma has a total of C. L A probability distribution channel diagram, C L Each probability distribution channel diagram and C L Feature channels Figure 1A corresponding, the probability distribution channel graph is represented as sigma_ch. The feature adaptive edge enhancement is performed on the feature channel graph of the channel ch being the important channel identification important_channel, and the feature adaptive stretching is performed on the feature channel graph of the channel ch being the non-important channel identification.

[0265] Step S34, for the feature adaptive edge enhancement process, the input data is the important feature channel graph y_hat_ch, the important probability distribution channel graph sigma_ch, a plurality of edge enhancement segment intensity values (magl-1, magl-2, …, magl-n), a plurality of edge enhancement segment threshold values (thrl-1, thrl-2, …, thrl-n), based on the above input data, the important feature channel graph y_hat_ch can be feature adaptive edge enhanced, and a reconstructed feature graph after feature adaptive edge enhancement is obtained, which is called the first reconstructed feature y_hat_sharp. The feature adaptive edge enhancement process is described in Embodiment 10 and will not be repeated here.

[0266] Step S35, for the feature adaptive stretching process, the input data is the non-important feature channel graph (C L -1 non-important feature channel graph in addition to the important feature channel graph), the non-important probability distribution channel graph (C L -1 non-important probability distribution channel graph in addition to the important probability distribution channel graph), the stretching parameter value ρ, based on the above input data, each non-important feature channel graph can be feature adaptive stretched, and a reconstructed feature graph after feature adaptive stretching is obtained, which is called the second reconstructed feature y_hat_scale. The feature adaptive stretching process is described in Embodiment 11 and will not be repeated here. The size of the non-important feature channel graph subjected to feature adaptive stretching is (C L -1) x H L x W L , and the size of the corresponding probability distribution parameter sigma is also (C L -1) x H L x W L Each element of this three-dimensional tensor can be subjected to the following adaptive stretching algorithm to obtain the stretched second reconstructed feature: y_hat_scale = y_hat + ρ * clip3(sigma * y_hat, -0.5, 0.5), clip3 is a limit operation, and the second reconstructed feature after the feature adaptive stretching algorithm is y_hat_scale.

[0267] Step S36, the second reconstructed feature y_hat_scale corresponds to the non-important channel enhancement of the initial reconstructed feature y_hat, the first reconstructed feature y_hat_sharp corresponds to the important channel enhancement of the initial reconstructed feature y_hat, and the first reconstructed feature y_hat_sharp and the second reconstructed feature y_hat_scale are combined to obtain the enhanced reconstructed feature y_hat_enhanced after image enhancement.

[0268] Step S37, input the enhanced reconstructed feature y_hat_enhanced after image enhancement to the synthesis transformation network to obtain the initial reconstructed image block x_hat, and the initial reconstructed image block x_hat is a two-dimensional tensor with a size of HxW. The important probability distribution channel map sigma_channel of the probability distribution parameter sigma is upsampled to the same size as the initial reconstructed image block x_hat, and the upsampled probability distribution channel map is denoted as the target probability distribution channel map sigma_channel_upscale.

[0269] Step S38, for the filter enhancement process, the input data of the filter enhancement module are the initial reconstructed image block x_hat, the target probability distribution channel map sigma_channel_upscale, a plurality of image enhancement segment intensity values (magy-1, magy-2, …, magy-m), a plurality of image enhancement segment threshold values (thry-1, thry-2, …, thry-m), and based on the above input data, the filter enhancement module can perform filter enhancement on the initial reconstructed image block x_hat to obtain the target reconstructed image block x_hat_enhanced corresponding to the current image block x. The filter enhancement process is described in Embodiments 12-14 and will not be repeated here.

[0270] Embodiment 17: An encoding method is provided in this embodiment, which can include the following steps:

[0271] Step S41, based on C L feature channel maps y_hat and C L probability distribution channel maps sigma, determine the important channel identifier (important channel identifier is also referred to as important channel number important_channel) corresponding to the important feature channel map.

[0272] For example, slice y_hat and sigma along the channel dimension to obtain the current y_hat_channel and sigma_channel, and calculate the bits_per_ch of each feature channel map through the two tensors, as shown in the following expression, and loop the above process C LSecondly, bits_per_ch of each feature channel map is obtained, and the channel serial number corresponding to the maximum bits_per_ch is selected as the important channel identifier.

[0273]

[0274] Step S42, a stretching parameter value p corresponding to the feature adaptive stretching process is determined. For example, N1 candidate stretching parameter values p are selected, the enhanced reconstructed feature and the target reconstructed image block x_hat_enhanced corresponding to each candidate stretching parameter value p are obtained, the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate stretching parameter value p corresponding to the minimum distortion index value is selected as the optimal stretching parameter value p corresponding to the feature adaptive stretching process.

[0275] Step S43, a feature domain enhancement parameter corresponding to the feature adaptive edge enhancement process is determined, and the feature domain enhancement parameter includes a plurality of edge enhancement segment intensity values (magl-1, magl-2, …, magl-n), a plurality of edge enhancement segment threshold values (thrl-1, thrl-2, …, thrl-n). For example, N2 candidate feature domain enhancement parameters are selected, the enhanced reconstructed feature and the target reconstructed image block x_hat_enhanced corresponding to each candidate feature domain enhancement parameter are obtained, the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate feature domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal feature domain enhancement parameter corresponding to the feature adaptive edge enhancement process.

[0276] Step S44, an image domain enhancement parameter corresponding to the image domain enhancement process is determined, and the image domain enhancement parameter can include a plurality of image enhancement segment intensity values (magy-1, magy-2, …, magy-m), a plurality of image enhancement segment threshold values (thry-1, thry-2, …, thry-m). For example, N3 candidate image domain enhancement parameters are selected, the enhanced reconstructed feature and the target reconstructed image block x_hat_enhanced corresponding to each candidate image domain enhancement parameter are obtained, the distortion index value of the target reconstructed image block x_hat_enhanced and the current image block x is calculated using the distortion index, and the candidate image domain enhancement parameter corresponding to the minimum distortion index value is selected as the optimal image domain enhancement parameter corresponding to the image domain enhancement process.

[0277] Step S45, the important feature channel map corresponding important channel identifier, the best stretching parameter value p, the best feature domain enhancement parameter, the best image domain enhancement parameter are coded into the head information code stream (the third code stream # Bitstreams3 corresponding to the current image block). It needs to be noted that the feature domain enhancement module and the image domain enhancement module will not change # Bitstreams1 and # Bitstreams2, but the enhancement module will be passed through when the encoding end and the decoding end reconstruct the image, and therefore the reconstructed image will be consistent at the encoding and decoding end.

[0278] Embodiment 18: the method of image adaptive edge enhancement (i.e. the process of filtering and enhancing the initial reconstructed image block x_hat by the filtering enhancement module to obtain the target reconstructed image block x_hat_enhanced) can be a non-edge enhancement mask edge enhancement algorithm, i.e. USM edge enhancement algorithm (Unsharp Masking edge enhancement), which includes:

[0279] Step S51, two-dimensional convolution operation is performed on the original reconstructed image and the Gaussian blur convolution kernel to obtain a Gaussian blur image.

[0280] Step S52, the original reconstructed image is subtracted from the Gaussian blur image to obtain a high-frequency detail image.

[0281] Step S53, the high-frequency detail image is multiplied by an edge enhancement coefficient (i.e. the image enhancement segment intensity value corresponding to the probability distribution value corresponding to the eigenvalue), and added to the original reconstructed image to obtain a final edge enhanced image.

[0282] Step S54, the edge enhanced image is limited to the value range of the image.

[0283] Illustratively, the original reconstructed image is the initial reconstructed image block x_hat, and after the edge enhanced image is limited to the value range of the image, the target reconstructed image block x_hat_enhanced can be obtained, and the process can be referred to in the above embodiment.

[0284] Embodiment 19: the method of feature adaptive edge enhancement (i.e. the process of feature adaptive edge enhancement of the important feature channel map by the feature domain enhancement module based on the feature domain enhancement parameter and the important probability distribution channel map to obtain the first reconstructed feature) can be a non-edge enhancement mask edge enhancement algorithm, i.e. USM edge enhancement algorithm, which includes:

[0285] Step S61, the important channel feature map y_hat_channel is subtracted from the mean and divided by the variance to obtain a normalized feature map.

[0286] Step S62, multiply the normalized feature map by 0.1 and add 0.5, and limit the feature value to between 0 and 1.

[0287] Step S63, perform a two-dimensional convolution operation on the normalized feature map with a Gaussian blur kernel to obtain a Gaussian blur image.

[0288] Step S64, subtract the Gaussian blur image from the original reconstructed image to obtain a high-frequency detail image.

[0289] Step S65, multiply the high-frequency detail image by an edge enhancement coefficient (i.e., the edge enhancement segment intensity value corresponding to the probability distribution value of the feature value), and add it to the normalized feature map to obtain an edge-enhanced feature map.

[0290] Step S66, limit the feature value of the edge-enhanced feature map to between 0 and 1.

[0291] Step S67, subtract 0.5 from the edge-enhanced feature map and divide by 10 to obtain the normalized feature map after edge enhancement.

[0292] Step S68, multiply the normalized feature map after edge enhancement by the variance and add the mean to obtain the important channel feature edge-enhanced feature map y_hat_shap corresponding to the important channel feature map y_hat_channel, i.e., the first reconstructed feature.

[0293] For example, the original reconstructed image is the important feature channel map y_hat_channel, and the process can refer to Embodiment 10.

[0294] Embodiment 20: In the decoding process, the initial chroma reconstructed image block x_UV_hat is adaptively upsampled, and the chroma reconstructed image becomes the required resolution size, which is guided by the initial luminance reconstructed image block x_Y_hat. The chroma reconstructed image after adaptive upsampling will be cross-channel filtered to obtain the first enhanced chroma reconstructed image, which is guided by the initial luminance reconstructed image block x_Y_hat. The first enhanced chroma reconstructed image will be subjected to non-linear chroma filtering to obtain the second chroma image enhancement and output the final chroma enhanced image x_UV_hat_enhanced. In addition, the initial luminance reconstructed image block x_Y_hat is subjected to luminance edge enhancement to output the target luminance reconstructed image block x_Y_hat_enhanced.

[0295] For the enhancement process of cross-channel filtering, the chroma reconstructed image and the initial luma reconstructed image block x_Y_hat are connected in the channel dimension after wavelet transform, and then the chroma reconstructed image is enhanced after passing through the residual block network. Finally, the new chroma reconstructed image is obtained through wavelet inverse transform. And / or, for the enhancement process of nonlinear chroma filtering, after downsampling the initial luma reconstructed image block x_Y_hat, it becomes the same resolution size as the chroma reconstructed image, and then it is connected in the channel dimension. After passing through a nonlinear filter, the target chroma enhanced image x_UV_hat_enhanced can be obtained.

[0296] Embodiment 21: In the decoding process, the filter enhancement sub-module can extract the important channel identifier from the bitstream.

[0297] Embodiment 22: In the decoding process, for the Gaussian blur convolution kernel involved in each embodiment, the denominator of each weight in the Gaussian blur convolution kernel kernel is designed as a power of 2, such as a usable weight design as shown below:

[0298]

[0299] Embodiment 23: In the encoding process, the important channel identifier can be determined according to the feature variance scale_hat, and the method is as follows:

[0300] LinearToLog is a mapping table that can map the feature variance scale_hat to the logarithmic domain feature variance logarithmic_scale, for example, logarithmic_scale = LinearToLog(scale_hat).

[0301] Slice along the channel dimension to obtain the logarithmic domain feature variance logarithmic_scale_channel corresponding to a certain feature channel.

[0302] Based on logarithmic_scale_channel, calculate the logarithmic variance sum_per_channel corresponding to each feature channel:

[0303]

[0304] Loop the above process, a total of C channels of logarithmic_scale LThen, the sum_per_channel corresponding to the maximum sum_per_channel is selected as the important channel identifier important_channel, and the important channel identifier important_channel is also referred to as an important channel number.

[0305] From the above technical solutions, in the embodiments of the present application, an end-to-end video image compression method is proposed, which can realize encoding and decoding of video images based on a neural network, and achieve the purpose of improving encoding efficiency and decoding efficiency by combining image domain enhancement parameters and probability distribution parameters. By combining network structure design and header information code stream (such as the third code stream), the neural network effectively guarantees the quality of the reconstructed image block while maintaining low complexity, achieves the purpose of improving encoding performance and decoding performance, and reduces complexity. The image quality of the feature is enhanced by using the image domain enhancement parameters and the probability distribution parameters. The encoding end does not directly change the feature information, but encodes the image domain enhancement parameters into the header information code stream. The decoding end enhances the reconstructed image block by using the image domain enhancement parameters, improves the encoding performance, and improves the reconstructed image quality. Channel specificity can be considered, and a more fine-grained adaptive scaling of the probability distribution parameters is introduced. The edge enhancement method can greatly improve the VMAF index. The characteristics of the bottleneck layer tensor in the end-to-end image coding framework are used. The importance of the channel is determined by the code rate consumed by each channel in the feature domain, and different ways of feature domain enhancement are performed. The probability distribution parameters of the important channels in the feature domain are used to adaptively obtain edge enhancement, improve the quality of the encoded-reconstructed image, and improve the subjective image quality evaluation index.

[0306] For example, each of the above embodiments can be implemented alone or in combination. For example, each of embodiments 1-23 can be implemented alone, and at least two of embodiments 1-23 can be implemented in combination.

[0307] For example, the content of the encoding end in each of the above embodiments can also be applied to the decoding end, that is, the decoding end can be processed in the same way. The content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way.

[0308] Based on the same application concept as the above method, the embodiments of the present application also propose a decoding device. The device is applied to the decoding end, and the device includes a memory configured to store video data, and a decoder configured to implement the decoding method in embodiments 1-23 above, that is, the processing flow of the decoding end.

[0309] Based on the same application concept as the above method, the application embodiment also provides an encoding device. The device is applied to an encoding end. The device comprises a memory configured to store video data; and an encoder configured to implement the encoding method in the above embodiments 1-23, that is, the processing flow of the encoding end.

[0310] Based on the same application concept as the above method, the decoding end device (also referred to as a video decoder) provided by the application embodiment can refer to the hardware architecture schematic diagram shown in FIG. 7 from the hardware level. Figure 7A The decoding end device comprises a processor 711 and a machine readable storage medium 712. The machine readable storage medium 712 stores machine executable instructions capable of being executed by the processor 711. The processor 711 is used to execute the machine executable instructions to implement the decoding method in the above embodiments 1-23 of the application.

[0311] For example, in a possible implementation, when the processor 711 executes the machine executable instructions, it is used to implement:

[0312] decoding a first code stream corresponding to a current image block to obtain a coefficient hyperparameter feature corresponding to the current image block;

[0313] determining a probability distribution parameter based on the coefficient hyperparameter feature, decoding a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block;

[0314] determining an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstruction image block comprises an initial luminance reconstruction image block and an initial chroma reconstruction image block;

[0315] performing luminance edge enhancement on the initial luminance reconstruction image block to obtain a target luminance reconstruction image block.

[0316] Based on the same application concept as the above method, the encoding end device (also referred to as a video encoder) provided by the application embodiment can refer to the hardware architecture schematic diagram shown in FIG. 8 from the hardware level. Figure 7B The encoding end device comprises a processor 721 and a machine readable storage medium 722. The machine readable storage medium 722 stores machine executable instructions capable of being executed by the processor 721. The processor 721 is used to execute the machine executable instructions to implement the encoding method in the above embodiments 1-23 of the application.

[0317] For example, in a possible implementation, when the processor 721 executes the machine executable instructions, it is used to implement:

[0318] Encode the coefficient hyperparameter feature corresponding to the current image block to obtain a first code stream corresponding to the current image block;

[0319] Determine a probability distribution parameter based on the coefficient hyperparameter feature, and encode the initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block;

[0320] Determine an initial reconstructed image block corresponding to the current image block based on the initial image feature; wherein the initial reconstructed image block includes an initial luminance reconstructed image block and an initial chrominance reconstructed image block;

[0321] For each candidate image domain enhancement parameter, perform luminance edge enhancement on the initial luminance reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstructed image block, and determine a generation value corresponding to the candidate image domain enhancement parameter based on the target luminance reconstructed image block;

[0322] Select the image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the generation value corresponding to each candidate image domain enhancement parameter, and encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0323] Based on the same application concept as the above method, the embodiments of the present application provide an electronic device. It includes a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions that can be executed by the processor; the processor is used to execute the machine executable instructions to realize the decoding method or the encoding method of the above embodiments 1-23 of the present application.

[0324] Based on the same application concept as the above method, the embodiments of the present application also provide a machine readable storage medium, the machine readable storage medium stores a plurality of computer instructions, the computer instructions are executed by the processor, and the decoding method or the encoding method disclosed in the above examples of the present application can be realized.

[0325] Based on the same application concept as the above method, the embodiments of the present application also provide a computer application program, which can realize the decoding method or the encoding method disclosed in the above examples of the present application when executed by the processor.

[0326] Based on the same application concept as the above method, the embodiments of the present application also propose a decoding device which can be applied to a decoding end (also referred to as a video decoder), and the decoding device comprises: a decoding module configured to decode a first code stream corresponding to a current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, decode a second code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; a determining module configured to determine an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstruction image block comprises an initial luminance reconstruction image block and an initial chrominance reconstruction image block; an enhancement module configured to perform luminance edge enhancement on the initial luminance reconstruction image block to obtain a target luminance reconstruction image block.

[0327] For example, the enhancement module is further configured to perform chrominance filter enhancement on the initial chrominance reconstruction image block to obtain a target chrominance reconstruction image block; and the decoding device further comprises: a generating module configured to generate a target reconstruction image block based on the target luminance reconstruction image block and the target chrominance reconstruction image block.

[0328] For example, when the enhancement module performs luminance edge enhancement on the initial luminance reconstruction image block to obtain a target luminance reconstruction image block, the enhancement module is specifically configured to: decode a third code stream corresponding to the current image block to obtain an image domain enhancement parameter corresponding to the current image block; and perform luminance edge enhancement on the initial luminance reconstruction image block based on the image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstruction image block corresponding to the current image block.

[0329] For example, the image domain enhancement parameter comprises a plurality of image enhancement segmentation intensity values and a plurality of image enhancement segmentation threshold values, the plurality of image enhancement segmentation threshold values form a plurality of image enhancement threshold intervals, and the plurality of image enhancement threshold intervals correspond to the plurality of image enhancement segmentation intensity values one by one; when the enhancement module performs luminance edge enhancement on the initial luminance reconstruction image block based on the image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstruction image block corresponding to the current image block, the enhancement module is specifically configured to: obtain a target probability distribution channel map based on the probability distribution parameter; if the target probability distribution channel map comprises a plurality of probability distribution values, for each probability distribution value, determine an image enhancement segmentation intensity value corresponding to the probability distribution value based on an image enhancement threshold interval corresponding to the probability distribution value; and perform luminance edge enhancement on the initial luminance reconstruction image block based on the image enhancement segmentation intensity value corresponding to each probability distribution value to obtain a target luminance reconstruction image block.

[0330] Illustratively, when the enhancement module performs the brightness edge enhancement on the initial luminance reconstructed image block to obtain the target luminance reconstructed image block corresponding to the current image block, the enhancement module is specifically configured to: decode a third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameter includes a plurality of probability distribution channel maps, select a probability distribution channel map corresponding to the important channel identifier from the plurality of probability distribution channel maps included in the probability distribution parameter as an important probability distribution channel map; perform up-sampling on the important probability distribution channel map to obtain the target probability distribution channel map; and perform the brightness edge enhancement on the initial luminance reconstructed image block according to the target probability distribution channel map to obtain the target luminance reconstructed image block corresponding to the current image block.

[0331] Illustratively, when the enhancement module performs the brightness edge enhancement on the initial luminance reconstructed image block to obtain the target luminance reconstructed image block corresponding to the current image block based on the image enhancement segment intensity value corresponding to each probability distribution value, the enhancement module is specifically configured to: generate a high-frequency detail image based on the initial luminance reconstructed image block; for each feature value in the high-frequency detail image, perform the brightness edge enhancement on the feature value based on the image enhancement segment intensity value corresponding to the probability distribution value corresponding to the feature value to obtain an image enhancement feature value; and determine the target luminance reconstructed image block based on the image enhancement feature value corresponding to each feature value.

[0332] Illustratively, when the enhancement module generates a high-frequency detail image based on the initial luminance reconstructed image block, the enhancement module is specifically configured to: perform a convolution operation on the initial luminance reconstructed image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; and subtract the Gaussian blur image from the initial luminance reconstructed image block to obtain the high-frequency detail image; wherein for each value in the Gaussian blur convolution kernel, the denominator of the value is a power of 2.

[0333] Illustratively, when the enhancement module performs the chroma filter enhancement on the initial chroma reconstructed image block to obtain a target chroma reconstructed image block, the enhancement module is specifically configured to: perform adaptive up-sampling based on the initial chroma reconstructed image block and the initial luminance reconstructed image block to obtain an up-sampled chroma reconstructed image block, the up-sampled chroma reconstructed image block having the same resolution as the initial luminance reconstructed image block; and perform a filter operation on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filter operation includes cross-channel filtering and / or non-linear chroma filtering.

[0334] For example, when the enhancement module performs filtering operation on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block, the enhancement module is specifically configured to: perform cross-channel filtering on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block; or perform non-linear chroma filtering on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block; or perform cross-channel filtering on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and perform non-linear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block; or perform non-linear chroma filtering on the up-sampled chroma reconstructed image block based on the initial luma reconstructed image block to obtain a non-linear chroma filtered chroma reconstructed image block, and perform cross-channel filtering on the non-linear chroma filtered chroma reconstructed image block based on the initial luma reconstructed image block to obtain the target chroma reconstructed image block.

[0335] For example, when the enhancement module performs chroma filtering enhancement on the initial chroma reconstructed image block to obtain the target chroma reconstructed image block, the enhancement module is specifically configured to: perform adaptive up-sampling based on the initial chroma reconstructed image block and the target luma reconstructed image block to obtain an up-sampled chroma reconstructed image block, the up-sampled chroma reconstructed image block having the same resolution as the target luma reconstructed image block; and perform filtering operation on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block; wherein the filtering operation includes cross-channel filtering and / or non-linear chroma filtering.

[0336] For example, when the enhancement module performs filtering operation on the up-sampled chroma reconstructed image block to obtain the target chroma reconstructed image block, the enhancement module is specifically configured to: perform cross-channel filtering on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block; or perform non-linear chroma filtering on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block; or perform cross-channel filtering on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain a cross-channel filtered chroma reconstructed image block, and perform non-linear chroma filtering on the cross-channel filtered chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block; or perform non-linear chroma filtering on the up-sampled chroma reconstructed image block based on the target luma reconstructed image block to obtain a non-linear chroma filtered chroma reconstructed image block, and perform cross-channel filtering on the non-linear chroma filtered chroma reconstructed image block based on the target luma reconstructed image block to obtain the target chroma reconstructed image block.

[0337] Illustratively, when the determining module determines the initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature, the determining module is specifically configured to: input the initial reconstruction feature into a synthesis transformation network to obtain the initial reconstruction image block; or decode a third code stream corresponding to the current image block to obtain a feature domain enhancement parameter corresponding to the current image block; perform feature domain enhancement on the initial reconstruction feature based on the feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature; and input the enhanced reconstruction feature into the synthesis transformation network to obtain the initial reconstruction image block.

[0338] Illustratively, the decoding module is further configured to decode a third code stream corresponding to the current image block to obtain an important channel identifier corresponding to the current image block; wherein the initial reconstruction feature includes C feature channel maps, and the probability distribution parameter includes C probability distribution channel maps, and the C probability distribution channel maps correspond to the C feature channel maps in a one-to-one manner; the important channel identifier corresponds to a feature channel map as an important feature channel map, and the remaining feature channel maps are non-important feature channel maps; the important channel identifier corresponds to a probability distribution channel map as an important probability distribution channel map, and the remaining probability distribution channel maps are non-important probability distribution channel maps; and when the determining module performs feature domain enhancement on the initial reconstruction feature based on the feature domain enhancement parameter and the probability distribution parameter to obtain an enhanced reconstruction feature, the determining module is specifically configured to: perform feature adaptive edge enhancement on the important feature channel map based on the feature domain enhancement parameter and the important probability distribution channel map to obtain a first reconstruction feature after feature adaptive edge enhancement; perform feature adaptive stretching on the non-important feature channel map based on the feature domain enhancement parameter and the non-important probability distribution channel map to obtain a second reconstruction feature after feature adaptive stretching; and generate the enhanced reconstruction feature based on the first reconstruction feature and the second reconstruction feature. Figure 1

[0339] ​Based on the same application concept as the above method, the embodiment of the present application also proposes an encoding device applied to an encoding end (also referred to as a video encoder), the device comprising: an encoding module configured to encode a coefficient hyperparameter feature corresponding to a current image block to obtain a first code stream corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, encode an initial image feature corresponding to the current image block based on the probability distribution parameter to obtain a second code stream corresponding to the current image block; a determining module configured to determine an initial reconstructed image block corresponding to the current image block based on an initial reconstructed feature corresponding to the initial image feature; the initial reconstructed image block comprises an initial luminance reconstructed image block and an initial chrominance reconstructed image block; an enhancement module configured to, for each candidate image domain enhancement parameter, perform luminance edge enhancement on the initial luminance reconstructed image block based on the candidate image domain enhancement parameter and the probability distribution parameter to obtain a target luminance reconstructed image block; the determining module is further configured to determine a generation value corresponding to the candidate image domain enhancement parameter based on the target luminance reconstructed image block; select an image domain enhancement parameter corresponding to the current image block from all candidate image domain enhancement parameters based on the generation value corresponding to each candidate image domain enhancement parameter; and the encoding module is further configured to encode the image domain enhancement parameter to obtain a third code stream corresponding to the current image block.

[0340] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. The present application can be in the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. The embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. The above is only an embodiment of the present application and is not intended to limit the present application.

[0341] The present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. An image decoding method characterized by, The method comprises: decoding a code stream corresponding to a current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determining a probability distribution parameter based on the coefficient hyperparameter feature, and decoding the code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; determining an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature, wherein the initial reconstruction image block comprises an initial luminance reconstruction image block; Convolution is performed on the initial brightness reconstruction image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; wherein the Gaussian blur convolution kernel is an N*N convolution kernel, and if the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel includes: ; wherein A is a positive integer, B is a positive integer, and M is a power of 2. subtracting the initial luminance reconstruction image block from the Gaussian blur image to obtain a high-frequency detail image; determining a target luminance reconstruction image block corresponding to the current image block based on the high-frequency detail image; generating a target reconstruction image block corresponding to the current image block based on the target luminance reconstruction image block.

2. The method of claim 1, wherein, The Gaussian blur convolution kernel is: 。 3. The method of claim 1, wherein the determining of the target luminance reconstruction image block corresponding to the current image block based on the high-frequency detail image comprises: obtaining an image domain enhancement parameter corresponding to the current image block; performing luminance edge enhancement on the high-frequency detail image based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block to obtain the target luminance reconstruction image block corresponding to the current image block.

4. The method of claim 3, wherein, The image domain enhancement parameter comprises a plurality of image enhancement segmentation intensity values and a plurality of image enhancement segmentation threshold values, the plurality of image enhancement segmentation threshold values form a plurality of image enhancement threshold intervals, and the plurality of image enhancement threshold intervals correspond to the plurality of image enhancement segmentation intensity values one by one. The performing of the luminance edge enhancement on the high-frequency detail image based on the image domain enhancement parameter and the probability distribution parameter corresponding to the current image block to obtain the target luminance reconstruction image block comprises: obtaining a target probability distribution channel map based on the probability distribution parameter; if the target probability distribution channel map comprises a plurality of probability distribution values, for each probability distribution value, determining an image enhancement segmentation intensity value corresponding to the probability distribution value based on an image enhancement threshold interval corresponding to the probability distribution value; for each feature value in the high-frequency detail image, performing luminance edge enhancement on the feature value based on an image enhancement segmentation intensity value corresponding to a probability distribution value corresponding to the feature value to obtain an image enhancement feature value; and determining the target luminance reconstruction image block based on the image enhancement feature value corresponding to each feature value.

5. The method of claim 4, wherein the obtaining of the target probability distribution channel map based on the probability distribution parameter comprises: decoding a code stream corresponding to a current image block to obtain an important channel identifier corresponding to the current image block; if the probability distribution parameter comprises a plurality of probability distribution channel maps, selecting a probability distribution channel map corresponding to the important channel identifier from the plurality of probability distribution channel maps of the probability distribution parameter as an important probability distribution channel map; performing up-sampling on the important probability distribution channel map to obtain the target probability distribution channel map.

6. The method of any one of claims 1-5, wherein the determining of the initial reconstruction image block corresponding to the current image block comprises: decode the code stream corresponding to the current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; determine an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature.

7. An image coding method characterized by, The method comprises: decode the code stream corresponding to the current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; determine an initial reconstruction image block corresponding to the current image block based on the initial reconstruction feature; wherein the initial reconstruction image block comprises an initial luminance reconstruction image block; Convolution is performed on the initial brightness reconstruction image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; wherein the Gaussian blur convolution kernel is an N*N convolution kernel, and if the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel includes: ; wherein A is a positive integer, B is a positive integer, and M is a power of 2. subtract the Gaussian blur image from the initial luminance reconstruction image block to obtain a high-frequency detail image; determine a target luminance reconstruction image block corresponding to the current image block based on the high-frequency detail image; generate a target reconstruction image block corresponding to the current image block based on the target luminance reconstruction image block.

8. An image decoding apparatus characterized by comprising: The apparatus comprises: a decoding module configured to decode the code stream corresponding to the current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; A determining module is configured to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature; wherein the initial reconstructed image block comprises an initial luma reconstructed image block; and a convolution operation is performed on the initial luma reconstructed image block and a Gaussian blur convolution kernel to obtain a Gaussian blur image; wherein the Gaussian blur convolution kernel is an N*N convolution kernel, and if the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel comprises: ; wherein A is a positive integer, B is a positive integer, and M is a power of 2; the initial luma reconstructed image block is subtracted by the Gaussian blur image to obtain a high-frequency detail image; and a target luma reconstructed image block corresponding to the current image block is determined based on the high-frequency detail image. a generating module configured to generate a target reconstruction image block corresponding to the current image block based on the target luminance reconstruction image block.

9. An image coding apparatus characterized by comprising: The apparatus comprises: a decoding module configured to decode the code stream corresponding to the current image block to obtain a coefficient hyperparameter feature corresponding to the current image block; determine a probability distribution parameter based on the coefficient hyperparameter feature, and decode the code stream corresponding to the current image block based on the probability distribution parameter to obtain an initial reconstruction feature corresponding to the current image block; The determining module is configured to determine an initial reconstructed image block corresponding to the current image block based on the initial reconstructed feature; wherein the initial reconstructed image block comprises an initial luma reconstructed image block; and the initial luma reconstructed image block is subjected to a convolution operation with a Gaussian blur convolution kernel to obtain a Gaussian blur image; wherein the Gaussian blur convolution kernel is an N*N convolution kernel, and if the Gaussian blur convolution kernel is a 3*3 convolution kernel, the Gaussian blur convolution kernel comprises: ; wherein A is a positive integer, B is a positive integer, and M is a power of 2; the initial luma reconstructed image block is subtracted by the Gaussian blur image to obtain a high-frequency detail image; and a target luma reconstructed image block corresponding to the current image block is determined based on the high-frequency detail image. a generating module configured to generate a target reconstruction image block corresponding to the current image block based on the target luminance reconstruction image block.

10. An image decoding apparatus characterized by comprising: The image decoding device comprises a processor and a machine readable storage medium, and the machine readable storage medium stores machine executable instructions executable by the processor; the processor is configured to execute the machine executable instructions to implement the method of any one of claims 1-6.

11. An image coding apparatus characterized by comprising: The image encoding device comprises a processor and a machine readable storage medium, and the machine readable storage medium stores machine executable instructions executable by the processor; the processor is configured to execute the machine executable instructions to implement the method of claim 7.

12. A machine-readable storage medium, characterized in that, The machine readable storage medium stores a plurality of computer instructions, and the computer instructions are executed by the processor to implement the method of any one of claims 1-6, or the computer instructions are executed by the processor to implement the method of claim 7.

13. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7. The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images