Image decoding and coding method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of image processing, and discloses an image decoding and coding method and device, equipment and a storage medium. The method comprises the following steps: obtaining feature channel distinguishing information and / or spatial point distinguishing information, and decoding an image code stream to obtain a quantized residual value; constructing an inverse quantization precision parameter corresponding to each quantization residual value according to the feature channel distinguishing information and / or the spatial point distinguishing information; performing inverse quantization on the quantized residual value based on the inverse quantization precision parameter to obtain a reconstructed residual value; and performing synthesis transformation on the reconstructed residual value to obtain a reconstructed image block. As the corresponding inverse quantization parameters are constructed according to the feature channel distinguishing information and / or the space point distinguishing information for inverse quantization, different quantization parameters can be set according to the difference between the feature channel image texture and the space point image texture in the encoding process, and more important features bear smaller quantization loss; therefore, the coding and decoding performance is improved.

Description

[0001] This invention patent application is a divisional application of Chinese invention patent application filed on March 1, 2023, with application number 202310209226.1 and titled "Image Decoding and Encoding Method, Apparatus, Device and Storage Medium". Technical Field

[0002] This invention relates to the field of image processing technology, and in particular to an image decoding and encoding method, apparatus, device, and storage medium. Background Technology

[0003] Currently, end-to-end image encoding and decoding technologies generally include modules such as analytical transform networks, synthetic transform networks, context-based prediction, quantization, entropy coding, super-coding networks, and superscale decoding networks. Among them, quantization is a "many-to-one" mapping process that introduces signal loss. It acts on the residual and can change the range of signal values, enabling the encoder to provide a good approximation of the original signal with a small number of symbols, thereby improving the compression ratio. However, existing quantization modules do not consider the differences in feature channels and image textures during quantization, thus limiting the quantization performance of the quantizer.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide an image decoding and encoding method, apparatus, device, and storage medium, aiming to solve the technical problem of poor quantizer performance in the image encoding and decoding process in the prior art.

[0006] To achieve the above objectives, the present invention provides an image decoding method, the method comprising the following steps: Acquire feature channel differentiation information and / or spatial point differentiation information, and decode the image bitstream to obtain quantization residual values; Construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information; Based on the inverse quantization precision parameter, the quantization residual value is inversely quantized to obtain the reconstructed residual value; The reconstructed residual values ​​are synthesized and transformed to obtain reconstructed image blocks.

[0007] In one possible implementation of this application, the image stream includes a first image stream and a second image stream, wherein the first image stream is used to transmit decoding auxiliary information and the second image stream is used to transmit residual data; The step of decoding the image bitstream to obtain the quantization residual value includes: Extract coding distribution parameters from the first image bitstream; The encoded distribution parameters are decoded to obtain the probability distribution parameters; The probability distribution parameters are quantized to obtain the distribution quantization parameters; The second image bitstream is decoded based on the distributed quantization parameters to obtain the quantization residual value.

[0008] In one possible implementation of this application, the step of quantizing the probability distribution parameters to obtain distribution quantization parameters includes: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

[0009] In one possible implementation of this application, the step of obtaining feature channel differentiation information and / or spatial point differentiation information includes: Decode the first image stream or the second image stream to obtain feature channel differentiation information and / or spatial point differentiation information.

[0010] In one possible implementation of this application, the image stream further includes a third image stream, which is used to transmit distinguishing information; The steps of obtaining feature channel differentiation information and / or spatial point differentiation information include: The third image stream is decoded to obtain feature channel differentiation information and / or spatial point differentiation information.

[0011] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0012] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Extract the quantization step size corresponding to each type of spatial point from the spatial point differentiation information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0013] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in each type of spatial point is read from the feature channel differentiation information. Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0014] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the quantization step size includes: Decode the image bitstream to obtain probability distribution parameters; Extract parameter interval thresholds from the feature channel differentiation information; Based on the probability distribution parameters, the quantization step size, and the parameter interval threshold, inverse quantization accuracy parameters corresponding to each quantization residual value are constructed.

[0015] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Extract the image block information and parameter setting rules corresponding to each feature channel from the feature channel differentiation information; Decode the image bitstream to obtain probability distribution parameters; Calculate the mean value of the distribution parameters corresponding to each image block information based on the probability distribution parameters; The mean of the distribution parameters is matched with the parameter setting rules to obtain the quantization step size corresponding to each image block information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0016] In one possible implementation of this application, the step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: The feature channel differentiation information is parsed to obtain the segmented step size set corresponding to each type of feature channel; Extract the quantization step size corresponding to each channel segment from the set of segmented step sizes; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0017] Furthermore, to achieve the above objectives, the present invention also proposes an image encoding method, which includes the following steps: The image blocks to be processed are analyzed and transformed to obtain image features; The image features are subjected to residual calculation to obtain image residual values ​​and probability distribution parameters; Construct quantization precision parameters corresponding to the residual values ​​of each image based on feature channel differentiation information and / or spatial point differentiation information; The image residual value is quantized based on the quantization accuracy parameter to obtain the quantization residual value; The probability distribution parameters and the quantization residual values ​​are written into the image bitstream.

[0018] In one possible implementation of this application, the image stream includes a first image stream and a second image stream, wherein the first image stream is used to transmit decoding auxiliary information and the second image stream is used to transmit residual data; The step of writing the probability distribution parameters and the quantization residual values ​​into the image bitstream includes: Write the probability distribution parameters into the first image bitstream; Construct distribution quantization parameters based on the probability distribution parameters; The quantization residual value is written into the second image bitstream based on the distributed quantization parameters.

[0019] In one possible implementation of this application, the step of constructing distribution quantization parameters based on the probability distribution parameters includes: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

[0020] In one possible implementation of this application, after the step of writing the quantization residual value into the second image bitstream based on the distributed quantization parameters, the method further includes: Write the feature channel differentiation information and / or the spatial point differentiation information into the first image stream or the second image stream.

[0021] In one possible implementation of this application, the image stream further includes a third image stream, which is used to transmit distinguishing information; After the step of writing the quantization residual value into the second image bitstream based on the distributed quantization parameters, the method further includes: Write the feature channel differentiation information and / or the spatial point differentiation information into the third image bitstream.

[0022] In one possible implementation of this application, the step of constructing quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0023] In one possible implementation of this application, the step of constructing quantization accuracy parameters corresponding to each image residual value based on the quantization step size includes: Extract parameter interval thresholds from the feature channel differentiation information; The quantization accuracy parameters corresponding to each image residual value are constructed based on the probability distribution parameters, the parameter interval threshold, and the quantization step size.

[0024] In one possible implementation of this application, the step of constructing quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: The feature channel differentiation information is parsed to obtain the segmented step size set corresponding to each type of feature channel; Extract the quantization step size corresponding to each channel segment from the set of segmented step sizes; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0025] In one possible implementation of this application, the step of constructing quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Extract the quantization step size corresponding to various types of spatial points from spatial point differentiation information; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0026] In one possible implementation of this application, the step of constructing quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in various spatial points is read from the feature channel differentiation information. Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0027] In one possible implementation of this application, the step of writing the probability distribution parameters and the quantization residual values ​​into the image bitstream includes: Image decoding is performed based on the probability distribution parameters, the quantization residual value, and the feature channel differentiation information to obtain reconstructed image blocks and image bitrate. The feature channel differentiation information and / or spatial point differentiation information are adjusted based on the reconstructed image blocks and the image bitrate; If the current adjustment round is greater than or equal to the preset adjustment round, then the probability distribution parameter and the quantization residual value are written into the image bitstream.

[0028] In one possible implementation of this application, after the step of adjusting the feature channel discrimination information and / or spatial point discrimination information according to the reconstructed image block and the image bitrate, the method further includes: If the current adjustment round is less than the preset adjustment round, then return to the step of constructing the quantization accuracy parameters corresponding to each feature channel based on the feature channel differentiation information.

[0029] Furthermore, to achieve the above objectives, the present invention also proposes an image decoding device, the image decoding device comprising: The entropy decoding module is used to obtain feature channel differentiation information and / or spatial point differentiation information, and to decode the image bitstream to obtain quantization residual values; The parameter construction module is used to construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information. The dequantization module is used to dequantize the quantization residual value based on the dequantization accuracy parameter to obtain the reconstructed residual value; The synthesis transformation module is used to perform a synthesis transformation on the reconstructed residual values ​​to obtain reconstructed image blocks.

[0030] Furthermore, to achieve the above objectives, the present invention also proposes an image encoding device, the image encoding device comprising: The analysis and transformation module is used to perform analysis and transformation processing on the image blocks to be processed to obtain image features; The residual calculation module is used to perform residual calculation on the image features to obtain image residual values ​​and probability distribution parameters; The parameter construction module is used to construct the quantization accuracy parameters corresponding to each feature channel based on feature channel differentiation information and / or spatial point differentiation information. The quantization module is used to perform quantization processing on the image residual value based on the quantization accuracy parameter to obtain the quantization residual value; The entropy coding module is used to write the probability distribution parameters and the quantization residual values ​​into the image bitstream.

[0031] In addition, to achieve the above objectives, the present invention also proposes a decoding device, the image decoding device comprising: a processor, a memory, and a decoding program stored in the memory and executable on the processor, wherein the decoding program, when executed by the processor, implements the image decoding method as described above.

[0032] In addition, to achieve the above objectives, the present invention also proposes an encoding device, which includes: a processor, a memory, and an encoding program stored in the memory and executable on the processor. When the encoding program is executed by the processor, it implements the image encoding method described above.

[0033] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium on which an image decoding program and / or an image encoding program are stored; When the image decoding program is executed, it implements the image decoding method as described above; or, when the image encoding program is executed, it implements the image encoding method as described above.

[0034] This invention obtains quantization residual values ​​by acquiring feature channel differentiation information and / or spatial point differentiation information and decoding the image bitstream. Based on the feature channel differentiation information and / or the spatial point differentiation information, it constructs inverse quantization precision parameters corresponding to each quantization residual value. Based on the inverse quantization precision parameters, it performs inverse quantization on the quantization residual values ​​to obtain reconstructed residual values. Finally, it performs a synthetic transformation on the reconstructed residual values ​​to obtain reconstructed image blocks. Because inverse quantization is performed by constructing corresponding inverse quantization parameters based on feature channel differentiation information and / or spatial point differentiation information, different quantization parameters can be set according to the differences in image texture between feature channels and spatial points during the encoding process. This allows more important features to bear less quantization loss, thereby improving the performance of encoding and decoding. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the image decoding method of the present invention; Figure 3 This is a schematic diagram of an image encoding and decoding process according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a feature channel according to an embodiment of the present invention; Figure 5 This is a two-dimensional schematic diagram of the average value of the feature channel probability distribution parameters according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the second embodiment of the image decoding method of the present invention; Figure 7This is a schematic diagram of the decoding process according to an embodiment of the present invention; Figure 8 This is a flowchart illustrating the third embodiment of the image decoding method of the present invention; Figure 9 This is a schematic diagram of an identifier matrix according to an embodiment of the present invention; Figure 10 This is a schematic diagram of row and column division according to an embodiment of the present invention; Figure 11 This is a flowchart illustrating the fourth embodiment of the image decoding method of the present invention; Figure 12 This is a schematic diagram of spatial point classification according to an embodiment of the present invention; Figure 13 This is a flowchart illustrating the first embodiment of the image encoding method of the present invention; Figure 14 This is a schematic diagram of an image encoding process according to an embodiment of the present invention; Figure 15 This is a flowchart illustrating the second embodiment of the image encoding method of the present invention; Figure 16 This is a flowchart illustrating the third embodiment of the image encoding method of the present invention; Figure 17 This is a schematic diagram illustrating the function of the optimal threshold mode flag bit in an embodiment of the present invention; Figure 18 This is a flowchart of parameter optimization according to an embodiment of the present invention; Figure 19 This is a structural block diagram of the first embodiment of the image decoding device of the present invention; Figure 20 This is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0036] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0037] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0038] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of the encoding or decoding device in the hardware operating environment involved in the embodiments of the present invention.

[0039] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0040] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0041] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an image encoding program and / or an image decoding program.

[0042] exist Figure 1 In the illustrated electronic device, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the electronic device of the present invention can be set in the encoding device or the decoding device. The electronic device calls the encoding program stored in the memory 1005 through the processor 1001 and executes the image encoding method provided in the embodiment of the present invention, or calls the image decoding program stored in the memory 1005 through the processor 1001 and executes the image decoding method provided in the embodiment of the present invention.

[0043] In some embodiments of this application, the performance of the quantizer is improved by considering the differences in feature channels to enhance quantization accuracy. For example, feature channels can be classified according to their differences, with different importance channels having different quantization accuracies; features can also be classified according to their differences in texture complexity, with different texture complexities having different quantization accuracies. In this application, all information used for feature channel classification is related to distinguishing feature channels. Features classified by the above information will be quantized with different accuracies, i.e., different quantization step sizes will be used so that more important features bear less quantization loss, making the prediction of the feature to be encoded more accurate, thereby obtaining a better feature reconstruction value y_hat, and ultimately achieving the goal of obtaining better encoding performance.

[0044] This invention provides an image decoding method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of an image decoding method according to the present invention.

[0045] In this embodiment, the image decoding method includes the following steps: Step S10: Obtain feature channel differentiation information and / or spatial point differentiation information, and decode the image bitstream to obtain quantization residual values.

[0046] It should be noted that the execution subject of this embodiment can be a decoding device when decoding image data. The decoding device can be an electronic device such as a camera, mobile terminal, personal computer, or server. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method of the present invention is described using a decoding device as an example.

[0047] Since the encoding device typically decodes the encoded image stream after encoding is completed during the image encoding process, and determines whether the parameters used in the encoding need to be adjusted based on the image quality of the decoded image, the execution subject in this embodiment can also be the encoding device.

[0048] Some technical terms that may be involved in the encoding or decoding of images include: quantization and dequantization, scalar quantization (SQ), entropy encoding, convolutional neural network (CNN), and feature channel, which will be explained here.

[0049] Quantization refers to the process of mapping continuous (or numerous discrete) values ​​of a signal to a finite number of discrete amplitudes, achieving a many-to-one mapping of signal values. In video coding, the transform coefficients of the residual signal often have a large dynamic range after transformation. Therefore, quantizing the transform coefficients can effectively reduce the signal value space and achieve better compression results. However, due to the many-to-one mapping mechanism, the quantization process inevitably introduces distortion, which is the root cause of distortion in video coding.

[0050] Dequantization is the reverse process of quantization. It maps the quantized coefficients to a reconstructed signal in the input signal space, which is an approximation of the input signal.

[0051] Scalar quantization is the most basic quantization method. It maps a continuous signal (or a signal with a large number of discrete values) into several discrete signals. Specifically, the input of scalar quantization is a one-dimensional scalar signal. It first divides the input signal space into a series of non-overlapping intervals, and selects a representative signal for each interval. Then, for each input signal, scalar quantization maps it to the representative signal of its corresponding interval.

[0052] The simplest scalar quantization method is uniform scalar quantization, which divides the input signal space into equally spaced intervals, with the midpoint of each interval representing the signal. The interval length is called the quantization step, the interval index is called the level, and the parameter characterizing the quantization step is the quantization parameter (QP).

[0053] The optimal scalar quantizer is the Lloyd-Max quantizer, which takes into account the distribution of the input signal. The interval division is non-uniform, the representative signal of each interval is the probability centroid of that interval, and the boundary point of two adjacent intervals is the midpoint of the representative signals of those two intervals.

[0054] Entropy coding is encoding that follows the entropy principle without losing any information during the encoding process. Information entropy is the average amount of information in a source (a measure of uncertainty). Common entropy coding methods include Shannon coding, Huffman coding, and arithmetic coding.

[0055] Neural Network (NN): The neural network discussed here refers to an artificial neural network, not a biological neural network. A neural network is a computational model composed of numerous interconnected nodes (or neurons). In an artificial neural network, neurons can represent different objects, such as features, letters, concepts, or meaningful abstract patterns. There are three types of processing units in a network: input units, output units, and hidden units. Input units receive signals and data from the external world; output units output the system's processing results; hidden units are located between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the strength of the connections between units; the representation and processing of information are reflected in the connections between the network's processing units. Artificial neural networks are a non-programmed, brain-like information processing method. Essentially, they achieve parallel and distributed information processing capabilities through network transformations and dynamic behaviors, mimicking the information processing functions of the human brain's nervous system to varying degrees and levels. Currently, in the field of video processing, commonly used neural networks include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and fully connected networks.

[0056] Convolutional neural networks (CNNs) are a type of feedforward neural network and one of the most representative network structures in deep learning technology. Their artificial neurons can respond to a portion of the surrounding units within their coverage area, making them excellent for large-scale image processing.

[0057] Generally, the basic structure of a CNN consists of two layers. The first is the feature extraction layer (also called a convolutional layer), where the input of each neuron is connected to the local receptive field of the previous layer, extracting the local features. Once the local features are extracted, their positional relationship with other features is determined. The second is the feature mapping layer (also called an activation layer). Each computational layer of the network consists of multiple feature maps, each of which is a plane where all neurons have equal weights. Feature mapping structures can use functions such as the Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, and GDN function as activation functions for convolutional networks. Furthermore, because neurons on a single feature map share weights, the number of free parameters in the network is reduced.

[0058] One of the advantages of CNNs over traditional image processing algorithms is that they avoid complex preprocessing steps (such as extracting manual features) and can directly input the original image for end-to-end learning. Another advantage of CNNs over traditional neural networks is that traditional neural networks use fully connected layers, meaning all neurons from the input layer to the hidden layers are connected. This results in a huge number of parameters, making network training time-consuming or even difficult. CNNs, however, avoid this difficulty through local connectivity and weight sharing.

[0059] In neural networks, the input signal passes through convolutional layers to obtain many feature maps. Each feature map is called a feature channel, which contains a part of the information of the input signal.

[0060] It should be noted that feature channel discrimination information can be used to classify image feature channels. This information may include the rules for classifying feature channels and the quantization precision parameters for quantizing the residual values ​​corresponding to each type of feature channel after classification. Spatial point discrimination information can be used to classify points in an image. This information may include the rules for classifying spatial points in the image based on image texture. If necessary, the spatial point discrimination information may also include the quantization precision parameters for quantizing the residual values ​​corresponding to each type of spatial point after classification.

[0061] The feature channel or spatial point division rules can be set based on probability distribution parameters, or based on the mean or variance of the probability distribution parameters. For example, a corresponding threshold can be set to obtain multiple value intervals, and the feature channel can be divided into multiple categories according to the value interval to which the corresponding probability distribution parameter belongs.

[0062] Step S20: Construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information.

[0063] It should be noted that constructing the corresponding feature channel category or spatial point category for each quantization residual value based on feature channel differentiation information and / or spatial point differentiation information can be based on determining the feature channel category or spatial point category corresponding to each quantization residual value based on feature channel differentiation information and / or spatial point information, thereby obtaining the inverse quantization precision parameter that should be used when performing inverse quantization processing on each quantization residual value.

[0064] Step S30: Perform dequantization on the quantization residual value based on the dequantization accuracy parameter to obtain the reconstructed residual value.

[0065] It is understandable that after determining the inverse quantization precision parameters corresponding to each quantization residual value, the quantization residual value can be subjected to the inverse quantization process according to the inverse quantization precision parameters, that is, inverse quantization can be performed, and the value obtained after processing can be used as the reconstructed residual value.

[0066] Step S40: Perform a synthetic transformation on the reconstructed residual values ​​to obtain reconstructed image blocks.

[0067] It should be noted that after obtaining the reconstructed residual values, the mean value used to calculate the feature residual values ​​can be obtained through context prediction. The reconstructed residual values ​​can be reconstructed into image features through this mean value. Subsequently, the image features can be reconstructed into image patches through the synthetic transformation network, thereby obtaining the reconstructed image patches.

[0068] Understandably, if the encoding device processes the image data into only one image block, the resulting reconstructed image block will be the complete reconstructed image data corresponding to the original image data. However, if the encoding device processes the image data into multiple image blocks, the resulting reconstructed image block will only be the reconstructed image data corresponding to a single image block within the original image data.

[0069] To facilitate understanding, we will now combine... Figure 3 , 4 The following points will be explained, but will not limit the scope of this scheme. Figure 3 This is a schematic diagram of the image encoding and decoding process in this embodiment. Figure 3 As shown, the image encoding and decoding process mainly involves modules such as analysis transform network, synthesis transform network, context-based prediction, quantization, entropy coding, super coding network, and superscale decoding network, among which: Analysis of the transform network: Transforms the image x to the feature y in the feature domain (latent domain), so that all subsequent processes can be operated in the latent domain; Super-encoding network: Input the probability distribution parameters σ of the residual r to obtain the prior information z_hat. z_hat will be written into the image bitstream biasstream#1, which mainly transmits the probability model parameters σ of the residual.

[0070] Context-based prediction module: The input of this module includes not only z_hat, but also the decoded y_hat. The two are combined as input to obtain a more accurate mean mu, which is the predicted value of the original feature. The reconstructed y_hat is obtained by adding the reconstructed residual value r_hat.

[0071] Superscale decoding network: The output z_hat of the super-encoding network is used as the input of the superscale decoding network to obtain the probability distribution parameter σ. After quantization, the distribution quantization parameter σ_hat is obtained. σ_hat will be used to obtain the quantization residual value r_q by combining with the second image bitstream (Bitstream#2).

[0072] Synthesis Transformation Module: The obtained reconstructed latent domain feature y_hat is processed by the synthesis transformation module to obtain the reconstructed image.

[0073] Entropy coding: a lossless coding method based on the principle of information entropy, which transforms a series of element symbols used to represent a video sequence (such as transform coefficients and mode information) into a binary bitstream, removing the statistical redundancy of these video element symbols.

[0074] Quantization, a many-to-one mapping process, introduces signal loss and affects the residual r. It alters the signal's value range, allowing the encoder to approximate the original signal with fewer symbols, thus improving compression ratio. Furthermore, quantization affects σ, yielding σ_hat, which helps to better estimate the probability distribution of the quantization residual r_q. Dequantization is the inverse process of quantization. The quantization or dequantization precision parameters used in quantization and dequantization are constructed based on feature channel discrimination information and / or spatial point discrimination information.

[0075] Figure 4 This is a schematic diagram of the feature channel in this embodiment. Figure 5 This is a two-dimensional schematic diagram of the average value of the feature channel probability distribution parameters in this embodiment, as described above. Figure 3 As shown, after image x is processed by the analysis transform network, the resulting feature y contains C feature channels. The number of C channels depends on the number of convolutional kernels used by the analysis transform network, such as... Figure 4 As shown, each feature channel contains a different amount of information. Some feature channels contain a lot of information, almost identical to the input image, while others contain almost no information and are mostly noise. Therefore, using different precision quantization for different feature channels helps to obtain better coding performance.

[0076] Furthermore, averaging the σ values ​​of all feature channels yields a 2D image, such as... Figure 5 As shown, in Figure 5 In the process, it can be determined that some regions have higher texture values, meaning that these regions are more important. Allocating more codewords to the complex texture parts of the image helps improve encoding performance.

[0077] This embodiment obtains quantization residual values ​​by acquiring feature channel differentiation information and / or spatial point differentiation information and decoding the image bitstream. Based on the feature channel differentiation information and / or the spatial point differentiation information, it constructs inverse quantization precision parameters corresponding to each quantization residual value. Based on the inverse quantization precision parameters, it performs inverse quantization on the quantization residual values ​​to obtain reconstructed residual values. Finally, it performs a synthetic transformation on the reconstructed residual values ​​to obtain reconstructed image patches. Because inverse quantization is performed by constructing corresponding inverse quantization parameters based on feature channel differentiation information and / or spatial point differentiation information, different quantization parameters can be set according to the differences in image texture between feature channels and spatial points during the encoding process. This allows more important features to bear less quantization loss, thereby improving the performance of encoding and decoding.

[0078] refer to Figure 6 , Figure 6 This is a flowchart illustrating a second embodiment of an image decoding method according to the present invention.

[0079] In this embodiment, the image stream may include a first image stream and a second image stream, wherein the first image stream is used to transmit decoding auxiliary information and the second image stream is used to transmit residual data.

[0080] Based on the first embodiment described above, step S10 of the image decoding method in this embodiment includes: Step S101: Obtain feature channel differentiation information and / or spatial point differentiation information, and extract coding distribution parameters from the first image bitstream.

[0081] It should be noted that extracting the coding distribution parameters from the first image bitstream can be done by performing entropy decoding on the first image bitstream and reading the decoding auxiliary information contained in the first image bitstream, thereby obtaining the coding distribution parameters.

[0082] Step S102: Decode the encoded distribution parameters to obtain the probability distribution parameters.

[0083] It should be noted that the probability distribution parameters can be obtained by decoding the encoded distribution parameters through a superscale decoding network to decode the read decoding auxiliary information.

[0084] Step S103: Quantize the probability distribution parameters to obtain distribution quantization parameters.

[0085] It should be noted that when the encoding device writes the residual data into the second image bitstream, it does so based on the quantized probability distribution parameters. Therefore, when decoding the second image bitstream, it is also necessary to obtain the quantized probability distribution parameters for decoding. In order to ensure that the second image bitstream can be decoded normally, the probability distribution parameters can be quantized in the same way as when the encoding device encodes, to obtain the distribution quantization parameters.

[0086] In the specific implementation process, in order to ensure that the obtained distribution quantization parameters are consistent with those used by the encoding device, it is necessary to ensure that the parameters used when quantizing the probability distribution parameters are consistent with those used by the encoding device. Step S103 in this embodiment may include: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

[0087] It should be noted that the probability quantization parameters constructed based on feature channel differentiation information and / or spatial point differentiation information can be the parameters extracted from the feature channel differentiation information and / or spatial point differentiation information used when quantizing the probability distribution parameters.

[0088] In one possible implementation of this embodiment, in order to facilitate decoding by the decoding device, the encoding device can write the feature channel differentiation information and / or spatial point differentiation information used in the encoding process into the first image bitstream or the second image bitstream. In this case, obtaining the feature channel differentiation information and / or spatial point differentiation information can be achieved by decoding the first image bitstream or the second image bitstream and extracting the feature channel differentiation information and / or spatial point differentiation information from it.

[0089] In one possible implementation of this embodiment, in order to avoid code stream confusion, the encoding device may also set a third image code stream, and transmit the feature channel differentiation information and / or spatial point differentiation information used in the encoding process through the third image code stream. In this case, obtaining the feature channel differentiation information and / or spatial point differentiation information can be done by decoding the third image code stream and extracting the feature channel differentiation information and / or spatial point differentiation information from it.

[0090] Step S104: Decode the second image bitstream based on the distributed quantization parameters to obtain the quantization residual value.

[0091] It should be noted that when the encoding device writes the quantization residual value into the second image bitstream, it performs entropy encoding on the quantization residual value based on the quantized distributed quantization parameters. Therefore, after obtaining the distributed quantization parameters, the second image bitstream can be reversed according to the distributed quantization parameters, i.e., entropy decoding, to obtain the quantization residual value.

[0092] To facilitate understanding, we will now combine... Figure 7 This explanation is provided, but it does not limit the scope of this solution. Figure 7 This is a schematic diagram of the decoding process in this embodiment, as shown below. Figure 7 As shown, the decoding segment will receive at least two image streams (the first image stream Bitstream#1 and the second image stream Bitstream#2), and may also receive a third image stream Bitstream#3; At this point, the decoding process includes: parsing a z_hat from the Bitstream#1 auxiliary bitstream, through which the parameters σ of the probability distribution model of the residual can be obtained; obtaining relevant information for distinguishing feature channels or texture regions (feature channel distinction information and / or spatial point distinction information), using this information to classify feature channels or spatial points, with different categories of feature channels or different categories of spatial points having different quantization precisions; the relevant information for distinguishing feature channels can be transmitted from the encoding device to the decoding device through Bitstream#1 or Bitsstream#2, or it can be transmitted through a new bitstream Bitsstream#3, or the relevant information can be pre-written into the encoding and decoding devices without bitstream transmission; performing entropy decoding from Bitsstream#2, and obtaining the quantization residual r_q by combining σ with the σ_hat obtained after quantization; the relevant information for distinguishing feature channels or spatial points will be applied to the dequantization of the quantization residual value r_q and to make the quantization residual values ​​r_q and σ corresponding to different categories of feature channels or different categories of spatial points have different quantization precisions.

[0093] refer to Figure 8 , Figure 8 This is a flowchart illustrating a third embodiment of an image decoding method according to the present invention.

[0094] Based on the first embodiment described above, step S20 of the image decoding method in this embodiment includes: Step S201: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information.

[0095] It should be noted that when distinguishing feature channels, feature channels can be divided into multiple categories, and different quantization step sizes can be assigned to feature channels of different categories. In this case, the feature channel distinction information will include the classification rules of feature channels and the quantization step size corresponding to each type of feature channel.

[0096] The classification rules for the feature channels can be set as follows: 1. Configure according to the feature channel number: Set a corresponding quantization step size for each feature channel corresponding to each index, and then group feature channels with the same quantization step size into one category. For example, if the feature channel index is 1-X, then the feature channel quantization step size list can be: channel_list = [channel1, channel2, ..., channelX]; Furthermore, the same feature channel may also include components such as luminance or chrominance. In this case, luminance or chrominance can be further subdivided. For example, assuming the luminance channel number is 1-X1 and the chrominance channel number is 1-X2, the quantization step size list for the luminance channel is: channel_listY = [channel1, channel2, ..., channelX1], and the quantization step size list for the chrominance channel is: channel_listUV = [channel1, channel2, ..., channelX2].

[0097] 2. Set the corresponding categories based on the channel flag list; A corresponding flag is set for each feature channel corresponding to each serial number, and the category of the feature channel corresponding to each serial number is identified by the flag.

[0098] For example, taking the luminance channel as an example, the luminance channel number is 1-128, and the flag list channel_listY = [flag0,flag1,、、、,flag128], where the value of flag is the category corresponding to the luminance channel of that number. If there are only two categories, the value of flag is 0 and 1.

[0099] 3. Classify feature channels by setting thresholds: The corresponding feature channel category is set by using the threshold calculated from the probability distribution parameter σ (mean or variance), the threshold calculated from the quantization residual r_q (sum of absolute values ​​of quantization residuals or bit rate), for example: thr_list = [thr1, thr2, ..., thrX].

[0100] 4. Classify feature channels by channel number: Sort the channels by the relevant threshold list calculated from the distribution parameter σ using the mean or variance of σ, or the sum of the absolute values ​​of the quantization residuals r_q or the bit rate, and set multiple number thresholds (multiple number thresholds can be represented by a list, such as: num_list=[num1,num2,…、numX], where X is the list length, i.e. the number of number thresholds). Classify the channels according to the number thresholds. For example, sort the first 3 channels into one category and the rest into another category. Assuming that the channel number threshold can be set to 3, then num_list=[3].

[0101] Step S202: Construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the quantization step size.

[0102] It should be noted that after obtaining the quantization step size corresponding to each feature channel, the quantization step size and other parameters used when quantizing each quantization residual value can be obtained according to the feature channel corresponding to each quantization residual value, thereby calculating the inverse quantization precision parameters required to inverse quantize each quantization residual value.

[0103] For example, assuming the feature channels are divided into X classes, the quantization step size corresponding to each class of feature channels in the feature channel discrimination information can be represented by a list scale_list = [scale1, scale2, ..., scaleX], where X is the number of quantization steps. In this case, assume the quantization residual value corresponding to the j-th class of feature channels is r. qj The characteristic residual is r j The reconstructed residual value obtained by dequantization is r_hat j Then the quantization process r qj =round(r j scale j scale i During the dequantization process, r_hat j =r qj / (scale j scale i ), where scale j To eliminate the quantization step size used when quantizing the feature residuals corresponding to the j-th feature channel, scale i The quantization step size used in the original scheme when quantizing the feature residuals corresponding to the j-th feature channel is 'scale'. i It can be set to 1, meaning that the quantization step size of the original scheme can be retained or not. Similarly, in the following embodiments, you can choose whether to retain the quantization step size of the original scheme.

[0104] In the specific implementation process, for the same feature channel, different quantization step sizes can be further set, such as dividing multiple value intervals by setting a threshold, and further setting the value interval to which the value belongs by the probability distribution parameter or the value calculated based on the probability distribution parameter. Step S202 in this embodiment may include: Decode the image bitstream to obtain probability distribution parameters; Extract parameter interval thresholds from the feature channel differentiation information; Based on the probability distribution parameters, the quantization step size, and the parameter interval threshold, inverse quantization accuracy parameters corresponding to each quantization residual value are constructed.

[0105] It should be noted that when constructing the inverse quantization precision parameters corresponding to the quantization difference residual, one can first obtain the feature channel category to which the quantization residual value belongs, obtain the quantization step size and parameter interval threshold corresponding to that feature channel, divide the data into multiple value intervals based on the parameter interval threshold, and then calculate the value interval to which the probability distribution parameter corresponding to the quantization residual value belongs (or the value interval to which the mean or variance of the probability distribution parameter belongs). Based on the value interval, select the corresponding quantization step size from the multiple quantization step sizes corresponding to that feature channel, and then calculate the inverse quantization precision parameters corresponding to the inverse quantization of the quantization residual value based on the quantization step size. Here, the multiple quantization step sizes corresponding to a feature channel category each correspond to the divided value intervals.

[0106] For example: Suppose the feature channel corresponding to the quantization residual value is of class i, and the quantization step size corresponding to this feature channel includes scale1, scale2, and scaleN. The parameter interval thresholds include thr1, thr2, ..., thrN, which are different thresholds (thr1, thr2, ..., thrN in descending order). Then, the value interval can include: (thr1, ∞), (thr2, thr1], ..., (thrN, thrN-1]. Each value interval corresponds one-to-one with the quantization step size. If the probability distribution parameter σ corresponding to the quantization residual value is greater than thr1, then the corresponding quantization step size is scale1. If the probability distribution parameter thr1 ≥ σ > thr2, then the corresponding quantization step size is scale2, and so on.

[0107] In the specific implementation process, for the same type of feature channels, they can be further classified. For example, a type of feature channel can be divided into multiple segments, with each segment corresponding to a different quantization step size. Then, step S20 in this embodiment may include: The feature channel differentiation information is parsed to obtain the segmented step size set corresponding to each type of feature channel; Extract the quantization step size corresponding to each channel segment from the set of segmented step sizes; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0108] It should be noted that after further dividing a type of feature channel, each type of feature channel can correspond to multiple different quantization step sizes. These can be stored in the form of a set. At this point, the feature channel differentiation information can be parsed to obtain the set of segmented step sizes corresponding to each type of feature channel. Then, the quantization step size corresponding to each channel segment can be extracted from the set of segmented step sizes. The segmentation method for different types of feature channels can be different or the same; this embodiment does not impose any restrictions on this.

[0109] At this point, constructing the dequantization precision parameters corresponding to the quantization residual values ​​based on the quantization step size can involve obtaining the feature channel category corresponding to the quantization residual value, determining the channel segment to which the quantization residual value belongs in that feature channel category, and determining parameters such as the quantization step size used to calculate the quantization residual value based on the feature channel category and channel segment, thereby calculating the dequantization precision parameters required to dequantize the quantization residual value.

[0110] For example, taking the i-th type of feature channel as an example, the i-th type of feature channel can be divided into X segments. Then, the set of segmentation step sizes corresponding to the i-th type of feature channel can be represented as: scale_list i =[scale i1 ...scale ij ...sscle iX ], where scaleij is the quantization step size of the j-th segment in the i-th feature channel, and j takes the value [1, X].

[0111] In practical implementation, features within the same feature channel can be divided into multiple categories, and the classification rules can be any of the following: 1. A list of relevant thresholds calculated from the distribution parameter σ, such as the mean or variance of σ, or a list of relevant thresholds calculated from the absolute values ​​of the quantization residuals r_q, such as the sum of the absolute values ​​of the quantization residuals r_q, or the bit rate, and the list length X. The luminance and chrominance components thr can be combined into one list, thr_list = [thr1, thr2, ..., thrX], or they can be passed separately.

[0112] 2. Channel two-dimensional flag matrix, as shown in the diagram. Figure 7 As shown, the luminance and chrominance components can be combined into a single matrix or transmitted separately, similar to the channel flag list in Example 1. Different flags represent different categories.

[0113] 3. Row, column, and diagonal numbering, as well as list length, similar to channel numbering, as illustrated in the diagram. Figure 7 As shown.

[0114] 4. Pass row, column, and diagonal flags, similar to a channel flag list.

[0115] 5. The table of row, column, and diagonal number of columns, num_list=[num1,num2,…、numX], and the list length X, are similar to the list of channel numbers.

[0116] 6. Row, column, and diagonal markers, similar to channel markers.

[0117] This combination Figure 9 and 10 To explain, Figure 9 This is a schematic diagram of the identifier matrix in this embodiment, as shown below. Figure 9 As shown, the relevant data are divided into multiple categories, and each category corresponds to a different quantization step size. Then, the relevant feature data within the channel can be classified and labeled using a matrix approach, such as... Figure 9 As shown, 0 and 1 in the matrix represent the classification labels of the relevant feature data within the same channel.

[0118] Figure 10 This is a schematic diagram of the row and column division in this embodiment. The specific row, column, and diagonal column division can be as follows: Figure 8 As shown, from left to right, the relevant feature data in the channel can be divided into multiple rows; from top to bottom, the relevant feature data in the channel can be divided into multiple columns; and from bottom left to top right, the relevant feature data in the channel can be divided into multiple diagonal columns.

[0119] In one possible implementation of this example, the feature channels can be divided into blocks and different quantization step sizes can be set. In this case, step S20 of this embodiment may include: Extract the image block information and parameter setting rules corresponding to each feature channel from the feature channel differentiation information; Decode the image bitstream to obtain probability distribution parameters; Calculate the mean value of the distribution parameters corresponding to each image block information based on the probability distribution parameters; The mean of the distribution parameters is matched with the parameter setting rules to obtain the quantization step size corresponding to each image block information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0120] It should be noted that during encoding, each feature channel can be divided into blocks, and the block division rules are recorded in the feature channel differentiation information. Subsequently, different quantization step sizes are set according to the different mean values ​​of the probability distribution parameters corresponding to each block. The size of the block can be set freely.

[0121] Extracting image block information and parameter setting rules corresponding to each feature channel from the feature channel differentiation information can involve extracting parameter setting rules and block rules from the feature channel differentiation information, determining the blocks corresponding to each feature channel based on the block rules, and thus obtaining the image block information corresponding to each feature channel. The parameter setting rules include the quantization step size corresponding to the mean of different distribution parameters.

[0122] In practical applications, the mean of the distribution parameters corresponding to each image block information can be calculated by averaging the probability distribution parameters corresponding to each image block information, thereby obtaining the mean of the distribution parameters.

[0123] In the implementation process, the mean of the distribution parameter is matched with the parameter setting rules to obtain the quantization step size corresponding to each image block information. This can be achieved by matching the mean of the distribution parameter with multiple value ranges contained in the parameter setting rules, and using the quantization step size corresponding to the value range to which the mean of the distribution parameter belongs as the quantization step size corresponding to the image block information.

[0124] It is understandable that after obtaining the quantization step size corresponding to the image block information, the quantization step size and other parameters used to calculate the quantization residual value can be determined based on the image block information to which the quantization residual value belongs, thereby calculating the inverse quantization accuracy parameters required to inverse quantize the quantization residual value.

[0125] This embodiment constructs probabilistic quantization parameters based on feature channel differentiation information and / or spatial point differentiation information; the probability distribution parameters are then quantized based on these probabilistic quantization parameters to obtain distribution quantization parameters. Since the probabilistic quantization parameters are constructed based on feature channel differentiation information and / or spatial point differentiation information, and the probability distribution parameters are quantized based on these probabilistic quantization parameters to obtain distribution quantization parameters, it ensures that different quantization parameters can be set for the probability distribution parameters according to the differences in feature channel and spatial point image texture, further improving the performance of the encoding method in this embodiment.

[0126] refer to Figure 11 , Figure 11 This is a flowchart illustrating the fourth embodiment of an image decoding method according to the present invention.

[0127] Based on the first embodiment described above, step S20 of the image decoding method in this embodiment includes: Step S201': Extract the quantization step size corresponding to each type of spatial point from the spatial point differentiation information.

[0128] It should be noted that when setting the quantization step size, the spatial points in the image can be divided into multiple categories according to the image texture, and different quantization step sizes can be set for different categories of spatial points. For example, if the spatial points in the image are divided into X categories according to the image texture, then the quantization step size of the j-th category of spatial points is scale_j, and the value range of j is [1, X].

[0129] When classifying spatial points, they can be distinguished based on the image texture complexity corresponding to the spatial points. For example, three complexity intervals can be set: [0, F1], (F1, F2], (F2, ∞). Then, based on the complexity interval to which the image texture complexity of the spatial points belongs, the spatial points can be divided into three categories.

[0130] Step S202': Construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the quantization step size.

[0131] It is understandable that constructing the inverse quantization precision parameters corresponding to each quantization residual value based on the quantization step size corresponding to each type of spatial point can be done by obtaining parameters such as the quantization step size used when calculating the quantization residual value based on the spatial point category corresponding to the quantization residual value, thereby calculating the inverse quantization precision parameters required to inverse quantize each quantization residual value.

[0132] It should be noted that each spatial point actually contains C points (C being the number of feature channels). After classifying spatial points into different categories, the multiple points contained within a spatial point can be further divided, allowing different feature channels within a category of spatial points to correspond to different quantization step sizes. In this case, spatial point differentiation information and feature channel differentiation information can be used in conjunction. Therefore, step S20 in this embodiment may include: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in each type of spatial point is read from the feature channel differentiation information. Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

[0133] It should be noted that, based on the different feature channels, the multiple points contained in each spatial point of each type of spatial point are further divided. Then, the points of multiple different feature channels corresponding to a type of spatial point can correspond to different quantization step sizes. In this case, we can first read the multiple types of spatial points that can be divided from the spatial point differentiation information, and then read the quantization step size corresponding to each feature channel in each type of spatial point.

[0134] For example, if spatial points are divided into two classes, a and b, and the feature channels contained in a and b are all 1-X, then the quantization step size corresponding to each feature channel in class a spatial points can be represented as scale_lista = [scale1, scale2, ..., scaleX], and the quantization step size corresponding to each feature channel in class b spatial points can be represented as scale_listb = [scale1, scale2, ..., scaleX].

[0135] In practical implementation, in order to make it easier for the decoding device to clearly define the specific quantization step size setting method, a corresponding flag bit can be set in the image bit stream. The flag bit determines how to obtain the quantization step size. For example, if the flag bit is set to 0, the quantization step size is determined based on the feature channel differentiation information. If the flag bit is set to 1, the quantization step size is determined based on the spatial point differentiation information. If the flag bit is set to 2, the quantization step size is determined by combining the feature channel differentiation information and the spatial point differentiation information.

[0136] It should be noted that this embodiment only illustrates one way of using spatial point differentiation information and feature channel differentiation information together. Further division of multiple points contained in the spatial points can also be achieved by using the implementation method of further division of feature channels provided in any of the above embodiments. This embodiment does not limit this.

[0137] To facilitate understanding, we will now combine... Figure 12 This explanation is provided, but it does not limit the scope of this solution. Figure 12 This is a schematic diagram of spatial point classification in this embodiment. Based on the image texture complexity corresponding to the spatial points, the spatial points can be divided into three categories: the first category, the second category, and the third category (as shown by the arrows in the figure). The specific classification of spatial points is as follows: Figure 12 As shown ( Figure 12 The color intensity of each point in the image is used to represent the image texture complexity. Points with the same color intensity belong to the same category, and the darker the color, the higher the image texture complexity.

[0138] This embodiment extracts the quantization step size corresponding to each type of spatial point from the spatial point differentiation information; and constructs the inverse quantization precision parameters corresponding to each quantization residual value based on the quantization step size. Since the inverse quantization precision parameters corresponding to each quantization residual value are constructed based on the quantization step size corresponding to each type of spatial point, different quantization step sizes can be set for spatial points with different image textures, allowing for differentiated settings based on differences in image textures during quantization and inverse quantization.

[0139] This invention provides an image encoding method, referring to... Figure 13 , Figure 13 This is a flowchart illustrating a first embodiment of an image encoding method according to the present invention.

[0140] In this embodiment, the image encoding method includes the following steps: Step S100: Analyze and transform the image block to be processed to obtain image features.

[0141] It should be noted that the execution subject of this embodiment can be an encoding device for encoding image data. The encoding device can be a personal computer, server or other electronic device. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image encoding method of the present invention is described using an encoding device as an example.

[0142] It should be noted that analyzing and transforming the image block to be processed to obtain image features can be done by using an analysis transform network to analyze and transform the image block to be processed, extract feature information, and thus obtain image features.

[0143] Step S200: Perform residual calculation on the image features to obtain image residual values ​​and probability distribution parameters.

[0144] It should be noted that the residual calculation of image features to obtain image residual values ​​and probability distribution parameters can be performed by generating predicted values ​​through a context-based prediction module, then performing residual calculation based on image features and predicted values ​​to obtain image residual values, and finally analyzing the image residual values ​​through a super-coding network to obtain probability distribution parameters.

[0145] Step S300: Construct quantization accuracy parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information.

[0146] It should be noted that feature channel differentiation information can be used to divide image feature channels, while spatial point differentiation information can be used to classify points in the image. Both feature channel differentiation information and spatial point differentiation information can be preset by the administrator of the encoding device according to actual needs.

[0147] It should be noted that constructing the corresponding feature channel category or spatial point category for each quantized residual value based on feature channel differentiation information and / or spatial point differentiation information can involve determining the feature channel category or spatial point category corresponding to each image residual value based on the feature channel differentiation information and / or spatial point differentiation information. The quantization precision parameters required for quantizing each image residual value are then read from the feature channel differentiation information and / or spatial point differentiation information based on the feature channel category or spatial point category. These quantization precision parameters can be parameters that limit the quantization precision, such as the quantization step size.

[0148] Step S400: Quantize the image residual value based on the quantization accuracy parameter to obtain the quantization residual value.

[0149] It should be noted that the image residual value is quantized based on the quantization precision parameter. The quantization residual value can be obtained by quantizing the image residual value according to the quantization precision parameter and using the calculated value as the quantization residual value.

[0150] Step S500: Write the probability distribution parameters and the quantization residual values ​​into the image bitstream.

[0151] It should be noted that writing the probability distribution parameters and quantization residual values ​​into the image bitstream can be done by processing the probability distribution parameters through a super-coding network, then writing them into the image bitstream through entropy encoding, and then entropy encoding the quantization residual values ​​based on the quantized probability distribution parameters and writing them into the image bitstream.

[0152] In a specific implementation, the image bitstream may include a first image bitstream and a second image bitstream. The first image bitstream is used to transmit decoding auxiliary information, and the second image bitstream is used to transmit residual data. Therefore, step S500 in this embodiment may include: Write the probability distribution parameters into the first image bitstream; Construct distribution quantization parameters based on the probability distribution parameters; The quantization residual value is written into the second image bitstream based on the distributed quantization parameters.

[0153] It should be noted that writing the probability distribution parameters into the first image bitstream can be done by processing the probability distribution parameters through a super-coding network and then writing them into the first image bitstream through entropy encoding. Constructing distributed quantization parameters based on the probability distribution parameters can be done by quantizing the probability distribution parameters to obtain distributed quantization parameters. Writing the quantization residual values ​​into the second image bitstream based on the distributed quantization parameters can be done by entropy encoding the quantization residual values ​​based on the distributed quantization parameters and then writing them into the second image bitstream.

[0154] To ensure the performance of the quantizer, the quantization parameters used when quantizing the probability distribution parameters can be constructed based on feature channel differentiation information and / or spatial point differentiation information. Therefore, the step of constructing the distribution quantization parameters based on the probability distribution parameters in this embodiment may include: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

[0155] It should be noted that by constructing probabilistic quantization parameters based on feature channel differentiation information and / or spatial point differentiation information, different quantization parameters can be set according to the different feature channels or spatial points corresponding to the probability distribution parameters. This allows the quantization process to fully adapt to the differences in feature channels and the complexity of image textures, thereby ensuring the performance of the quantizer as much as possible.

[0156] In practical use, to ensure that the decoding device can decode the generated image bitstream normally, feature channel differentiation information and / or spatial point differentiation information can also be encoded into the image bitstream. Therefore, after the step of writing the quantization residual value into the second image bitstream based on the distributed quantization parameters described in this embodiment, the method further includes: Write the feature channel differentiation information and / or the spatial point differentiation information into the first image stream or the second image stream.

[0157] Understandably, since the parameters used for quantization are adjusted based on the feature channel differentiation information and / or spatial point differentiation information during encoding, the decoding device also needs to obtain the feature channel differentiation information and / or spatial point differentiation information in order to decode normally. In this case, writing the feature channel differentiation information and / or spatial point differentiation information into the first image bitstream or the second image bitstream can ensure that the decoding device can obtain the feature channel differentiation information and / or spatial point differentiation information, thereby ensuring that the decoding device can decode normally.

[0158] In practical use, to avoid data confusion or excessive complexity in the bitstream, a third image bitstream can be set up to transmit distinguishing information. In this case, after the step of writing the quantization residual value into the second image bitstream based on the distributed quantization parameters described in this embodiment, the method further includes: Write the feature channel differentiation information and / or the spatial point differentiation information into the third image bitstream.

[0159] Understandably, by setting up a dedicated third image bitstream, the feature channel differentiation information and / or spatial point differentiation information do not need to be written into the original bitstream, and will not be confused with the original data. This can avoid the phenomenon of bitstream data confusion or the bitstream data being too complex, leading to decoding difficulties.

[0160] To facilitate understanding, we will now combine... Figure 14 This explanation is provided, but it does not limit the scope of this solution. Figure 14 This is a schematic diagram of the image encoding process in this embodiment, as shown below. Figure 14 As shown, the encoding device can write the quantization residual value into the second image bitstream, write the encoding distribution parameter z_hat calculated from the probability distribution parameter σ into the first image bitstream, and write the relevant information for distinguishing feature channels or spatial points (feature channel distinction information and / or the spatial point distinction information) into the first or second image bitstream, or into the third image bitstream. If necessary, the relevant information for distinguishing feature channels or spatial points can also be directly written into the encoding device as local parameters in advance, without the need for bitstream transmission, as long as the encoding device and decoding device are consistent.

[0161] This embodiment obtains image features by analyzing and transforming the image blocks to be processed; it then calculates residuals from these image features to obtain image residual values ​​and probability distribution parameters; it constructs quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information; it performs quantization processing on the image residual values ​​based on the quantization precision parameters to obtain quantization residual values; and finally, it writes the probability distribution parameters and the quantization residual values ​​into the image bitstream. Since different quantization parameters can be set according to the differences in image texture between feature channels and spatial points during the encoding process, more important features can bear less quantization loss, thereby improving the performance of encoding and decoding.

[0162] refer to Figure 15 , Figure 15 This is a flowchart illustrating a second embodiment of an image encoding method according to the present invention.

[0163] Based on the first embodiment of the image encoding method described above, step S300 of the image encoding method in this embodiment includes: Step S3001: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information.

[0164] It should be noted that when distinguishing feature channels, feature channels can be divided into multiple categories, and different quantization steps can be assigned to different categories of feature channels. In this case, the feature channel distinction information will include the classification rules of feature channels and the quantization steps corresponding to each category of feature channels. At this time, the quantization steps corresponding to each category of feature channels can be directly extracted from the feature channel distinction information.

[0165] The classification rules for feature channels are set in a similar way to those described above, and will not be repeated here.

[0166] Step S3002: Construct quantization accuracy parameters corresponding to each image residual value based on the quantization step size.

[0167] It is understandable that after determining the quantization step size corresponding to each type of feature channel, the quantization step size and other parameters when quantizing the image residual value can be determined according to the feature channel category corresponding to each image residual value, thereby determining the quantization accuracy parameters corresponding to each image residual value.

[0168] In practical implementation, for the same feature channel, different quantization step sizes can be further set, such as dividing multiple value intervals by setting a threshold, and further setting the value interval to which the value belongs by the probability distribution parameter or the value calculated based on the probability distribution parameter. Step S3002 in this embodiment may include: Extract parameter interval thresholds from the feature channel differentiation information; The quantization accuracy parameters corresponding to each image residual value are constructed based on the probability distribution parameters, the parameter interval threshold, and the quantization step size.

[0169] It should be noted that when constructing the quantization precision parameters corresponding to each feature channel based on the probability distribution parameters, parameter interval thresholds, and quantization step size, one can first obtain the feature channel category to which the quantization residual value belongs, obtain the quantization step size and parameter interval threshold corresponding to that feature channel, divide multiple value intervals according to the parameter interval threshold, and then calculate the value interval to which the probability distribution parameter corresponding to the quantization residual value belongs (or the value interval to which the mean or variance of the probability distribution parameter belongs). Based on the value interval, select the corresponding quantization step size from the multiple quantization step sizes corresponding to that feature channel, thereby obtaining the quantization precision parameters corresponding to each image residual value.

[0170] In the specific implementation process, for the same type of feature channels, they can be further classified. For example, a type of feature channel can be divided into multiple segments, with each segment corresponding to a different quantization step size. Then, step S3002 in this embodiment can include: The feature channel differentiation information is parsed to obtain the segmented step size set corresponding to each type of feature channel; Extract the quantization step size corresponding to each channel segment from the set of segmented step sizes; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0171] It should be noted that after further dividing a type of feature channel, each type of feature channel can correspond to multiple different quantization step sizes. These can be stored in the form of a set. At this point, the feature channel differentiation information can be parsed to obtain the set of segmented step sizes corresponding to each type of feature channel. Then, the quantization step size corresponding to each channel segment can be extracted from the set of segmented step sizes. The segmentation method for different types of feature channels can be different or the same; this embodiment does not impose any restrictions on this.

[0172] At this point, constructing the quantization precision parameters corresponding to each image residual value based on the quantization step size can involve obtaining the feature channel category corresponding to the image residual value, determining the channel segment to which the image residual value belongs in that feature channel category, and determining parameters such as the quantization step size used to quantize the image residual value based on the feature channel category and channel segment, thereby obtaining the quantization precision parameters corresponding to each image residual value.

[0173] In a specific implementation, the quantization step size can be set according to the image texture. In this case, step S3002 in this embodiment may include: Extract the quantization step size corresponding to various types of spatial points from spatial point differentiation information; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0174] It should be noted that when setting the quantization step size, the spatial points in the image can be divided into multiple categories according to the image texture, and different quantization step sizes can be set for different categories of spatial points. For example, if the spatial points in the image are divided into X categories according to the image texture, then the quantization step size of the j-th category of spatial points is scale_j, and the value range of j is [1, X].

[0175] When classifying spatial points, they can be distinguished based on the image texture complexity corresponding to the spatial points. For example, three complexity intervals can be set: [0, F1], (F1, F2], (F2, ∞). Then, based on the complexity interval to which the image texture complexity of the spatial points belongs, the spatial points can be divided into three categories.

[0176] It is understandable that constructing the quantization precision parameters corresponding to each image residual value based on the quantization step size corresponding to each type of spatial point can be done by obtaining parameters such as the quantization step size used when quantizing the image residual value based on the spatial point category corresponding to the image residual value, thereby obtaining the quantization precision parameters corresponding to each image residual value.

[0177] It should be noted that each spatial point actually contains C points (C being the number of feature channels). After classifying spatial points into different categories, the multiple points contained within a spatial point can be further divided, allowing different feature channels within a category of spatial points to correspond to different quantization step sizes. In this case, spatial point differentiation information and feature channel differentiation information can be used in conjunction. Therefore, step S3002 in this embodiment may include: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in various spatial points is read from the feature channel differentiation information. Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

[0178] It should be noted that, based on the different feature channels, the multiple points contained in each spatial point of each type of spatial point are further divided. Then, the points of multiple different feature channels corresponding to a type of spatial point can correspond to different quantization step sizes. In this case, we can first read the multiple types of spatial points that can be divided from the spatial point differentiation information, and then read the quantization step size corresponding to each feature channel in each type of spatial point.

[0179] For example, if spatial points are divided into two classes, a and b, and the feature channels contained in a and b are all 1-X, then the quantization step size corresponding to each feature channel in class a spatial points can be represented as scale_lista = [scale1, scale2, ..., scaleX], and the quantization step size corresponding to each feature channel in class b spatial points can be represented as scale_listb = [scale1, scale2, ..., scaleX].

[0180] In practical implementation, in order to make it easier for the encoding device to clearly define the specific quantization step size setting method, a corresponding flag bit can be pre-set in the encoding device. The flag bit determines how to obtain the quantization step size. For example, if the flag bit is set to 0, the quantization step size is determined based on the feature channel differentiation information. If the flag bit is set to 1, the quantization step size is determined based on the spatial point differentiation information. If the flag bit is set to 2, the quantization step size is determined by combining the feature channel differentiation information and the spatial point differentiation information.

[0181] It should be noted that this embodiment only illustrates one way of using spatial point differentiation information and feature channel differentiation information together. Further division of multiple points contained in the spatial points can also be achieved by using the implementation method of further division of feature channels provided in any of the above embodiments. This embodiment does not limit this.

[0182] refer to Figure 16 , Figure 16 This is a flowchart illustrating a third embodiment of an image encoding method according to the present invention.

[0183] Based on the first embodiment of the image encoding method described above, step S500 of the image encoding method in this embodiment includes: Step S5001: Decode the image based on the probability distribution parameters, the quantization residual value, and the feature channel differentiation information to obtain the reconstructed image block and image bitrate.

[0184] It should be noted that the image decoding based on probability distribution parameters, quantization residual values, and feature channel differentiation information to obtain reconstructed image blocks and image bitrate can be performed according to the image decoding method provided in any of the above embodiments. The image decoding is performed based on probability distribution parameters, quantization residual values, and feature channel differentiation information to obtain reconstructed image blocks, and the image bitrate is calculated based on probability distribution parameters.

[0185] Step S5002: Adjust the feature channel differentiation information and / or spatial point differentiation information according to the reconstructed image block and the image bitrate.

[0186] It should be noted that the encoding device can also optimize and adjust the parameters used in the encoding process. In this case, adjusting the feature channel differentiation information based on the reconstructed image block and image bitrate can be an optimization and adjustment of the quantization-related parameters in the feature channel differentiation information and / or spatial point differentiation information based on the reconstructed image block and image bitrate.

[0187] In practice, administrators of the encoding device can determine whether parameter optimization adjustments are needed by modifying the optimal threshold mode flag bit in the encoding device. Specifically, a value of 0 for the optimal threshold mode flag bit indicates that no parameter optimization adjustments will be made; a value of 1 indicates that parameter optimization adjustments will be made.

[0188] To facilitate understanding, we will now combine... Figure 17 This explanation is provided, but it does not limit the scope of this solution. Figure 17 This is a schematic diagram illustrating the function of the optimal threshold mode flag, as follows: Figure 17 As shown, the encoding device user can set the optimal threshold mode flag (Flag_useRDAQ) and pass the set flag and the relevant thresholds or parameters for distinguishing feature channels to the encoding device. The encoding device will then check if the optimal threshold mode flag value is 1. If it is 1, parameter optimization adjustment will be enabled, using the input relevant thresholds or parameters as initial values, and then multiple iterations will be performed. The encoding will be performed based on the optimal thresholds or parameters for distinguishing feature channels obtained after iteration, and the generated image code stream will be sent to the decoding device. If the value is not 1, the encoding will be performed directly based on the input relevant thresholds or parameters for distinguishing feature channels, and the encoded image code stream will be sent to the decoding device.

[0189] Step S5003: If the current adjustment round is greater than or equal to the preset adjustment round, then write the probability distribution parameter and the quantization residual value into the image bitstream.

[0190] It should be noted that the preset adjustment rounds can be a pre-set number of times to adjust the feature channel differentiation information. The current adjustment round can be the number of times the feature channel differentiation information has been adjusted.

[0191] It is understood that if the current adjustment round is greater than or equal to the preset adjustment round, it indicates that the parameter optimization adjustment has been completed. At this time, the probability distribution parameters and the quantization residual values ​​can be written into the image bitstream, and the image bitstream can be transmitted to the decoding device. Furthermore, the adjusted feature channel differentiation information and / or spatial point differentiation information can also be written into the image bitstream.

[0192] If the current adjustment round is less than the preset adjustment round, it means that the parameter optimization adjustment has not yet ended. Therefore, we can return to the step of constructing the quantization accuracy parameters corresponding to each feature channel based on the feature channel differentiation information.

[0193] To facilitate understanding, we will now combine... Figure 18 This explanation is provided, but it does not limit the scope of this solution. Figure 18 This is a flowchart of the parameter optimization process in this embodiment. Figure 18 In this context, the distortion D can be represented by one or more of the following metrics: msssim, vif, fsim, nlpd, iw-ssim, vmaf, psnr_HVS, etc. X is the relevant threshold or parameter of the input distinguishing feature channel, res_q is the quantization residual value, sigmma is the probability distribution parameter σ, and step_X can be preset by the administrator of the encoding device according to actual needs. Meeting the iteration limit can be achieved by comparing the current adjustment round with the preset adjustment round. If the current adjustment round is greater than or equal to the preset adjustment round, the iteration limit is met; if the current adjustment round is less than the preset adjustment round, the iteration limit is not met.

[0194] This embodiment performs image decoding based on the probability distribution parameters, the quantization residual value, and the feature channel differentiation information to obtain reconstructed image blocks and image bitrate. The feature channel differentiation information and / or spatial point differentiation information are adjusted based on the reconstructed image blocks and the image bitrate. If the current adjustment round is greater than or equal to a preset adjustment round, the probability distribution parameters and the quantization residual value are written into the image bitstream. Because image decoding is performed, and the feature channel differentiation information and / or spatial point differentiation information are adjusted based on the reconstructed image blocks and image bitrate, the parameters used in the final encoding are guaranteed to be of high quality, thereby further improving image encoding performance.

[0195] Furthermore, embodiments of the present invention also propose a storage medium storing an image encoding or image decoding program. When the image encoding program is executed by a processor, it implements the steps of the image encoding method described above, and when the image decoding program is executed by a processor, it implements the steps of the image decoding method described above.

[0196] Reference Figure 19 , Figure 19 This is a structural block diagram of the first embodiment of the image decoding device of the present invention.

[0197] like Figure 19 As shown, the image decoding device proposed in this embodiment of the invention includes: The entropy decoding module 10 is used to acquire feature channel differentiation information and / or spatial point differentiation information, and to decode the image bitstream to obtain quantization residual values. Parameter construction module 20 is used to construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information; The dequantization module 30 is used to dequantize the quantization residual value based on the dequantization accuracy parameter to obtain the reconstructed residual value; The synthesis transformation module 40 is used to perform a synthesis transformation on the reconstructed residual values ​​to obtain reconstructed image blocks.

[0198] For ease of understanding, this section combines the above... Figure 3 This explanation does not limit the scope of the solution. In this embodiment, the entropy decoding module can perform the following: Figure 3 The entropy decoding process shown in the figure involves the parameter construction module and the dequantization module executing as follows: Figure 3 The inverse quantization process shown is executed by the synthesis transform module through a synthesis transform network. Figure 3 The synthesis and transformation process in the process.

[0199] This embodiment obtains quantization residual values ​​by acquiring feature channel differentiation information and / or spatial point differentiation information and decoding the image bitstream. Based on the feature channel differentiation information and / or the spatial point differentiation information, it constructs inverse quantization precision parameters corresponding to each quantization residual value. Based on the inverse quantization precision parameters, it performs inverse quantization on the quantization residual values ​​to obtain reconstructed residual values. Finally, it performs a synthetic transformation on the reconstructed residual values ​​to obtain reconstructed image patches. Because inverse quantization is performed by constructing corresponding inverse quantization parameters based on feature channel differentiation information and / or spatial point differentiation information, different quantization parameters can be set according to the differences in image texture between feature channels and spatial points during the encoding process. This allows more important features to bear less quantization loss, thereby improving the performance of encoding and decoding.

[0200] In one possible implementation of this embodiment, the image stream includes a first image stream and a second image stream, wherein the first image stream is used to transmit decoding auxiliary information and the second image stream is used to transmit residual data; The entropy decoding module 10 is further configured to extract coding distribution parameters from the first image bitstream; decode the coding distribution parameters to obtain probability distribution parameters; quantize the probability distribution parameters to obtain distribution quantization parameters; and decode the second image bitstream based on the distribution quantization parameters to obtain quantization residual values.

[0201] In one possible implementation of this embodiment, the entropy decoding module 10 is further configured to construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; and to quantize the probability distribution parameters based on the probability quantization parameters to obtain distribution quantization parameters.

[0202] In one possible implementation of this embodiment, the entropy decoding module 10 is further configured to decode the first image stream or the second image stream to obtain feature channel differentiation information and / or spatial point differentiation information.

[0203] In one possible implementation of this embodiment, the image stream further includes a third image stream, which is used to transmit distinguishing information; The entropy decoding module 10 is also used to decode the third image stream to obtain feature channel differentiation information and / or spatial point differentiation information.

[0204] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; and construct the inverse quantization accuracy parameter corresponding to each quantization residual value based on the quantization step size.

[0205] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to extract the quantization step size corresponding to each type of spatial point from the spatial point differentiation information; and construct the inverse quantization accuracy parameter corresponding to each quantization residual value based on the quantization step size.

[0206] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to read the various types of spatial points from the spatial point differentiation information; read the quantization step size corresponding to each type of feature channel in the various types of spatial points from the feature channel differentiation information; and construct the inverse quantization accuracy parameter corresponding to each quantization residual value based on the quantization step size.

[0207] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to decode the image bitstream to obtain probability distribution parameters; extract parameter interval thresholds from the feature channel differentiation information; and construct inverse quantization precision parameters corresponding to each quantization residual value based on the probability distribution parameters, the quantization step size, and the parameter interval thresholds.

[0208] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to extract image block information and parameter setting rules corresponding to each feature channel from the feature channel differentiation information; decode the image bitstream to obtain probability distribution parameters; calculate the mean of distribution parameters corresponding to each image block information based on the probability distribution parameters; match the mean of distribution parameters with the parameter setting rules to obtain the quantization step size corresponding to each image block information; and construct the inverse quantization precision parameters corresponding to each quantization residual value based on the quantization step size.

[0209] In one possible implementation of this embodiment, the parameter construction module 20 is further configured to parse the feature channel differentiation information to obtain a set of segmented step lengths corresponding to various feature channels; extract the quantization step length corresponding to each channel segment from the set of segmented step lengths; and construct the inverse quantization precision parameters corresponding to each quantization residual value based on the quantization step length.

[0210] Reference Figure 20 , Figure 20 This is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0211] like Figure 20 As shown, the image encoding device proposed in this embodiment of the invention includes: The analysis and transformation module 100 is used to perform analysis and transformation processing on the image block to be processed to obtain image features; The residual calculation module 200 is used to perform residual calculation on the image features to obtain image residual values ​​and probability distribution parameters. The parameter construction module 300 is used to construct the quantization accuracy parameters corresponding to each feature channel based on feature channel differentiation information and / or spatial point differentiation information. The quantization module 400 is used to perform quantization processing on the image residual value based on the quantization accuracy parameter to obtain the quantization residual value. The entropy coding module 500 is used to write the probability distribution parameters and the quantization residual values ​​into the image bitstream.

[0212] For ease of understanding, this section combines the above... Figure 3 This explanation does not limit the scope of the solution. In this embodiment, the analysis transformation module can perform operations such as analyzing the transformation network. Figure 3 The analysis and transformation process shown in the figure involves the residual calculation module, parameter construction module, and quantization module executing as follows: Figure 3 The quantization process shown in the diagram, the entropy encoding module executes as follows: Figure 3 The entropy coding process in [the context of the text].

[0213] This embodiment obtains image features by analyzing and transforming the image blocks to be processed; it then calculates residuals from these image features to obtain image residual values ​​and probability distribution parameters; it constructs quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information; it performs quantization processing on the image residual values ​​based on the quantization precision parameters to obtain quantization residual values; and finally, it writes the probability distribution parameters and the quantization residual values ​​into the image bitstream. Since different quantization parameters can be set according to the differences in image texture between feature channels and spatial points during the encoding process, more important features can bear less quantization loss, thereby improving the performance of encoding and decoding.

[0214] In one possible implementation of this embodiment, the image stream includes a first image stream and a second image stream, wherein the first image stream is used to transmit decoding auxiliary information and the second image stream is used to transmit residual data; The entropy coding module 500 is further configured to write the probability distribution parameters into the first image bitstream; construct distribution quantization parameters based on the probability distribution parameters; and write the quantization residual value into the second image bitstream based on the distribution quantization parameters.

[0215] In one possible implementation of this embodiment, the entropy coding module 500 is further configured to construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; and to quantize the probability distribution parameters based on the probability quantization parameters to obtain distribution quantization parameters.

[0216] In one possible implementation of this embodiment, the entropy coding module 500 is further configured to write the feature channel differentiation information and / or the spatial point differentiation information into the first image bitstream or the second image bitstream.

[0217] In one possible implementation of this embodiment, the image stream further includes a third image stream, which is used to transmit distinguishing information; The entropy coding module 500 is further configured to write the feature channel differentiation information and / or the spatial point differentiation information into the third image bitstream.

[0218] In one possible implementation of this embodiment, the parameter construction module 300 is further configured to extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; and construct the quantization accuracy parameters corresponding to each image residual value based on the quantization step size.

[0219] In one possible implementation of this embodiment, the parameter construction module 300 is further configured to extract parameter interval thresholds from the feature channel differentiation information; and construct quantization accuracy parameters corresponding to each image residual value based on the probability distribution parameters, the parameter interval thresholds, and the quantization step size.

[0220] In one possible implementation of this embodiment, the parameter construction module 300 is further configured to parse the feature channel differentiation information to obtain a set of segmented step lengths corresponding to various feature channels; extract the quantization step length corresponding to each channel segment from the set of segmented step lengths; and construct quantization accuracy parameters corresponding to each image residual value based on the quantization step length.

[0221] In one possible implementation of this embodiment, the parameter construction module 300 is further used to extract the quantization step size corresponding to various spatial points from the spatial point differentiation information; and to construct the quantization accuracy parameters corresponding to each image residual value according to the quantization step size.

[0222] In one possible implementation of this embodiment, the parameter construction module 300 is further configured to read various types of spatial points from the spatial point differentiation information; read the quantization step size corresponding to each type of feature channel in the various types of spatial points from the feature channel differentiation information; and construct quantization accuracy parameters corresponding to each image residual value based on the quantization step size.

[0223] In one possible implementation of this embodiment, the entropy coding module 500 is further configured to perform image decoding based on the probability distribution parameters, the quantization residual value, and the feature channel differentiation information to obtain reconstructed image blocks and image bitrate; adjust the feature channel differentiation information and / or spatial point differentiation information based on the reconstructed image blocks and the image bitrate; if the current adjustment round is greater than or equal to the preset adjustment round, then write the probability distribution parameters and the quantization residual value into the image bitstream.

[0224] In one possible implementation of this embodiment, the entropy coding module 500 is further configured to, if the current adjustment round is less than the preset adjustment round, return to the step of constructing the quantization precision parameters corresponding to each feature channel based on the feature channel differentiation information.

[0225] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0226] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0227] In addition, for technical details not described in detail in this embodiment, please refer to the image encoding or image decoding methods provided in any embodiment of the present invention, which will not be repeated here.

[0228] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0229] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0230] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0231] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image decoding method, characterized in that, The image decoding method includes the following steps: Acquire feature channel differentiation information and / or spatial point differentiation information, and decode the image bitstream to obtain quantization residual values; Construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information; For any quantization residual value, the quantization residual value is dequantized based on the dequantization precision parameter corresponding to the quantization residual value to obtain the reconstructed residual value; The mean value used when calculating the feature residual value is obtained through context prediction. Based on the mean value, the reconstructed residual value is reconstructed into image features, and the image features are synthesized and transformed to obtain reconstructed image patches.

2. The image decoding method as described in claim 1, characterized in that, The step of decoding the image bitstream to obtain the quantization residual value includes: Extract coding distribution parameters from the image bitstream; The encoded distribution parameters are decoded to obtain the probability distribution parameters; The probability distribution parameters are quantized to obtain the distribution quantization parameters; The image bitstream is decoded based on the distributed quantization parameters to obtain the quantization residual value.

3. The image decoding method as described in claim 2, characterized in that, The step of quantizing the probability distribution parameters to obtain distribution quantization parameters includes: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

4. The image decoding method as described in claim 3, characterized in that, The step of constructing probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information includes: The probability quantization parameters are constructed based on the feature channels or spatial points corresponding to the probability distribution parameters. Different feature channels or spatial points correspond to different probability quantization parameters.

5. The image decoding method as described in claim 1, characterized in that, The steps of obtaining feature channel differentiation information and / or spatial point differentiation information include: The image stream is decoded to obtain feature channel differentiation information and / or spatial point differentiation information.

6. The image decoding method as described in claim 1, characterized in that, The step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

7. The image decoding method as described in claim 6, characterized in that, Different types of feature channels correspond to different quantization step sizes.

8. The image decoding method as described in claim 1, characterized in that, The step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Extract the quantization step size corresponding to each type of spatial point from the spatial point differentiation information; Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

9. The image decoding method as described in claim 8, characterized in that, Different types of spatial points correspond to different quantization step sizes.

10. The image decoding method as described in claim 1, characterized in that, The step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in each type of spatial point is read from the feature channel differentiation information. Based on the quantization step size, construct the inverse quantization accuracy parameters corresponding to each quantization residual value.

11. The image decoding method as described in claim 6, characterized in that, The step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the quantization step size includes: Decode the image bitstream to obtain probability distribution parameters; Extract parameter interval thresholds from the feature channel differentiation information; Based on the probability distribution parameters, the quantization step size, and the parameter interval threshold, inverse quantization accuracy parameters corresponding to each quantization residual value are constructed.

12. The image decoding method as described in claim 1, characterized in that, The step of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information includes: The feature channel differentiation information is parsed. For any quantization residual value, obtain the set of segmented step sizes corresponding to each type of feature channel. The set of segmented step sizes corresponding to each type of feature channel includes multiple different quantization step sizes. Extract the quantization step size corresponding to the channel segment to which the quantization residual value belongs from the set of segmented step sizes corresponding to one of the feature channels; Construct the inverse quantization accuracy parameters corresponding to the quantization residual values ​​based on the quantization step size.

13. The image decoding method as described in claim 1, characterized in that, The method further includes: For any quantization residual value, the feature channel category corresponding to the quantization residual value is obtained, and the channel segment to which the quantization residual value belongs in the feature channel category is determined. The quantization step size corresponding to the quantization residual value is determined according to the feature channel category and the channel segment. The inverse quantization precision parameter corresponding to the quantization residual value is determined according to the quantization step size. Herein, the set of segment step sizes corresponding to a feature channel category includes multiple different quantization step sizes, a feature channel category is divided into multiple channel segments, and one channel segment corresponds to one quantization step size.

14. The image decoding method as described in claim 1, characterized in that, The steps of constructing the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information include: The feature channel category and / or spatial point category corresponding to each quantized residual value are determined based on the feature channel differentiation information and / or the spatial point information. Based on the feature channel category and / or the spatial point category, determine the inverse quantization accuracy parameter corresponding to each quantization residual value; Different categories of feature channels and / or spatial point categories employ quantization with different precision.

15. An image encoding method, characterized in that, The image encoding method includes the following steps: The image blocks to be processed are analyzed and transformed to obtain image features; The image features are subjected to residual calculation to obtain image residual values ​​and probability distribution parameters; Construct quantization precision parameters corresponding to the residual values ​​of each image based on feature channel differentiation information and / or spatial point differentiation information; The image residual value is quantized based on the quantization accuracy parameter to obtain the quantization residual value; The probability distribution parameters and the quantization residual values ​​are written into the image bitstream.

16. The image encoding method as described in claim 15, characterized in that, The step of writing the probability distribution parameters and the quantization residual values ​​into the image bitstream includes: Write the probability distribution parameters into the image bitstream; Construct distribution quantization parameters based on the probability distribution parameters; The quantization residual value is written into the image bitstream based on the distributed quantization parameters.

17. The image encoding method as described in claim 15, characterized in that, The step of constructing distribution quantization parameters based on the probability distribution parameters includes: Construct probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information; The probability distribution parameters are quantized based on the probability quantization parameters to obtain the distribution quantization parameters.

18. The image encoding method as described in claim 17, characterized in that, The step of constructing probability quantization parameters based on the feature channel differentiation information and / or spatial point differentiation information includes: The probability quantization parameters are constructed based on the feature channels or spatial points corresponding to the probability distribution parameters. Different feature channels or spatial points correspond to different probability quantization parameters.

19. The image encoding method as described in claim 15, characterized in that, The image encoding method further includes: Write the feature channel differentiation information and / or the spatial point differentiation information into the image bitstream.

20. The image encoding method as described in claim 15, characterized in that, The step of constructing the quantization accuracy parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Extract the quantization step size corresponding to each type of feature channel from the feature channel differentiation information; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

21. The image encoding method as described in claim 20, characterized in that, The image encoding method further includes: The feature channels are divided into multiple categories; For each type of feature channel, determine the corresponding quantization step size; Different types of feature channels correspond to different quantization step sizes.

22. The image encoding method as described in claim 20, characterized in that, The step of constructing the quantization accuracy parameters corresponding to each image residual value based on the quantization step size includes: Extract parameter interval thresholds from the feature channel differentiation information; The quantization accuracy parameters corresponding to each image residual value are constructed based on the probability distribution parameters, the parameter interval threshold, and the quantization step size.

23. The image encoding method as described in claim 15, characterized in that, The step of constructing the quantization accuracy parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: The feature channel differentiation information is parsed. For any image residual value, Obtain the set of segmented step sizes corresponding to various feature channels, wherein the set of segmented step sizes corresponding to one type of feature channel includes multiple different quantization step sizes; Extract the quantization step size corresponding to the channel segment to which the image residual value belongs from the set of segmented step sizes corresponding to one of the feature channels; The quantization accuracy parameters corresponding to the image residual values ​​are constructed based on the quantization step size.

24. The image encoding method as described in claim 15, characterized in that, The step of constructing the quantization accuracy parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Extract the quantization step size corresponding to various types of spatial points from spatial point differentiation information; Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

25. The image encoding method as described in claim 24, characterized in that, The image encoding method further includes: Based on image texture, spatial points in an image are classified into multiple categories; For each type of spatial point, determine the quantization step size for that type of spatial point; Different categories of spatial points correspond to different quantization step sizes.

26. The image encoding method as described in claim 15, characterized in that, The step of constructing the quantization accuracy parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information includes: Read the various types of spatial points from the spatial point differentiation information; The quantization step size corresponding to each feature channel in various spatial points is read from the feature channel differentiation information. Based on the quantization step size, construct the quantization accuracy parameters corresponding to the residual values ​​of each image.

27. The image encoding method as described in claim 15, characterized in that, The method further includes: For any image residual value, the feature channel category corresponding to the image residual value is obtained, and the channel segment to which the image residual value belongs in the feature channel category is determined. The quantization step size used for quantization when quantizing the image residual value is determined according to the feature channel category and the channel segment. The quantization precision parameter corresponding to the image residual value is determined according to the quantization step size. Herein, the segment step size set corresponding to a feature channel category includes multiple different quantization step sizes, a feature channel category is divided into multiple channel segments, and one channel segment corresponds to one quantization step size.

28. The image encoding method as described in claim 15, characterized in that, The steps of constructing quantization precision parameters corresponding to each image residual value based on the feature channel differentiation information and / or the spatial point differentiation information include: The feature channel category and / or spatial point category corresponding to each image residual value are determined based on the feature channel differentiation information and / or the spatial point information. Based on the feature channel category and / or the spatial point category, determine the quantization accuracy parameter corresponding to each image residual value; Different types of feature channels and / or different precision quantization are used.

29. An image decoding device, characterized in that, The image decoding device includes: The entropy decoding module is used to obtain feature channel differentiation information and / or spatial point differentiation information, and to decode the image bitstream to obtain quantization residual values; The parameter construction module is used to construct the inverse quantization accuracy parameters corresponding to each quantization residual value based on the feature channel differentiation information and / or the spatial point differentiation information. The dequantization module is used to dequantize any given quantization residual value based on the dequantization precision parameter corresponding to that quantization residual value, and obtain a reconstructed residual value. The synthesis transformation module is used to obtain the mean value used when calculating the feature residual value through context prediction, reconstruct the image feature based on the mean value, and perform a synthesis transformation on the image feature to obtain the reconstructed image patch.

30. An image encoding device, characterized in that, The image encoding device includes: The analysis and transformation module is used to perform analysis and transformation processing on the image blocks to be processed to obtain image features; The residual calculation module is used to perform residual calculation on the image features to obtain image residual values ​​and probability distribution parameters; The parameter construction module is used to construct the quantization accuracy parameters corresponding to each feature channel based on feature channel differentiation information and / or spatial point differentiation information. The quantization module is used to perform quantization processing on the image residual value based on the quantization accuracy parameter to obtain the quantization residual value; The entropy coding module is used to write the probability distribution parameters and the quantization residual values ​​into the image bitstream.

31. A decoding device, characterized in that, The decoding device includes: a processor, a memory, and a decoding program stored in the memory and executable on the processor. When the decoding program is executed by the processor, it implements the image decoding method as described in any one of claims 1-14.

32. An encoding device, characterized in that, The encoding device includes: a processor, a memory, and an encoding program stored in the memory and executable on the processor, wherein the encoding program, when executed by the processor, implements the image encoding method as described in any one of claims 15-28.

33. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image decoding program and / or an image encoding program; When the image decoding program is executed, it implements the image decoding method as described in any one of claims 1-14; or when the image encoding program is executed, it implements the image encoding method as described in any one of claims 15-28.

34. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image decoding method as described in any one of claims 1-14, or when the image encoding program is executed by the processor, it implements the image encoding method as described in any one of claims 15-28.

Images