Image decoding and encoding method, apparatus, device, and storage medium

Abstract

The present invention relates to the field of image processing and discloses an image decoding and encoding method, apparatus, device, and storage medium. The present invention includes: obtaining feature channel distinction information and / or spatial point distinction information; decoding an image bitstream; obtaining quantized residual values; constructing inverse quantization precision parameters corresponding to each quantized residual value based on the feature channel distinction information and / or spatial point distinction information; inverse quantizing the quantized residual values ​​based on the inverse quantization precision parameters; obtaining reconstructed residual values; and performing synthesis transformation on the reconstructed residual values ​​to obtain a reconstructed image block. Since the corresponding inverse quantization precision parameters are constructed based on the feature channel distinction information and / or spatial point distinction information and then performing inverse quantization, different quantization parameters can be set during the encoding process based on differences in image texture between feature channels and spatial points, allowing more important features to bear smaller quantization losses and improving encoding and decoding performance.

Description

[Technical Field]

[0001] The present invention relates to the field of image processing technology, and in particular to an image decoding and encoding method, apparatus, device, and storage medium. [Background technology]

[0002] End-to-end image encoding and decoding technologies generally include modules such as analysis transform networks, synthesis transform networks, context-based prediction, quantization, entropy coding, super-coding networks, and super-scale decoding networks, where quantization is a "many-to-one" mapping process that results in signal loss. Quantization operates on the residual and changes the value range of the signal, allowing the encoder to provide a good approximation of the original signal with a small number of codes and improving the compression rate. However, the quantization module generally does not take into account the differences in feature channels and image textures when performing quantization, which limits the quantization performance of the quantizer.

[0003] The above content is only an aid for understanding the technical solution of the present invention, and is not intended to be an admission that the above content is prior art. Summary of the Invention [Problem to be solved by the invention]

[0004] The main objective of the present invention is to provide an image decoding and encoding method, apparatus, device and storage medium to solve the technical problem of poor performance of quantizer in image encoding and decoding process in the prior art. [Means for solving the problem]

[0005] In order to achieve the above object, the present invention provides: obtaining feature channel distinction information and / or spatial point distinction information, decoding the image bitstream, and obtaining quantized residual values; constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information; for any quantized residual value, dequantizing the quantized residual value based on the dequantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; performing a synthesis transform on the reconstructed residual values ​​to obtain a reconstructed image block.

[0006] In one possible embodiment of the present invention, the step of decoding the image bitstream to obtain quantized residual values ​​comprises: extracting coding distribution parameters from the first image bitstream; decoding the encoded distribution parameters to obtain probability distribution parameters; quantizing the probability distribution parameters to obtain a distribution quantization parameter; and decoding the second image bitstream based on the distribution quantization parameter to obtain the quantized residual value.

[0007] In one possible embodiment of the present invention, the step of quantizing the probability distribution parameters to obtain a distribution quantization parameter comprises: constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0008] In one possible embodiment of the present invention, the step of obtaining the feature channel distinction information and / or the spatial point distinction information comprises: The method includes decoding the image bitstream to obtain feature channel distinguishing information and / or spatial point distinguishing information.

[0009] In one possible embodiment of the present invention, the image bitstream further comprises a third image bitstream, the third image bitstream being used to transmit differentiation information; The step of obtaining the feature channel distinguishing information and / or the spatial point distinguishing information includes: The method includes decoding the third image bitstream to obtain feature channel distinguishing information and / or spatial point distinguishing information.

[0010] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: extracting quantization steps corresponding to each type of feature channel from the feature channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0011] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: extracting quantization steps corresponding to each type of spatial point from the spatial point distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0012] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0013] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step comprises: decoding the image bitstream to obtain probability distribution parameters; extracting a parameter section threshold from the characteristic channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the probability distribution parameter, the quantization step, and the parameter interval threshold.

[0014] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: extracting image block division information and parameter setting rules corresponding to each feature channel from the feature channel distinction information; decoding the image bitstream to obtain probability distribution parameters; calculating a distribution parameter average value corresponding to each image block division information based on the probability distribution parameters; a step of matching the distribution parameter mean value with the parameter setting rule to obtain a quantization step corresponding to each image block division information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0015] In one possible embodiment of the present invention, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: analyzing the characteristic channel distinction information; For any quantized residual value, obtaining a segment step set corresponding to each type of feature channel, where the segment step set corresponding to one type of feature channel includes multiple different quantization steps; extracting a quantization step corresponding to a channel segment to which the quantized residual value belongs from a segment step set corresponding to one type of feature channel; and constructing, based on the quantization step, inverse quantization precision parameters corresponding to the quantized residual values, respectively.

[0016] In order to achieve the above object, the present invention provides: performing an analysis and transformation process on the image block to be processed to obtain image features; performing a residual calculation on the image features to obtain an image residual value and a probability distribution parameter; constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information; performing a quantization process on the image residual value based on the quantization precision parameter to obtain a quantized residual value; and writing the probability distribution parameters and the quantized residual values ​​into an image bitstream.

[0017] In one possible embodiment of the present invention, the step of writing the probability distribution parameters and the quantized residual values ​​into an image bitstream comprises: writing said probability distribution parameters into a first image bitstream; constructing a distribution quantization parameter based on the probability distribution parameters; and writing the quantized residual values ​​into a second image bitstream based on the distribution quantization parameter.

[0018] In one possible embodiment of the present invention, the step of constructing a distribution quantization parameter based on the probability distribution parameters comprises: constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0019] In one possible embodiment of the present invention, the image coding method comprises the steps of: The method further includes writing the feature channel distinction information and / or the spatial point distinction information into the image bitstream.

[0020] In one possible embodiment of the present invention, the image bitstream further comprises a third image bitstream, the third image bitstream being used to transmit differentiation information; After writing the quantized residual values ​​into the second image bitstream based on the distributed quantization parameter, The method further includes writing the feature channel distinction information and / or the spatial point distinction information into the third image bitstream.

[0021] In one possible embodiment of the present invention, the step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: extracting quantization steps corresponding to each type of feature channel from the feature channel distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0022] In one possible embodiment of the present invention, the step of constructing a quantization precision parameter corresponding to each image residual value based on the quantization step comprises: extracting a parameter section threshold from the characteristic channel distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the probability distribution parameter, the parameter interval threshold, and the quantization step.

[0023] In one possible embodiment of the present invention, the step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: analyzing the characteristic channel distinction information; For any image residual value, obtaining a segment step set corresponding to each type of feature channel, wherein the segment step set corresponding to one type of feature channel includes a plurality of different quantization steps; Extracting a quantization step corresponding to a channel segment to which the image residual value belongs from a segment step set corresponding to one type of feature channel; and constructing quantization precision parameters corresponding to the image residual values ​​respectively based on the quantization step.

[0024] In one possible embodiment of the present invention, the step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: extracting quantization steps corresponding to each type of spatial point from the spatial point distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0025] In one possible embodiment of the present invention, the step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information comprises: reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinguishing information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0026] In one possible embodiment of the present invention, the step of writing the probability distribution parameters and the quantized residual values ​​into an image bitstream comprises: performing image decoding based on the probability distribution parameters, the quantized residual value, and the feature channel distinguishing information to obtain a reconstructed image block and an image bit rate; adjusting the feature channel distinguishing information and / or spatial point distinguishing information based on the reconstructed image block and the image bit rate; If the current number of adjustments is equal to or greater than a predetermined number of adjustments, writing the probability distribution parameters and the quantized residual values ​​into an image bitstream.

[0027] In one possible embodiment of the present invention, after the step of adjusting the feature channel distinguishing information and / or spatial point distinguishing information based on the reconstructed image block and the image bit rate, If the current number of adjustments is less than the predetermined number of adjustments, the method further includes returning to the step of constructing quantization precision parameters corresponding to each feature channel based on the feature channel distinguishing information.

[0028] In order to achieve the above object, the present invention provides: an entropy decoding module for obtaining feature channel distinction information and / or spatial point distinction information, decoding the image bitstream, and obtaining quantized residual values; a parameter construction module for constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the feature channel distinction information and / or the spatial point distinction information; an inverse quantization module for, for any quantized residual value, inverse quantizing the quantized residual value based on the inverse quantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; a synthesis transform module for performing a synthesis transform on the reconstructed residual values ​​to obtain a reconstructed image block.

[0029] In order to achieve the above object, the present invention provides: an analysis and transformation module for performing analysis and transformation processing on the image block to be processed and obtaining image features; a residual calculation module for performing residual calculation on the image features to obtain image residual values ​​and probability distribution parameters; a parameter construction module for constructing a quantization precision parameter corresponding to each feature channel based on the feature channel distinguishing information and / or the spatial point distinguishing information; a quantization module for performing a quantization process on the image residual values ​​based on the quantization precision parameter to obtain quantized residual values; an entropy coding module for writing the probability distribution parameters and the quantized residual values ​​into an image bitstream.

[0030] In addition, in order to achieve the above-mentioned object, the present invention further provides a decoding device including a processor, a memory, and a decoding program stored in the memory and executable by the processor, wherein when the decoding program is executed by the processor, the above-mentioned image decoding method is implemented.

[0031] In order to achieve the above object, the present invention further provides an encoding device including a processor, a memory, and an encoding program stored in the memory and executable by the processor, wherein when the encoding program is executed by the processor, the above-mentioned image encoding method is implemented.

[0032] In addition, in order to achieve the above-mentioned object, the present invention further provides a computer-readable storage medium on which an image decoding program and / or an image encoding program is stored, and when the image decoding program is executed, the above-mentioned image decoding method is implemented, or when the image encoding program is executed, the above-mentioned image encoding method is implemented.

[0033] In the present invention, feature channel distinction information and / or spatial point distinction information are obtained, the image bitstream is decoded to obtain quantized residual values, and inverse quantization precision parameters corresponding to each quantized residual value are constructed based on the feature channel distinction information and / or the spatial point distinction information, the quantized residual values ​​are inversely quantized based on the inverse quantization precision parameters, reconstructed residual values ​​are obtained, and synthesis transformation is performed on the reconstructed residual values ​​to obtain a reconstructed image block. Since corresponding inverse quantization parameters are constructed based on the feature channel distinction information and / or spatial point distinction information and inverse quantization is performed, different quantization parameters can be set during the encoding process based on the image texture differences of the feature channels and spatial points, allowing more important features to bear smaller quantization losses and improving encoding and decoding performance. [Brief explanation of the drawings]

[0034] [Figure 1] 1 is a structural schematic diagram of an electronic device in a hardware execution environment according to a technical solution of an embodiment of the present invention; [Figure 2] FIG. 1 is a schematic flow diagram of a first embodiment of an image decoding method of the present invention. [Figure 3] FIG. 1 is a schematic diagram of an image encoding / decoding flow according to an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram of a feature channel in one embodiment of the present invention. [Figure 5] FIG. 2 is a two-dimensional schematic diagram of the feature channel probability distribution parameter mean values ​​according to an embodiment of the present invention. [Figure 6] FIG. 10 is a schematic flow diagram of a second embodiment of the image decoding method of the present invention. [Figure 7]FIG. 2 is a schematic diagram of a decoding process according to one embodiment of the present invention. [Figure 8] FIG. 10 is a schematic flow diagram of a third embodiment of the image decoding method of the present invention. [Figure 9] FIG. 2 is a schematic diagram of an identity matrix according to an embodiment of the present invention. [Figure 10] FIG. 1 is a schematic diagram of matrix division according to one embodiment of the present invention. [Figure 11] FIG. 10 is a schematic flow diagram of a fourth embodiment of the image decoding method of the present invention. [Figure 12] FIG. 1 is a schematic diagram of spatial point classification according to one embodiment of the present invention. [Figure 13] FIG. 1 is a schematic flow diagram of a first embodiment of an image coding method of the present invention. [Figure 14] FIG. 2 is a schematic diagram of an image encoding process according to an embodiment of the present invention; [Figure 15] FIG. 4 is a schematic flow diagram of a second embodiment of the image coding method of the present invention. [Figure 16] FIG. 10 is a schematic flow diagram of a third embodiment of the image coding method of the present invention. [Figure 17] FIG. 10 is a flow diagram of the optimal threshold mode flag bit operation of one embodiment of the present invention. [Figure 18] 1 is a flowchart of parameter optimization according to an embodiment of the present invention. [Figure 19] FIG. 1 is a block diagram showing the configuration of a first embodiment of an image decoding device according to the present invention. [Figure 20] 1 is a block diagram showing the configuration of a first embodiment of an image encoding device according to the present invention. The realization of the object, the functional features and advantages of the present invention will be further explained in combination with the embodiments with reference to the drawings. DETAILED DESCRIPTION OF THE INVENTION

[0035] It should be understood that the specific examples described herein are used only to illustrate the present invention and are not intended to limit the present invention.

[0036] Please refer to FIG. 1, which is a structural schematic diagram of an encoding device or a decoding device in a hardware execution environment according to a technical solution of an embodiment of the present invention.

[0037] As shown in FIG. 1 , the electronic device may include a processor 1001 such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize communication connections between these components. The user interface 1003 may include input units such as a display and a keyboard, and optionally, the user interface 1003 may further include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM) such as a magnetic disk memory. The memory 1005 may optionally be a storage device independent of the processor 1001.

[0038] As will be appreciated by those skilled in the art, the structure shown in FIG. 1 is not limiting of electronic devices that may include more or fewer components, some combinations of components, or different arrangements of components.

[0039] As shown in FIG. 1, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, an image encoding program, and / or an image decoding program.

[0040] In the electronic device shown in Figure 1, the network interface 1004 is mainly used for data communication with a network server, and the user interface 1003 is mainly used for data interaction with a user. The processor 1001 and memory 1005 in the electronic device of the present invention may be provided in an encoding device or a decoding device, and the electronic device calls an encoding program stored in the memory 1005 via the processor 1001 and executes an image encoding method related to an embodiment of the present invention, or calls an image decoding program stored in the memory 1005 via the processor 1001 and executes an image decoding method related to an embodiment of the present invention.

[0041] In some embodiments of the present invention, the quantization accuracy is improved by taking into account the differences between feature channels to improve the performance of the quantizer. For example, feature channels may be classified based on the differences between feature channels, with feature channels of different importance having different quantization accuracy. Features may also be classified based on the differences in texture complexity, with feature regions of different texture complexity having different quantization accuracy. In the present invention, the information used to classify feature channels is related information that distinguishes feature channels, and the features classified according to this information are quantized with different accuracy, i.e., by using different quantization steps, more important features bear smaller quantization loss, resulting in more accurate prediction of the features to be coded, a better feature reconstruction value y_hat, and ultimately better coding performance.

[0042] An embodiment of the present invention provides an image decoding method, and FIG. 2 is a schematic flow diagram of a first embodiment of the image decoding method of the present invention.

[0043] In this embodiment, the image decoding method includes the following steps S10 to S40.

[0044] In step S10, the feature channel distinguishing information and / or the spatial point distinguishing information is obtained, the image bitstream is decoded, and the quantized residual value is obtained.

[0045] In addition, the entity that executes this embodiment may be a decoding device when performing decoding processing on image data, and the decoding device may be an electronic device such as an imaging device, a mobile terminal, a personal computer, a server, or any other device that can realize the same or similar functions.This embodiment is not limited to this, and in this embodiment and each of the following embodiments, the image decoding method of the present invention will be explained using a decoding device as an example.

[0046] In an image encoding process, the encoding device typically decodes the encoded image bitstream after encoding is completed, and determines whether the parameters used during encoding need to be adjusted based on the image quality of the image obtained by decoding, so the entity that performs this embodiment may be the encoding device.

[0047] In an embodiment of the present invention, some technical terms related to the process of encoding or decoding an image, including Quantization and Dequantization, Scalar Quantization (SQ), Entropy Encoding, Convolutional Neural Network (CNN), Feature Channel, etc., will be described herein.

[0048] Quantization is a process of mapping continuous signal values ​​(or a large number of discrete values) to a finite number of discrete amplitudes, achieving a many-to-one mapping of signal values. In video coding, after a residual signal is transformed, the transform coefficients usually have a large dynamic range. Therefore, quantizing the transform coefficients can effectively reduce the signal value space and achieve better compression. In addition, due to the many-to-one mapping mechanism, distortion inevitably occurs in the quantization process, which is the root cause of distortion in video coding.

[0049] Inverse quantization is the reverse process of quantization, mapping the quantized coefficients to a reconstructed signal in the input signal space, where the reconstructed signal is an approximation of the input signal.

[0050] Scalar quantization is the most basic quantization method, which maps a continuous signal (or a large number of discrete-valued signals) into several discrete signals. Specifically, the input of scalar quantization is a one-dimensional scalar signal. First, the input signal space is divided into a series of non-intersecting intervals, and a representative signal is selected for each interval. For each input signal, scalar quantization maps it to the representative signal of the interval in which it is located.

[0051] The simplest scalar quantization method is uniform scalar quantization, which divides the input signal space into equally spaced intervals, and the representative signal of each interval is the midpoint of the interval. The interval length is called the quantization step, the interval index is called the level, and the parameter representing the quantization step is the quantization parameter (QP).

[0052] The optimal scalar quantizer is the Lloyd-Max quantizer, which takes into account the distribution of the input signal, the division of which is uneven, the representative signal of each interval is the probability center of gravity of that interval, and the boundary point between two adjacent intervals is the midpoint of the representative signals of these two intervals.

[0053] Entropy coding is a coding method that follows the entropy principle to ensure that no information is lost during the coding process. Information entropy is the average amount of information (degree of uncertainty) in the information source. Common entropy coding methods include Shannon coding, Huffman coding, and arithmetic coding.

[0054] Neural network (NN): A neural network refers to an artificial neural network, rather than a biological one. A neural network is a computational model composed of a large number of interconnected nodes (also called neurons). In an artificial neural network, neuronal processing units can represent different objects, such as features, alphabets, concepts, or several meaningful abstract modes. The processing units in a network can be divided into three types: input units, output units, and hidden units. Input units receive external signals and data, output units output the system processing results, and hidden units are located between the input and output units and are not observable from outside the system. The connection weights between neurons reflect the connection strength between units, and information representation and processing are reflected in the connection relationships between network processing units. An artificial neural network is a non-programmed, brain-like information processing method. Its essence is to obtain parallel, distributed information processing functions through network transformations and dynamic behavior, imitating the information processing functions of the human nervous system to different degrees and levels. Currently, commonly used neural networks in the video processing field include convolutional neural networks (CNNs), recurrent neural networks (RNNs), fully connected networks, etc.

[0055] Convolutional neural networks are feedforward neural networks and are one of the representative network structures in deep learning technology. Their artificial neurons can respond to peripheral units within a partial coverage range, and they exhibit excellent performance in large-scale image processing.

[0056] Generally, the basic structure of a CNN includes two layers. One is a feature extraction layer (also called a convolutional layer), in which the input of each neuron is connected to the local receptive field of the previous layer to extract the local feature. Once the local feature is extracted, the spatial relationship between the local feature and other features is also determined. The other is a feature mapping layer (also called an activation layer), in which each computational layer of the network consists of multiple feature mappings. Each feature mapping is a plane, and all neurons in the plane have equal weights. The feature mapping structure may use a sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since neurons in a single mapping plane share weights, the number of free parameters of the network is reduced.

[0057] One of the advantages of CNN compared to image processing algorithms is that it avoids complex image preprocessing processes (such as artificial feature extraction) and can directly input the original image for end-to-end learning.One of the advantages of CNN compared to traditional neural networks is that traditional neural networks all use a fully connected method, meaning that all neurons from the input layer to the hidden layer are fully connected, resulting in a huge number of parameters and making network training time-consuming and difficult, whereas CNN avoids this difficulty by using methods such as local connections and weight value sharing.

[0058] In a neural network, an input signal passes through a convolutional layer to obtain a number of feature maps, each of which is called a feature channel and contains some information of the input signal.

[0059] The feature channel distinction information may be distinction information for distinguishing image feature channels, and may include a rule for distinguishing feature channels and a quantization accuracy parameter for quantizing residual values ​​corresponding to each type of feature channel after the distinction. The spatial point distinction information may be distinction information for distinguishing types of points in an image, and may include a rule for distinguishing each spatial point in an image based on image texture, and may, as necessary, include a quantization accuracy parameter for quantizing residual values ​​corresponding to each type of spatial point after the distinction.

[0060] Here, the distinction rule for feature channels or spatial points may be set based on a probability distribution parameter, or may be set based on a value such as the mean value or variance of the probability distribution parameter. For example, a corresponding threshold is set, multiple value intervals are obtained, and the feature channels are divided into multiple types based on the value interval to which the corresponding probability distribution parameter belongs.

[0061] In step S20, a dequantization precision parameter corresponding to each quantized residual value is constructed based on the feature channel distinguishing information and / or the spatial point distinguishing information.

[0062] In addition, constructing an inverse quantization accuracy parameter corresponding to each quantized residual value based on the feature channel distinction information and / or spatial point distinction information may also mean obtaining an inverse quantization accuracy parameter to be used when performing inverse quantization processing on each quantized residual value by determining a feature channel type or a spatial point type corresponding to each quantized residual value based on the feature channel distinction information and / or spatial point information.

[0063] In step S30, the quantized residual value is dequantized based on the dequantization precision parameter to obtain a reconstructed residual value.

[0064] After determining the inverse quantization precision parameter corresponding to each quantized residual value, the quantized residual value is subjected to the inverse process of quantization, i.e., inverse quantization, based on the inverse quantization precision parameter, and the value obtained after the process can be used as the reconstructed residual value.

[0065] In step S40, a synthesis transform is performed on the reconstructed residual values ​​to obtain a reconstructed image block.

[0066] After obtaining the reconstructed residual value, the mean value used in calculating the feature residual value can be obtained by context prediction, and the reconstructed residual value can be reconstructed into an image feature by the mean value, and then the image feature can be reconstructed into an image block by a synthesis transformation network, thereby obtaining a reconstructed image block.

[0067] If the encoding device divides the image data into only one image block during the encoding process and processes it, the reconstructed image block obtained at this time is reconstructed image data corresponding to the image data. If the encoding device divides the image data into multiple image blocks during the encoding process and processes it, the reconstructed image block obtained at this time is reconstructed image data corresponding to an image block in the image data.

[0068] For ease of understanding, the description will be made with reference to Figures 3, 4 and 5, but this does not limit the technical solution. Figure 3 is a schematic flow diagram of image encoding and decoding in this embodiment. As shown in Figure 3, the image encoding and decoding process mainly involves modules such as an analysis transformation network, a synthesis transformation network, context-based prediction, quantization, entropy coding, a super-coding network, and a super-scale decoding network.

[0069] Analysis-transformation network: Transforms an image x into features y in the feature domain (latent domain) so that all subsequent processes can operate in the latent domain.

[0070] Super coding network: The probability distribution parameter σ of the residual r is input to obtain the super prior information z_hat, which is written into the image bitstream biatstream#1 and mainly transmits the probability model parameter σ of the residual.

[0071] Context-based prediction module: The input of this module includes not only z_hat but also the decoded y_hat. The joint input of both obtains a more accurate mean value mu, which is the predicted value of the original feature, and adds the reconstructed residual value r_hat to obtain the reconstructed y_hat.

[0072] Super-scale decoding network: The output z_hat of the super-encoding network is used as the input of the super-scale decoding network in another path to obtain the probability distribution parameter σ, which is quantized to obtain the distribution quantization parameter σ_hat, which is used together with the second image bitstream (Bitstream#2) to obtain the quantized residual value r_q.

[0073] Synthesis transformation module: The obtained reconstructed latent domain feature y_hat is passed through the synthesis transformation module to obtain a reconstructed image.

[0074] Entropy coding: A lossless coding method based on the principle of information entropy, which converts a series of elementary codes (e.g., transform coefficients, mode information, etc.) used to represent a video sequence into a binary bitstream and removes statistical redundancy in these video elementary codes.

[0075] Quantization: A "many-to-one" mapping process that results in signal loss. Quantization operates on the residual r, changing the signal's value range, allowing the encoder to provide a good approximation of the original signal with fewer codes, improving compression ratio. Quantization also operates on σ, resulting in σ_hat, which can better estimate the probability distribution of the quantized residual r_q. Inverse quantization is the inverse process of quantization. The quantization precision parameter or inverse quantization precision parameter used in the quantization and inverse quantization processes is constructed based on feature channel differentiation information and / or spatial point differentiation information.

[0076] FIG. 4 is a schematic diagram of the feature channels in this embodiment, and FIG. 5 is a two-dimensional schematic diagram of the feature channel probability distribution parameter average values ​​in this embodiment. As shown in FIG. 3 above, the feature y obtained after processing image x with the analysis-transformation network contains C feature channels, where the number of C depends on the number of convolution kernels used in the analysis-transformation network. As shown in FIG. 4, each feature channel contains different amounts of information. Some feature channels contain a large amount of information and are nearly consistent with the input image, while some feature channels contain very little information and are basically noise. Therefore, adopting different quantization precisions for different feature channels can help achieve better coding performance.

[0077] In addition, the σ of all feature channels can be averaged to obtain a two-dimensional image. As shown in Figure 5, it can be seen that some regions have higher texture values, i.e., these regions are more important. Allocating more codewords to parts of the image with complex textures can help improve coding performance.

[0078] In this embodiment, feature channel distinction information and / or spatial point distinction information are obtained, an image bitstream is decoded to obtain quantized residual values, an inverse quantization precision parameter corresponding to each quantized residual value is constructed based on the feature channel distinction information and / or the spatial point distinction information, the quantized residual values ​​are inversely quantized based on the inverse quantization precision parameter, reconstructed residual values ​​are obtained, and synthesis transformation is performed on the reconstructed residual values ​​to obtain a reconstructed image block. Since the corresponding inverse quantization precision parameter is constructed based on the feature channel distinction information and / or the spatial point distinction information and then inverse quantization is performed, different quantization parameters can be set during the encoding process based on the image texture differences of the feature channels and spatial points, allowing more important features to bear smaller quantization losses and improving encoding and decoding performance.

[0079] FIG. 6 is a schematic flow diagram of a second embodiment of the image decoding method of the present invention.

[0080] In this embodiment, the image bitstream may include a first image bitstream and a second image bitstream, where the first image bitstream is used to transmit decoding auxiliary information and the second image bitstream is used to transmit residual data.

[0081] Based on the first embodiment, the step S10 of the image decoding method of this embodiment includes steps S101 to S104.

[0082] In step S101, feature channel distinguishing information and / or spatial point distinguishing information is obtained, and coding distribution parameters are extracted from the first image bitstream.

[0083] Extracting the coding distribution parameters from the first image bitstream may involve entropy decoding the first image bitstream and reading decoding auxiliary information included in the first image bitstream to obtain the coding distribution parameters.

[0084] In step S102, the encoded distribution parameters are decoded to obtain probability distribution parameters.

[0085] Decoding the encoded distribution parameters to obtain the probability distribution parameters may be decoding the decoding auxiliary information read through a super-scale decoding network to obtain the probability distribution parameters.

[0086] In step S103, the probability distribution parameters are quantized to obtain distribution quantization parameters.

[0087] When the encoding device writes the residual data into the second image bitstream, it encodes and writes it based on the quantized probability distribution parameters. Therefore, when decoding the second image bitstream, it is necessary to obtain and decode the quantized probability distribution parameters. In this case, to ensure that the second image bitstream can be decoded successfully, the probability distribution parameters can be quantized in the same manner as when the encoding device encodes, and the distribution quantization parameters can be obtained.

[0088] In particular, in order to ensure that the obtained distribution quantization parameters are consistent with those used by the encoding device during encoding, it is necessary to ensure that the parameters used to quantize the probability distribution parameters are consistent with those used by the encoding device during encoding. In this embodiment, step S103 is as follows: constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0089] Constructing a probability quantization parameter based on feature channel distinction information and / or spatial point distinction information may involve extracting a parameter used in quantizing a probability distribution parameter from the feature channel distinction information and / or spatial point distinction information.

[0090] In one possible embodiment of this example, to facilitate decoding by the decoding device, the encoding device may write feature channel distinction information and / or spatial point distinction information used in the encoding process into the first image bitstream or the second image bitstream, in which case obtaining the feature channel distinction information and / or spatial point distinction information may involve decoding the first image bitstream or the second image bitstream and extracting the feature channel distinction information and / or spatial point distinction information.

[0091] In one possible embodiment of this embodiment, in order to avoid bitstream confusion, the encoding device may further set a third image bitstream, and transmit feature channel distinction information and / or spatial point distinction information used in the encoding process through the third image bitstream, and in this case, obtaining the feature channel distinction information and / or spatial point distinction information may be decoding the third image bitstream and extracting the feature channel distinction information and / or spatial point distinction information.

[0092] In step S104, the second image bitstream is decoded according to the distribution quantization parameter to obtain a quantized residual value.

[0093] When writing the quantized residual value into the second image bitstream, the encoding device performs entropy encoding on the quantized residual value based on the distribution quantization parameter after quantization and writes it. In this case, after obtaining the distribution quantization parameter, the encoding device can perform an inverse operation, i.e., entropy decoding, on the second image bitstream based on the distribution quantization parameter to obtain the quantized residual value.

[0094] For ease of understanding, the description will be made with reference to Fig. 7, but this does not limit the present technical solution. Fig. 7 is a schematic diagram of the decoding process of this embodiment, and as shown in Fig. 7, the decoding side receives at least two image bitstreams (a first image bitstream Bitstream #1 and a second image bitstream Bitstream #2), and may also receive a third image bitstream Bitstream #3.

[0095] In this case, the decoding process includes the following: Analyzing z_hat from the Bitstream#1 auxiliary bitstream, and using z_hat to obtain a parameter σ of the residual probability distribution model. Obtaining related information (feature channel distinguishing information and / or spatial point distinguishing information) that distinguishes feature channels or texture regions, and using this information to classify feature channels or spatial points. Different types of feature channels or different types of spatial points have different quantization precisions. The related information that distinguishes feature channels may be transmitted from the encoding device to the decoding device via Bitstream#1 or Bitstream#2, or may be transmitted via a new bitstream, Bitstream#3. Alternatively, the related information may be pre-written in the encoding / decoding device without requiring bitstream transmission. Entropy decoding is performed on Bitstream#2, and σ_hat obtained by quantizing σ is combined to obtain the quantized residual r_q. The related information that distinguishes feature channels or spatial points is used in the inverse quantization of the quantized residual r_q, so that the quantized residuals r_q and σ corresponding to different types of feature channels or different types of spatial points have different quantization precisions.

[0096] FIG. 8 is a schematic flow diagram of a third embodiment of the image decoding method of the present invention.

[0097] Based on the first embodiment, the step S20 of the image decoding method of this embodiment includes steps S201 to S202.

[0098] In step S201, the quantization steps corresponding to each type of feature channel are extracted from the feature channel distinction information.

[0099] When distinguishing the feature channels, the feature channels can be divided into multiple types, and different quantization steps can be assigned to the different types of feature channels. In this case, the feature channel distinction information includes a classification rule for the feature channels and a quantization step corresponding to each type of feature channel.

[0100] Here, the classification rules for the feature channels may be set as in the following methods 1 to 4.

[0101] Method 1: Set based on the feature channel number.

[0102] A corresponding quantization step is set for each feature channel corresponding to each number, and feature channels with the same quantization step are classified into one type. For example, if the feature channel numbers are 1 to X, then the feature channel quantization step list may be channel_list=[channel1, channel2, ..., channelX].

[0103] Furthermore, the same feature channel may contain components such as luminance or chrominance, and in this case, luminance or chrominance can be further divided. For example, if the luminance channel number is 1 to X1 and the chrominance channel number is 1 to X2, the luminance channel quantization step list is channel_listY=[channel1, channel2, ..., channelX1], and the chrominance channel quantization step list is channel_listUV=[channel1, channel2, ..., channelX2].

[0104] Method 2: Set the corresponding classification based on the channel flag bit list.

[0105] A corresponding flag bit is set for each feature channel corresponding to each number, and the type of feature channel corresponding to each number is identified by the flag bit.

[0106] For example, taking the luminance channel as an example, the luminance channel numbers are 1 to 128, and the flag bit list is channel_listY=[flag0, flag1, ..., flag128], and the value of flag is the type corresponding to the luminance channel of that number. If there are only two types, the values ​​of flag are 0 and 1.

[0107] Method 3: Distinguish the types of feature channels by setting a threshold.

[0108] The type of the corresponding feature channel is set according to a threshold calculated by a probability distribution parameter such as the mean value or variance of the probability distribution parameter σ, and a related threshold calculated by the quantized residual r_q such as the sum of the absolute values ​​of the quantized residual values ​​or the bit rate, e.g., thr_list=[thr1, thr2, ..., thrX].

[0109] Method 4: Differentiate the type of feature channel by the number of channels. The channels are sorted using a related threshold list calculated according to a distribution parameter σ, such as the mean or variance of σ, or the sum of the absolute values ​​of the quantization residual r_q, or the bit rate. Then, multiple thresholds (multiple thresholds can be expressed as a list, e.g., num_list=[num1, num2, ..., numX], where X is the length of the list, i.e., the number of thresholds) are set, and the channels are classified according to the number of thresholds. For example, the first three are sorted into one type, and the remaining ones are sorted into another type. In this case, the threshold for the number of channels can be set to 3, so num_list=[3].

[0110] In step S202, based on the quantization step, a dequantization precision parameter corresponding to each quantized residual value is constructed.

[0111] After obtaining the quantization steps corresponding to each type of feature channel, parameters such as the quantization steps used in quantizing each quantized residual value can be obtained based on the feature channels corresponding to each quantized residual value, and then the inverse quantization precision parameters required to inverse quantize each quantized residual value can be calculated.

[0112] For example, if the feature channels are divided into X types, the quantization steps corresponding to each type of feature channel in the feature channel distinction information can be expressed as a list scale_list=[scale1, scale2, ..., scaleX], where X is the number of quantization steps. At this time, if the quantization residual value corresponding to the jth type of feature channel is rqj, the feature residual is rj, and the reconstructed residual value obtained by inverse quantization is r_hatj, then the quantization process rqj=round(rj*scalej*scalei), and the inverse quantization process r_hat j=rqj / (scalej*scalei), where scalej is the quantization step used when quantizing the feature residual corresponding to the j-th feature channel added based on the feature channel, and scalei is the quantization step used when quantizing the feature residual corresponding to the j-th feature channel in the original method, and scalei may be set to 1, that is, the quantization step of the original method may or may not be retained. Similarly, in each of the following embodiments, it is freely selectable whether to retain the quantization step of the original method.

[0113] In specific implementation, different quantization steps may be set for the same feature channel, for example, by setting a threshold value to divide into multiple value intervals, and further setting according to the value interval to which the probability distribution parameter or the value calculated based on the probability distribution parameter belongs, and the step S202 in this embodiment is as follows: decoding the image bitstream to obtain probability distribution parameters; extracting a parameter section threshold from the characteristic channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the probability distribution parameter, the quantization step, and the parameter interval threshold.

[0114] When constructing an inverse quantization accuracy parameter corresponding to a quantized residual value, first obtain the type of feature channel to which the quantized residual value belongs, obtain the quantization step and parameter interval threshold corresponding to the feature channel of that type, and divide it into multiple value intervals based on the parameter interval threshold, then calculate the value interval to which the probability distribution parameter corresponding to the quantized residual value belongs (which may be a value interval to which a value such as the mean or variance of the probability distribution parameter belongs), select a corresponding quantization step from the multiple quantization steps corresponding to the feature channel of that type based on the value interval, and then calculate the corresponding inverse quantization accuracy parameter when inverse quantizing the quantized residual value based on the quantization step. Here, the multiple quantization steps corresponding to one type of feature channel each correspond to the divided value interval.

[0115] For example, suppose the feature channel corresponding to the quantized residual value is of type i, the quantization steps corresponding to the feature channel of this type include scale1, scale2, ..., scaleN, and the parameter interval thresholds include different thresholds thr1, thr2, ..., thrN (thr1, thr2, ..., thrN are in descending order), then the value intervals may include (thr1, ∞), (thr2, thr1], ..., (thrN, thrN-1], and each value interval corresponds one-to-one to a quantization step. In this case, if the probability distribution parameter σ corresponding to the quantized residual value is σ>thr1, the corresponding quantization step is scale1, and if the probability distribution parameter thr1≧σ>thr2, the corresponding quantization step is scale2.

[0116] In specific implementation, the same type of feature channel may be further classified, for example, one type of feature channel may be divided into multiple segments, and each segment may correspond to a different quantization step. In this case, step S20 of this embodiment may be: analyzing the feature channel distinction information to obtain segment step sets corresponding to each type of feature channel; extracting a quantization step corresponding to each channel segment from the segment step set; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0117] After a type of feature channel is further divided, each type of feature channel can correspond to multiple different quantization steps, which can be stored in the form of a set, and the feature channel distinguishing information can be analyzed to obtain a segment step set corresponding to each type of feature channel, and the quantization step corresponding to each channel segment can be extracted from the segment step set. Here, the segmentation methods of different types of feature channels can be different or the same, and this embodiment is not limited thereto.

[0118] In this case, constructing the inverse quantization precision parameters corresponding to the quantized residual values ​​based on the quantization step may involve obtaining a feature channel type corresponding to the quantized residual value, determining a channel segment to which the quantized residual value belongs in the feature channel of that type, and determining parameters such as the quantization step to be used when calculating the quantized residual value based on the feature channel type and the channel segment, thereby calculating the inverse quantization precision parameters required to inverse quantize the quantized residual value.

[0119] For example, taking the i-th type of feature channel as an example, the i-th type of feature channel can be divided into X segments, and the segment step set corresponding to the i-th type of feature channel can be expressed as scale_listi=[scalei1...scaleij...sscleiX], where scaleij is the quantization step of the j-th segment in the i-th type of feature channel, and the value of j is [1,X].

[0120] In specific implementation, the features in the same feature channel may be divided into multiple categories, and the classification rule may be one of the following:

[0121] Rule 1: A list of associated thresholds calculated by a distribution parameter σ, such as the mean or variance of σ, or a list of associated thresholds calculated by the quantized residual r_q, such as the sum of the absolute values ​​of the quantized residual r_q or the bit rate, and the length of the list X can be used for classification. The thr luma and chroma components may share one list thr_list=[thr1, thr2, ..., thrX] or may be transmitted separately.

[0122] Rule 2: A channel two-dimensional flag bit matrix can be used for classification. As shown in Figure 9, the luma and chroma components can share one matrix or be transmitted separately, similar to the channel flag bit list in Rule 1. Different flag bits indicate different classifications.

[0123] Similar to Rule 3, row, column, diagonal column number and list length, channel number can be used for classification, the schematic diagram of which is shown in FIG.

[0124] Rule 4: The transmitted row, column, and diagonal flag bits are similar to the channel flag bit list and can be used for classification.

[0125] Rule 5: A list of numbers for rows, columns, and diagonals, num_list=[num1, num2, ..., numX], with a list length of X, is similar to a list of channel numbers and can be used for classification.

[0126] Rule 6: The row, column, and diagonal flag bits are similar to the channel flag bits and can be used for classification.

[0127] Here, we will refer to Figures 9 and 10. Figure 9 is a schematic diagram of the discrimination matrix of this embodiment. As shown in Figure 9, each related data in the matrix is ​​divided into multiple types, and data corresponding to each type can be made to correspond to different quantization steps. At this time, classification marks can be applied to the related feature data within a channel in the form of a matrix. As shown in Figure 9, 0 and 1 in the matrix respectively represent the classification marks of each related feature data within the same channel.

[0128] FIG. 10 is a schematic diagram of the matrix division in this embodiment. The specific row, column, and diagonal column division may be as shown in FIG. 10. From left to right, the related feature data in a channel can be divided into multiple rows; from top to bottom, the related feature data in a channel can be divided into multiple columns; and from the bottom left to the top right, the related feature data in a channel can be divided into multiple diagonal columns.

[0129] In one possible embodiment of this embodiment, each feature channel may be divided into blocks and set different quantization steps. In this case, step S20 of this embodiment can be extracting image block division information and parameter setting rules corresponding to each feature channel from the feature channel distinction information; decoding the image bitstream to obtain probability distribution parameters; calculating a distribution parameter average value corresponding to each image block division information based on the probability distribution parameters; a step of matching the distribution parameter mean value with the parameter setting rule to obtain a quantization step corresponding to each image block division information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0130] During encoding, each feature channel may be divided into blocks, and the block division rule may be recorded in the feature channel distinction information. Then, different quantization steps are set according to different average values ​​of the probability distribution parameters corresponding to these blocks, where the size of the blocks can be freely set.

[0131] Extracting image block division information and a parameter setting rule corresponding to each feature channel from the feature channel distinction information may involve extracting a parameter setting rule and a block division rule from the feature channel distinction information, and determining division blocks corresponding to each feature channel based on the block division rule, thereby obtaining image block division information corresponding to each feature channel. The parameter setting rule includes quantization steps corresponding to different distribution parameter mean values.

[0132] In practical use, calculating the distribution parameter average value corresponding to each image block division information based on the probability distribution parameters may also mean obtaining the distribution parameter average value by calculating the average value of the probability distribution parameters corresponding to each image block division information.

[0133] In the implementation process, matching the distribution parameter mean value with the parameter setting rule and obtaining the quantization step corresponding to each image block division information may also be matching the distribution parameter mean value with a plurality of value intervals included in the parameter setting rule and setting the quantization step corresponding to the value interval to which the distribution parameter mean value belongs as the quantization step corresponding to the image block division information.

[0134] After obtaining the quantization step corresponding to the image block division information, parameters such as the quantization step to be used when calculating the quantized residual value are determined based on the image block division information to which the quantized residual value belongs, thereby making it possible to calculate the inverse quantization precision parameter required to inverse quantize the quantized residual value.

[0135] In this embodiment, a probability quantization parameter is constructed based on feature channel distinction information and / or spatial point distinction information, and the probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter. Since the probability quantization parameter is constructed based on feature channel distinction information and / or spatial point distinction information, and the probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter, different quantization parameters can be set for the probability distribution parameter based on the image texture differences of the feature channels and spatial points, further improving the performance of the encoding method of this embodiment.

[0136] FIG. 11 is a schematic flow diagram of a fourth embodiment of the image decoding method of the present invention.

[0137] Based on the first embodiment, the step S20 of the image decoding method of this embodiment includes steps S201' to S202'.

[0138] In step S201', the quantization steps corresponding to each type of spatial point are extracted from the spatial point distinguishing information.

[0139] When setting the quantization step, each spatial point in an image may be divided into multiple types based on the image texture, and different quantization steps may be set for different types of spatial points. For example, if the spatial points in an image are divided into X types based on the image texture, the quantization step for the jth type of spatial point is scale_j, where j has a value range of [1,X].

[0140] Here, when classifying spatial points, they can be distinguished based on the complexity of the image texture corresponding to the spatial point. For example, three complexity intervals, [0, F1], (F1, F2], and (F2, ∞), are set, and the spatial points are divided into three types based on the complexity interval to which the complexity of the image texture corresponding to the spatial point belongs.

[0141] In step S202', based on the quantization step, a dequantization precision parameter corresponding to each quantized residual value is constructed.

[0142] Constructing the inverse quantization precision parameters corresponding to each quantized residual value based on the quantization steps corresponding to each type of spatial point may involve obtaining parameters such as the quantization steps used in calculating the quantized residual values ​​based on the type of spatial point corresponding to the quantized residual value, and calculating the inverse quantization precision parameters required to inverse quantize each quantized residual value.

[0143] Each spatial point includes C (C is the number of feature channels). After different types of spatial points are divided, the spatial points may be further divided into multiple points so that different feature channels in one type of spatial point correspond to different quantization steps. In this case, the spatial point distinguishing information and the feature channel distinguishing information can be used in combination. In this case, step S20 of this embodiment can be reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

[0144] When the points included in each spatial point at each type of spatial point are further divided based on the differences in the feature channels, the points of the different feature channels corresponding to one type of spatial point may correspond to different quantization steps. In this case, the distinguishable types of spatial points may be first read from the spatial point distinction information, and then the quantization steps corresponding to each feature channel at each type of spatial point may be read.

[0145] For example, if spatial points are divided into two types, a and b, and the feature channels included in a and b are all 1 to X, the quantization steps corresponding to each feature channel at spatial points of type a may be expressed as scale_lista=[scale1, scale2, ..., scaleX], and the quantization steps corresponding to each feature channel at spatial points of type b may be expressed as scale_listb=[scale1, scale2, ..., scaleX].

[0146] In a specific implementation, in order to allow a decoding device to clearly understand the specific quantization step setting method, a flag bit corresponding to the image bitstream may be set, and how to obtain the quantization step may be determined according to the flag bit. For example, when the flag bit is set to 0, the quantization step is determined based on the feature channel distinction information; when the flag bit is set to 1, the quantization step is determined based on the spatial point distinction information; and when the flag bit is set to 2, the quantization step is determined by combining the feature channel distinction information and the spatial point distinction information.

[0147] In this embodiment, only one method of combining spatial point distinction information and feature channel distinction information has been described, but further dividing multiple points contained in a spatial point may also utilize the embodiment of further dividing the feature channel provided in any of the above embodiments, and this embodiment is not limited to this.

[0148] For ease of understanding, the description will be made with reference to Fig. 12, but the present technical solution is not limited thereto. Fig. 12 is a schematic diagram of spatial point classification in this embodiment, in which spatial points can be divided into first, second, and third classes (as indicated by the arrows in the figure) based on the complexity of the image texture corresponding to the spatial points, and the specific divided spatial points are as shown in Fig. 12 (the color depth of each point in Fig. 12 indicates the complexity of the image texture, and points with the same color depth belong to the same class, and the darker the color, the higher the complexity of the image texture).

[0149] In this embodiment, the quantization steps corresponding to each type of spatial point are extracted from the spatial point distinction information, and the dequantization precision parameters corresponding to each quantized residual value are constructed based on the quantization steps. Since the dequantization precision parameters corresponding to each quantized residual value are constructed based on the quantization steps corresponding to each type of spatial point, different quantization steps can be set for spatial points with different image textures, and differentiation can be performed based on the differences in image textures when performing quantization and dequantization.

[0150] An embodiment of the present invention provides an image encoding method, and FIG. 13 is a schematic flow diagram of a first embodiment of the image encoding method of the present invention.

[0151] In this embodiment, the image encoding method includes the following steps S100 to S500.

[0152] In step S100, an analysis and conversion process is performed on the image block to be processed to obtain image features.

[0153] In addition, the entity that performs this embodiment may be an encoding device when performing encoding processing on image data, and the encoding device may be an electronic device such as a personal computer or a server, or may be another device that can realize the same or similar functions.This embodiment is not limited to this, and in this embodiment and each of the following embodiments, the image encoding method of the present invention will be explained using an encoding device as an example.

[0154] Performing an analysis and conversion process on an image block to be processed and obtaining image features may also mean performing an analysis and conversion process on an image block to be processed via an analysis and conversion network and extracting feature information to obtain image features.

[0155] In step S200, a residual calculation is performed on the image features to obtain image residual values ​​and probability distribution parameters.

[0156] Performing residual calculation on the image features and obtaining image residual values ​​and probability distribution parameters may include generating predicted values ​​through a context-based prediction module, performing residual calculation based on the image features and the predicted values, obtaining image residual values, and analyzing the image residual values ​​through a super-coding network to obtain the probability distribution parameters.

[0157] In step S300, a quantization precision parameter corresponding to each image residual value is constructed based on the feature channel distinguishing information and / or the spatial point distinguishing information.

[0158] The feature channel distinguishing information may be distinguishing information for distinguishing image feature channels, and the spatial point distinguishing information may be distinguishing information for distinguishing types of points in an image. Both the feature channel distinguishing information and the spatial point distinguishing information may be preset by an administrator of the encoding device according to actual needs.

[0159] Note that constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or spatial point distinction information may also mean determining a feature channel type or spatial point type corresponding to each image residual value based on the feature channel distinction information and / or spatial point information, and reading a quantization precision parameter required for quantizing each image residual value from the feature channel distinction information and / or spatial point distinction information based on the feature channel type or spatial point type. Here, the quantization precision parameter may be a parameter that can limit the quantization precision, such as a quantization step.

[0160] In step S400, a quantization process is performed on the image residual value based on the quantization precision parameter to obtain a quantized residual value.

[0161] Performing a quantization process on the image residual value based on the quantization accuracy parameter and obtaining the quantized residual value may also mean performing a quantization calculation on the image residual value based on the quantization accuracy parameter and using the calculated value as the quantized residual value.

[0162] In step S500, the probability distribution parameters and the quantized residual values ​​are written into an image bitstream.

[0163] Writing the probability distribution parameters and the quantized residual values ​​into the image bitstream may include processing the probability distribution parameters through a super-coding network and then entropy encoding them to write them into the image bitstream, and entropy encoding the quantized residual values ​​based on the quantized probability distribution parameters and writing them into the image bitstream.

[0164] In a specific implementation, the image bitstream may include a first image bitstream and a second image bitstream, where the first image bitstream is used to transmit decoding auxiliary information, and the second image bitstream is used to transmit residual data. In this embodiment, step S500 may include: writing said probability distribution parameters into said first image bitstream; constructing a distribution quantization parameter based on the probability distribution parameters; and writing the quantized residual values ​​into the second image bitstream based on the distribution quantization parameter.

[0165] Writing the probability distribution parameters into the first image bitstream may involve processing the probability distribution parameters through a super-coding network and then writing them into the first image bitstream by entropy coding. Constructing a distribution quantization parameter based on the probability distribution parameters may involve performing a quantization process on the probability distribution parameters to obtain the distribution quantization parameter. Writing the quantized residual values ​​into the second image bitstream based on the distribution quantization parameter may involve performing entropy coding on the quantized residual values ​​based on the distribution quantization parameter and writing them into the second image bitstream.

[0166] Here, in order to ensure the performance of the quantizer, the quantization parameters used in quantizing the probability distribution parameters may be constructed based on feature channel distinguishing information and / or spatial point distinguishing information. In this case, the step of constructing a distribution quantization parameter based on the probability distribution parameters in this embodiment can be: constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0167] By constructing the probability quantization parameters based on feature channel distinction information and / or spatial point distinction information, different quantization parameters can be set according to the differences in the feature channels or spatial points corresponding to the probability distribution parameters, so that the quantization can fully adapt to the differences in the feature channels and the complexity of the image texture, thereby ensuring the performance of the quantizer as much as possible.

[0168] In practical use, to ensure that a decoding device can successfully decode the generated image bitstream, the feature channel distinction information and / or spatial point distinction information may also be written into the image bitstream.

[0169] During encoding, the parameters used for quantization are adjusted based on the feature channel distinction information and / or the spatial point distinction information, so that the decoding device needs to acquire the feature channel distinction information and / or the spatial point distinction information in order to decode normally. In this case, by writing the feature channel distinction information and / or the spatial point distinction information into the first image bitstream or the second image bitstream, it can be ensured that the decoding device can acquire the feature channel distinction information and / or the spatial point distinction information, and that the decoding device can decode normally.

[0170] In practical use, in order to avoid bitstream data confusion or bitstream complexity, a third image bitstream may be further configured, and differentiation information may be transmitted through the third image bitstream. In this case, after writing the quantized residual value into the second image bitstream according to the distribution quantization parameter of this embodiment, further: writing the feature channel distinction information and / or the spatial point distinction information into the third image bitstream.

[0171] For ease of understanding, the description will be made with reference to Fig. 14, but this does not limit the technical solution. Fig. 14 is a schematic diagram of the image encoding process of this embodiment, and as shown in Fig. 14, the encoding device writes the quantized residual value into the second image bitstream, writes the encoding distribution parameter z_hat calculated using the probability distribution parameter σ into the first image bitstream, and may write related information for distinguishing feature channels or spatial points (feature channel distinguishing information and / or the spatial point distinguishing information) into the first image bitstream or the second image bitstream, or into the third image bitstream. If necessary, the related information for distinguishing feature channels or spatial points may be directly written into the encoding device as a local parameter, which does not need to be transmitted via a bitstream and only needs to ensure consistency between the encoding device side and the decoding device side.

[0172] In this embodiment, an analysis and transformation process is performed on an image block to be processed to obtain image features, residual calculation is performed on the image features to obtain image residual values ​​and probability distribution parameters, quantization precision parameters corresponding to each image residual value are established based on feature channel distinction information and / or spatial point distinction information, quantization is performed on the image residual values ​​based on the quantization precision parameters to obtain quantized residual values, and the probability distribution parameters and the quantized residual values ​​are written into an image bitstream. During the encoding process, different quantization parameters can be set based on the image texture differences of feature channels and spatial points, so that more important features bear smaller quantization losses and improve encoding and decoding performance.

[0173] FIG. 15 is a schematic flow diagram of a second embodiment of the image coding method of the present invention.

[0174] Based on the first embodiment of the image coding method, step S300 of the image coding method of this embodiment includes steps S3001 to S3002.

[0175] In step S3001, the quantization step corresponding to each type of feature channel is extracted from the feature channel distinction information.

[0176] When distinguishing the feature channels, the feature channels can be divided into multiple types, and different quantization steps can be assigned to different types of feature channels. In this case, the feature channel distinction information includes a classification rule for the feature channels and the quantization steps corresponding to each type of feature channel. In this case, the quantization steps corresponding to each type of feature channel can be directly extracted from the feature channel distinction information.

[0177] Here, the method for setting the classification rules for the feature channels is similar to the setting method described above, and therefore, a description thereof will be omitted here.

[0178] In step S3002, a quantization precision parameter corresponding to each image residual value is constructed based on the quantization step.

[0179] After determining the quantization step corresponding to each type of feature channel, parameters such as the quantization step when quantizing the image residual value may be determined based on the feature channel type corresponding to each image residual value, thereby determining the quantization accuracy parameter corresponding to each image residual value.

[0180] In specific implementation, different quantization steps may be set for the same feature channel, for example, by setting a threshold value to divide into multiple value intervals, and further setting the threshold value according to the value interval to which the probability distribution parameter or the value calculated based on the probability distribution parameter belongs. In this embodiment, step S3002 is as follows: extracting a parameter section threshold from the characteristic channel distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the probability distribution parameter, the parameter interval threshold, and the quantization step.

[0181] When constructing a quantization accuracy parameter corresponding to each feature channel based on the probability distribution parameter, the parameter interval threshold, and the quantization step, the following may be performed: first, the type of feature channel to which the quantized residual value belongs is obtained; then, the quantization step and the parameter interval threshold corresponding to the feature channel of that type are obtained; and then, the type of feature channel is divided into a plurality of value intervals based on the parameter interval threshold; next, the value interval to which the probability distribution parameter corresponding to the quantized residual value belongs (which may be the value interval to which the mean value, variance, or other value of the probability distribution parameter belongs) is calculated; and then, based on the value interval, a corresponding quantization step is selected from the plurality of quantization steps corresponding to the feature channel of that type, thereby obtaining a quantization accuracy parameter corresponding to each image residual value.

[0182] In specific implementation, the same type of feature channel may be further classified, for example, one type of feature channel may be divided into multiple segments, and each segment may correspond to a different quantization step. In this case, the step 3002 in this embodiment may be: analyzing the feature channel distinction information to obtain segment step sets corresponding to each type of feature channel; extracting a quantization step corresponding to each channel segment from the segment step set; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0183] After a type of feature channel is further divided, each type of feature channel can correspond to multiple different quantization steps, which can be stored in the form of a set, and the feature channel distinguishing information can be analyzed to obtain a segment step set corresponding to each type of feature channel, and the quantization step corresponding to each channel segment can be extracted from the segment step set. Here, the segmentation methods of different types of feature channels can be different or the same, and this embodiment is not limited thereto.

[0184] In this case, constructing a quantization accuracy parameter corresponding to each image residual value based on a quantization step may involve obtaining a feature channel type corresponding to the image residual value, determining a channel segment to which the image residual value belongs in the feature channel of that type, and determining parameters such as a quantization step to be used when quantizing the image residual value based on the feature channel type and the channel segment, thereby obtaining a quantization accuracy parameter corresponding to each image residual value.

[0185] In specific implementation, the quantization step may be set based on the image texture. In this case, step S3002 in this embodiment can be: extracting quantization steps corresponding to each type of spatial point from the spatial point distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0186] When setting the quantization step, each spatial point in an image may be divided into multiple types based on the image texture, and different quantization steps may be set for different types of spatial points. For example, if the spatial points in an image are divided into X types based on the image texture, the quantization step for the jth type of spatial point is scale_j, where j has a value range of [1,X].

[0187] Here, when classifying spatial points, they can be distinguished based on the complexity of the image texture corresponding to the spatial point. For example, three complexity intervals, [0, F1], (F1, F2], and (F2, ∞), are set, and the spatial points are divided into three types based on the complexity interval to which the complexity of the image texture corresponding to the spatial point belongs.

[0188] Constructing a quantization precision parameter corresponding to each image residual value based on a quantization step corresponding to each type of spatial point may involve obtaining a parameter such as a quantization step used when quantizing an image residual value based on a spatial point type corresponding to the image residual value, thereby obtaining a quantization precision parameter corresponding to each image residual value.

[0189] Each spatial point includes C (C is the number of feature channels). After different types of spatial points are divided, the spatial points may be further divided into multiple points so that different feature channels in one type of spatial point correspond to different quantization steps. In this case, the spatial point distinguishing information and the feature channel distinguishing information can be used in combination. In this case, step S3002 in this embodiment can be reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinguishing information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

[0190] When the points included in each spatial point at each type of spatial point are further divided based on the differences in the feature channels, the points of the different feature channels corresponding to one type of spatial point may correspond to different quantization steps. In this case, the distinguishable types of spatial points may be first read from the spatial point distinction information, and then the quantization steps corresponding to each feature channel at each type of spatial point may be read.

[0191] For example, if spatial points are divided into two types, a and b, and the feature channels included in a and b are all 1 to X, the quantization steps corresponding to each feature channel at spatial points of type a may be expressed as scale_lista=[scale1, scale2, ..., scaleX], and the quantization steps corresponding to each feature channel at spatial points of type b may be expressed as scale_listb=[scale1, scale2, ..., scaleX].

[0192] In a specific implementation, in order to allow the encoding device to clearly understand the specific quantization step setting method, a flag bit corresponding to the encoding device may be set in advance, and how to obtain the quantization step may be determined according to the flag bit. For example, when the flag bit is set to 0, the quantization step is determined based on the feature channel distinction information; when the flag bit is set to 1, the quantization step is determined based on the spatial point distinction information; and when the flag bit is set to 2, the quantization step is determined by combining the feature channel distinction information and the spatial point distinction information.

[0193] In this embodiment, only one method of combining spatial point distinction information and feature channel distinction information has been described, but further dividing multiple points contained in a spatial point may also utilize the embodiment of further dividing the feature channel provided in any of the above embodiments, and this embodiment is not limited to this.

[0194] FIG. 16 is a schematic flow diagram of a third embodiment of the image coding method of the present invention.

[0195] Based on the first embodiment of the image coding method, step S500 of the image coding method of this embodiment includes steps S5001 to S5003.

[0196] In step S5001, image decoding is performed based on the probability distribution parameters, the quantized residual values, and the feature channel distinguishing information to obtain a reconstructed image block and an image bit rate.

[0197] Performing image decoding based on the probability distribution parameters, the quantized residual values, and the feature channel distinction information, and obtaining a reconstructed image block and an image bit rate may also be performing image decoding based on the probability distribution parameters, the quantized residual values, and the feature channel distinction information, obtaining a reconstructed image block, and calculating the image bit rate based on the probability distribution parameters, according to the image decoding method of any of the above embodiments.

[0198] In step S5002, the feature channel distinguishing information and / or spatial point distinguishing information is adjusted based on the reconstructed image block and the image bit rate.

[0199] The encoding device may perform parameter optimization adjustment on parameters used in the encoding process, in which case adjusting the feature channel differentiation information based on the reconstructed image block and the image bit rate may be optimizing adjustment of parameters related to quantization in the feature channel differentiation information and / or spatial point differentiation information based on the reconstructed image block and the image bit rate.

[0200] In a specific implementation, an administrator of the encoding device may determine whether to perform parameter optimization adjustment by changing an optimal threshold mode flag bit in the encoding device, where a value of the optimal threshold mode flag bit of 0 indicates that parameter optimization adjustment is not performed, and a value of the optimal threshold mode flag bit of 1 indicates that parameter optimization adjustment is performed.

[0201] For ease of understanding, the description will be made with reference to Fig. 17, but this does not limit the technical solution. Fig. 17 is a flow diagram of the operation of the optimal threshold mode flag bit. As shown in Fig. 17, a user of an encoding device can set an optimal threshold mode flag bit Flag_useRDAQ and transmit the set flag bit and related thresholds or parameters for distinguishing feature channels to the encoding device. The encoding device detects whether the value of the optimal threshold mode flag bit is 1. If the value of the optimal threshold mode flag bit is 1, the encoding device turns on parameter optimization adjustment and sets the input related thresholds or parameters as initial values. Thereafter, the encoding device performs multiple iterations, performs encoding based on the optimal related thresholds or parameters for distinguishing feature channels obtained after the iterations, and transmits the generated image bitstream to the decoding device. If the value of the optimal threshold mode flag bit is not 1, the encoding device directly performs encoding based on the transmitted related thresholds or parameters for distinguishing feature channels, and transmits the image bitstream generated by encoding to the decoding device.

[0202] In step S5003, if the current number of adjustments is equal to or greater than the predetermined number of adjustments, the probability distribution parameters and the quantized residual values ​​are written into the image bitstream.

[0203] The predetermined number of adjustments may be a preset number of times to adjust the feature channel distinction information, and the current number of adjustments may be the number of times the feature channel distinction information has already been adjusted.

[0204] If the current number of adjustments is equal to or greater than the predetermined number of adjustments, it indicates that the parameter optimization adjustment is completed, and at this time, the probability distribution parameters and the quantized residual value can be written into the image bitstream, and the image bitstream can be transmitted to the decoding device side. Here, the adjusted feature channel distinction information and / or spatial point distinction information can also be written into the image bitstream.

[0205] If the current number of adjustments is smaller than the predetermined number of adjustments, it indicates that the parameter optimization adjustment has not yet been completed, and therefore, the method can return to the step of constructing quantization precision parameters corresponding to each feature channel based on the feature channel distinction information.

[0206] For ease of understanding, the description will be made with reference to Fig. 18, but this does not limit the technical solution. Fig. 18 is a flowchart of parameter optimization in this embodiment, in which the indicator representing distortion D may be one or a combination of multiple indicators such as msssim, vif, fsim, nlpd, iw-ssim, vmaf, and psnr_HVS, X is a related threshold or parameter for distinguishing input feature channels, res_q is a quantized residual value, sigmma is a probability distribution parameter σ, and step_X may be preset by an administrator of the encoding device according to actual needs. Satisfying the iteration limit may also be comparing the current adjustment count with a predetermined adjustment count, and determining that the iteration limit is met if the current adjustment count is equal to or greater than the predetermined adjustment count, or determining that the iteration limit is not met if the current adjustment count is less than the predetermined adjustment count.

[0207] In this embodiment, image decoding is performed based on the probability distribution parameters, the quantized residual values, and the feature channel distinction information to obtain a reconstructed image block and an image bit rate, and the feature channel distinction information and / or spatial point distinction information are adjusted based on the reconstructed image block and the image bit rate. If the current adjustment count is equal to or greater than a predetermined adjustment count, the probability distribution parameters and the quantized residual values ​​are written into an image bitstream. Image decoding is performed and the feature channel distinction information and / or spatial point distinction information are adjusted based on the reconstructed image block and the image bit rate, thereby ensuring that the parameters ultimately used in encoding are excellent, and further improving image encoding performance.

[0208] Furthermore, an embodiment of the present invention further provides a storage medium on which an image encoding program or an image decoding program is stored, and when the image encoding program is executed by a processor, the steps of the above-mentioned image encoding method are performed, and when the image decoding program is executed by a processor, the steps of the above-mentioned image decoding method are performed.

[0209] FIG. 19 is a block diagram showing the configuration of a first embodiment of an image decoding device according to the present invention.

[0210] As shown in FIG. 19, an image decoding apparatus according to an embodiment of the present invention includes: an entropy decoding module 10 for obtaining feature channel distinction information and / or spatial point distinction information, decoding the image bitstream, and obtaining quantized residual values; a parameter construction module 20 for constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the feature channel distinction information and / or the spatial point distinction information; an inverse quantization module 30 for inverse quantizing the quantized residual values ​​based on the inverse quantization precision parameter to obtain reconstructed residual values; a synthesis transform module 40 for performing a synthesis transform on the reconstructed residual values ​​to obtain a reconstructed image block.

[0211] For ease of understanding, the description will be made with reference to the above FIG. 3, but this does not limit the present technical solution. In this embodiment, the entropy decoding module may perform the entropy decoding process as shown in FIG. 3, the parameter construction module and the inverse quantization module perform the inverse quantization process as shown in FIG. 3, and the synthesis transformation module performs the synthesis transformation process in FIG. 3 through a synthesis transformation network.

[0212] In this embodiment, feature channel distinction information and / or spatial point distinction information are obtained, an image bitstream is decoded to obtain quantized residual values, an inverse quantization precision parameter corresponding to each quantized residual value is constructed based on the feature channel distinction information and / or the spatial point distinction information, the quantized residual values ​​are inversely quantized based on the inverse quantization precision parameter, reconstructed residual values ​​are obtained, and synthesis transformation is performed on the reconstructed residual values ​​to obtain a reconstructed image block. Since the corresponding inverse quantization precision parameter is constructed based on the feature channel distinction information and / or the spatial point distinction information and then inverse quantization is performed, different quantization parameters can be set during the encoding process based on the image texture differences of the feature channels and spatial points, allowing more important features to bear smaller quantization losses and improving encoding and decoding performance.

[0213] In one possible implementation of this embodiment, the image bitstream includes a first image bitstream and a second image bitstream, the first image bitstream is used to transmit decoding auxiliary information, and the second image bitstream is used to transmit residual data; The entropy decoding module 10 is further used for extracting coding distribution parameters from the first image bitstream, decoding the coding distribution parameters, obtaining probability distribution parameters, quantizing the probability distribution parameters, obtaining distribution quantization parameters, and decoding the second image bitstream based on the distribution quantization parameters, and obtaining quantized residual values.

[0214] In one possible embodiment of this embodiment, the entropy decoding module 10 is further used to construct a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information, and quantize the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0215] In one possible embodiment of this embodiment, the entropy decoding module 10 is further used to decode the first image bitstream or the second image bitstream to obtain feature channel distinction information and / or spatial point distinction information.

[0216] In one possible implementation of this embodiment, the image bitstream further includes a third image bitstream, and the third image bitstream is used to transmit differentiation information; The entropy decoding module 10 is further used for decoding the third image bitstream to obtain feature channel distinguishing information and / or spatial point distinguishing information.

[0217] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to extract quantization steps corresponding to each type of feature channel from the feature channel distinction information, and construct inverse quantization precision parameters respectively corresponding to each quantized residual value according to the quantization steps.

[0218] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to extract quantization steps corresponding to each type of spatial point from the spatial point distinction information, and construct inverse quantization precision parameters corresponding to each quantized residual value based on the quantization steps.

[0219] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to read each type of distinguished spatial point from the spatial point distinction information, read quantization steps respectively corresponding to each type of feature channel at each type of spatial point from the feature channel distinction information, and construct inverse quantization precision parameters respectively corresponding to each quantized residual value based on the quantization steps.

[0220] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to decode the image bitstream, obtain a probability distribution parameter, extract a parameter interval threshold from the feature channel distinction information, and construct an inverse quantization precision parameter corresponding to each quantized residual value based on the probability distribution parameter, the quantization step and the parameter interval threshold.

[0221] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to extract image block division information and parameter setting rules corresponding to each feature channel from the feature channel distinction information, decode the image bitstream to obtain probability distribution parameters, calculate distribution parameter average values ​​corresponding to each image block division information based on the probability distribution parameters, match the distribution parameter average values ​​with the parameter setting rules, obtain quantization steps corresponding to each image block division information, and construct inverse quantization precision parameters corresponding to each quantized residual value based on the quantization steps.

[0222] In one possible embodiment of this embodiment, the parameter construction module 20 is further used to analyze the feature channel distinction information, obtain segment step sets corresponding to each type of feature channel, extract quantization steps corresponding to each channel segment from the segment step sets, and construct inverse quantization precision parameters corresponding to each quantized residual value based on the quantization steps.

[0223] FIG. 20 is a block diagram showing the configuration of a first embodiment of an image coding apparatus according to the present invention.

[0224] As shown in FIG. 20, an image coding apparatus according to an embodiment of the present invention includes: an analysis and transformation module 100 for performing analysis and transformation processing on the image block to be processed to obtain image features; a residual calculation module 200 for performing residual calculation on the image features to obtain image residual values ​​and probability distribution parameters; a parameter construction module 300 for constructing a quantization precision parameter corresponding to each feature channel based on the feature channel distinction information and / or the spatial point distinction information; a quantization module 400 for performing a quantization process on the image residual values ​​according to the quantization precision parameter to obtain quantized residual values; an entropy coding module 500 for writing the probability distribution parameters and the quantized residual values ​​into an image bitstream.

[0225] For ease of understanding, the description will be made with reference to the above FIG. 3, but this does not limit the present technical solution. In this embodiment, the analysis and transformation module may perform the analysis and transformation process as shown in FIG. 3 through an analysis and transformation network, the residual calculation module, the parameter construction module and the quantization module perform the quantization process as shown in FIG. 3, and the entropy coding module performs the entropy coding process as shown in FIG. 3.

[0226] In this embodiment, an analysis and transformation process is performed on an image block to be processed to obtain image features, residual calculation is performed on the image features to obtain image residual values ​​and probability distribution parameters, quantization precision parameters corresponding to each image residual value are established based on feature channel distinction information and / or spatial point distinction information, quantization is performed on the image residual values ​​based on the quantization precision parameters to obtain quantized residual values, and the probability distribution parameters and the quantized residual values ​​are written into an image bitstream. During the encoding process, different quantization parameters can be set based on the image texture differences of feature channels and spatial points, so that more important features bear smaller quantization losses and improve encoding and decoding performance.

[0227] In one possible implementation of this embodiment, the image bitstream includes a first image bitstream and a second image bitstream, the first image bitstream is used to transmit decoding auxiliary information, and the second image bitstream is used to transmit residual data; The entropy coding module 500 is further used for writing the probability distribution parameters into the first image bitstream, constructing a distribution quantization parameter based on the probability distribution parameters, and writing the quantized residual values ​​into the second image bitstream based on the distribution quantization parameter.

[0228] In one possible embodiment of this embodiment, the entropy encoding module 500 is further used to construct a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information, and quantize the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.

[0229] In one possible embodiment of this embodiment, the entropy encoding module 500 is further used to write the feature channel distinction information and / or the spatial point distinction information into the first image bitstream or the second image bitstream.

[0230] In one possible implementation of this embodiment, the image bitstream further includes a third image bitstream, and the third image bitstream is used to transmit differentiation information; The entropy encoding module 500 is further used for writing the feature channel distinction information and / or the spatial point distinction information into the third image bitstream.

[0231] In one possible embodiment of this embodiment, the parameter construction module 300 is further used to extract quantization steps corresponding to each type of feature channel from the feature channel distinction information, and construct quantization precision parameters respectively corresponding to each image residual value according to the quantization steps.

[0232] In one possible embodiment of this embodiment, the parameter construction module 300 is further used to extract a parameter interval threshold from the feature channel distinction information, and construct a quantization precision parameter corresponding to each image residual value according to the probability distribution parameter, the parameter interval threshold and the quantization step.

[0233] In one possible embodiment of this embodiment, the parameter construction module 300 is further used to analyze the feature channel distinction information, obtain segment step sets corresponding to each type of feature channel, extract quantization steps corresponding to each channel segment from the segment step sets, and construct quantization precision parameters respectively corresponding to each image residual value based on the quantization steps.

[0234] In one possible embodiment of this embodiment, the parameter construction module 300 is further used to extract quantization steps corresponding to each type of spatial point from the spatial point distinction information, and construct quantization precision parameters respectively corresponding to each image residual value based on the quantization steps.

[0235] In one possible embodiment of this embodiment, the parameter construction module 300 is further used to read each type of distinguished spatial point from the spatial point distinction information, read quantization steps respectively corresponding to each type of feature channel at each type of spatial point from the feature channel distinction information, and construct quantization precision parameters respectively corresponding to each image residual value based on the quantization steps.

[0236] In one possible embodiment of this embodiment, the entropy encoding module 500 is further used to perform image decoding based on the probability distribution parameters, the quantized residual values ​​and the feature channel distinction information to obtain a reconstructed image block and an image bit rate, adjust the feature channel distinction information and / or the spatial point distinction information based on the reconstructed image block and the image bit rate, and if the current adjustment count is equal to or greater than a predetermined adjustment count, write the probability distribution parameters and the quantized residual values ​​into an image bit stream.

[0237] In one possible embodiment of this embodiment, the entropy encoding module 500 is further used to return to performing the step of constructing quantization precision parameters corresponding to each feature channel based on the feature channel distinction information if the current adjustment number is less than a predetermined adjustment number.

[0238] It should be noted that the above is merely an example and does not limit the technical solution of the present invention. In specific applications, those skilled in the art can set it as needed, and the present invention is not limited thereto.

[0239] It should be noted that the above-described flow is merely an example and does not limit the scope of protection of the present invention. In actual applications, those skilled in the art can select part or all of it according to actual needs to achieve the objective of the technical solution of this embodiment, and no limitation is made here.

[0240] Furthermore, for technical details not described in detail in this embodiment, reference may be made to the image decoding or image decoding method provided in any embodiment of the present invention, and the description will be omitted here.

[0241] It should be noted that, as used herein, the terms "comprise," "contain," or any other variation thereof, are intended to include a non-exclusive inclusion, whereby a process, method, article, or system that includes a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes the inherent elements of such process, method, article, or system. Absent more limitations, an element qualified by "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or system that includes that element.

[0242] The numbers of the above embodiments of the present invention are for illustrative purposes only and do not represent the superiority of the embodiments.

[0243] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be realized by software and a necessary general-purpose hardware platform, or by hardware, and in many cases, the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or a part that contributes to the prior art, may be embodied in the form of a software product, and the computer software product is stored in a storage medium (e.g., a read-only memory (ROM) / RAM, a magnetic disk, an optical disk), and includes several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0244] The above is merely a preferred embodiment of the present invention, and does not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by utilizing the contents of the specification and drawings of the present invention, or any direct or indirect application to other related technical fields, is also included in the protection scope of the present invention.

Claims

1. obtaining feature channel distinction information and / or spatial point distinction information, decoding the image bitstream, and obtaining quantized residual values; constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information; for any quantized residual value, dequantizing the quantized residual value based on the dequantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; performing a synthesis transform on the reconstructed residual values ​​to obtain a reconstructed image block.

1. An image decoding method comprising:

2. The step of decoding the image bitstream to obtain quantized residual values ​​includes: extracting coding distribution parameters from the first image bitstream; decoding the encoded distribution parameters to obtain probability distribution parameters; quantizing the probability distribution parameters to obtain a distribution quantization parameter; decoding a second image bitstream based on the distribution quantization parameter to obtain the quantized residual value; 2. The image decoding method according to claim 1.

3. quantizing the probability distribution parameters to obtain a distribution quantization parameter, constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter; 3. The image decoding method according to claim 2.

4. The step of constructing a probability quantization parameter based on the feature channel distinguishing information and / or spatial point distinguishing information includes: constructing the probability quantization parameters based on feature channels or spatial points corresponding to the probability distribution parameters; Different feature channels or spatial points correspond to different probability quantization parameters.

4. The image decoding method according to claim 3.

5. The step of obtaining the feature channel distinguishing information and / or the spatial point distinguishing information includes: decoding the image bitstream to obtain feature channel distinction information and / or spatial point distinction information; 2. The image decoding method according to claim 1.

6. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information includes: extracting quantization steps corresponding to each type of feature channel from the feature channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

2. The image decoding method according to claim 1.

7. Different types of feature channels correspond to different quantization steps.

7. The image decoding method according to claim 6.

8. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information includes: extracting quantization steps corresponding to each type of spatial point from the spatial point distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

2. The image decoding method according to claim 1.

9. Different types of spatial points correspond to different quantization steps.

9. The image decoding method according to claim 8.

10. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information includes: reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step.

2. The image decoding method according to claim 1.

11. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step includes: decoding the image bitstream to obtain probability distribution parameters; extracting a parameter section threshold from the characteristic channel distinction information; and constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the probability distribution parameter, the quantization step, and the parameter interval threshold.

7. The image decoding method according to claim 6.

12. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information includes: analyzing the characteristic channel distinction information; For any quantized residual value, obtaining a segment step set corresponding to each type of feature channel, where the segment step set corresponding to one type of feature channel includes multiple different quantization steps; extracting a quantization step corresponding to a channel segment to which the quantized residual value belongs from a segment step set corresponding to one type of feature channel; and constructing, based on the quantization step, inverse quantization precision parameters corresponding to the quantized residual values ​​respectively.

2. The image decoding method according to claim 1.

13. The method further includes the steps of, for any quantized residual value, obtaining a feature channel type corresponding to the quantized residual value, determining a channel segment to which the quantized residual value belongs in the feature channel of the type, determining a quantization step corresponding to the quantized residual value based on the feature channel type and the channel segment, and determining an inverse quantization precision parameter corresponding to the quantized residual value based on the quantization step, wherein a segment step set corresponding to one type of feature channel includes multiple different quantization steps, one type of feature channel is divided into multiple channel segments, and one channel segment corresponds to one quantization step.

2. The image decoding method according to claim 1.

14. The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point distinction information includes: determining a feature channel type and / or a spatial point type corresponding to each quantized residual value based on the feature channel distinction information and / or the spatial point information; determining an inverse quantization precision parameter corresponding to each quantized residual value based on the feature channel type and / or the spatial point type; Different types of feature channels and / or spatial point types employ different precisions of quantization; 2. The image decoding method according to claim 1.

15. performing an analysis and transformation process on the image block to be processed to obtain image features; performing a residual calculation on the image features to obtain an image residual value and a probability distribution parameter; constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information; performing a quantization process on the image residual value based on the quantization precision parameter to obtain a quantized residual value; and writing the probability distribution parameters and the quantized residual values ​​into an image bitstream.

1. An image coding method comprising:

16. writing the probability distribution parameters and the quantized residual values ​​into an image bitstream, writing said probability distribution parameters into a first image bitstream; constructing a distribution quantization parameter based on the probability distribution parameters; and writing the quantized residual values ​​to a second image bitstream based on the distributed quantization parameter.

16. The image coding method according to claim 15.

17. constructing a distribution quantization parameter based on the probability distribution parameter, constructing a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; quantizing the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter; 16. The image coding method according to claim 15.

18. The step of constructing a probability quantization parameter based on the feature channel distinguishing information and / or spatial point distinguishing information includes: constructing the probability quantization parameters based on feature channels or spatial points corresponding to the probability distribution parameters; Different feature channels or spatial points correspond to different probability quantization parameters.

18. The image coding method according to claim 17.

19. further comprising writing the feature channel distinction information and / or the spatial point distinction information into the image bitstream.

16. The image coding method according to claim 15.

20. The step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information includes: extracting quantization steps corresponding to each type of feature channel from the feature channel distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

16. The image coding method according to claim 15.

21. Separating the feature channels into a plurality of types; for each type of feature channel, determining a quantization step corresponding to that type of feature channel; Different types of feature channels correspond to different quantization steps.

21. The image coding method according to claim 20.

22. The step of constructing a quantization precision parameter corresponding to each image residual value based on the quantization step includes: extracting a parameter section threshold from the characteristic channel distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the probability distribution parameter, the parameter interval threshold, and the quantization step.

21. The image coding method according to claim 20.

23. The step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information includes: analyzing the characteristic channel distinction information; For any image residual value, obtaining a segment step set corresponding to each type of feature channel, wherein the segment step set corresponding to one type of feature channel includes a plurality of different quantization steps; Extracting a quantization step corresponding to a channel segment to which the image residual value belongs from a segment step set corresponding to one type of feature channel; and constructing a quantization precision parameter corresponding to each of the image residual values ​​based on the quantization step.

16. The image coding method according to claim 15.

24. The step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information includes: extracting quantization steps corresponding to each type of spatial point from the spatial point distinction information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

16. The image coding method according to claim 15.

25. classifying spatial points in an image into a plurality of classes based on image texture; for each type of spatial point, determining a quantization step for that type of spatial point; Different types of spatial points correspond to different quantization steps.

25. The image coding method according to claim 24.

26. The step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information includes: reading each type of distinguished spatial point from the spatial point distinguishing information; reading quantization steps corresponding to each type of feature channel at each type of spatial point from the feature channel distinguishing information; and constructing a quantization precision parameter corresponding to each image residual value based on the quantization step.

16. The image coding method according to claim 15.

27. The method further includes the steps of, for any image residual value, obtaining a feature channel type corresponding to the image residual value, determining a channel segment to which the image residual value belongs in the feature channel of the type, determining a quantization step to be used when quantizing the image residual value based on the feature channel type and the channel segment, and determining a quantization precision parameter corresponding to the image residual value based on the quantization step, wherein a segment step set corresponding to one type of feature channel includes multiple different quantization steps, one type of feature channel is divided into multiple channel segments, and one channel segment corresponds to one quantization step.

16. The image coding method according to claim 15.

28. The step of constructing a quantization precision parameter corresponding to each image residual value based on the feature channel distinction information and / or the spatial point distinction information includes: determining a feature channel type and / or a spatial point type corresponding to each image residual value based on the feature channel distinction information and / or the spatial point information; determining a quantization precision parameter corresponding to each image residual value based on the feature channel type and / or the spatial point type; employing different types of feature channels and / or different precisions of quantization; 16. The image coding method according to claim 15.

29. an entropy decoding module for obtaining feature channel distinction information and / or spatial point distinction information, decoding the image bitstream, and obtaining quantized residual values; a parameter construction module for constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the feature channel distinction information and / or the spatial point distinction information; an inverse quantization module for, for any quantized residual value, inverse quantizing the quantized residual value based on the inverse quantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; a synthesis transform module for performing a synthesis transform on the reconstructed residual values ​​to obtain a reconstructed image block.

1. An image decoding device comprising:

30. an analysis and transformation module for performing analysis and transformation processing on the image block to be processed and obtaining image features; a residual calculation module for performing residual calculation on the image features to obtain image residual values ​​and probability distribution parameters; a parameter construction module for constructing a quantization precision parameter corresponding to each feature channel based on the feature channel distinguishing information and / or the spatial point distinguishing information; a quantization module for performing a quantization process on the image residual values ​​based on the quantization precision parameter to obtain quantized residual values; an entropy coding module for writing the probability distribution parameters and the quantized residual values ​​into an image bitstream. An image encoding device comprising:

31. A decoding device including a processor, a memory, and a decoding program stored in the memory and executable by the processor, wherein when the decoding program is executed by the processor, the image decoding method according to any one of claims 1 to 14 is performed.

10. A decoding device comprising:

32. An encoding device including a processor, a memory, and an encoding program stored in the memory and executable by the processor, wherein when the encoding program is executed by the processor, the image encoding method according to any one of claims 15 to 28 is implemented.

10. A coding device comprising:

33. A computer-readable storage medium storing an image decoding program and / or an image encoding program, wherein, when the image decoding program is executed, the image decoding method according to any one of claims 1 to 14 is implemented, or when the image encoding program is executed, the image encoding method according to any one of claims 15 to 28 is implemented. A computer-readable storage medium comprising:

34. A computer program product including a computer program, the computer program being executed by a processor to implement the image decoding method according to any one of claims 1 to 14, or the image encoding program being executed to implement the image encoding method according to any one of claims 15 to 28.

1. A computer program product comprising:

Images