Image decoding and encoding method and apparatus, device, and storage medium
Patent Information
- Application Number
- MYPI2025005301
- Authority / Receiving Office
- MY · MY
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-03-01
- Filing Date
- 2024-03-01
- Publication Date
- 2024-09-01
AI Technical Summary
The performance of the quantizer in the image encoding and decoding process in the prior art is poor and fails to effectively consider the differences in feature channels and image textures, which limits the quantization performance.
By obtaining feature channel distinction information and/or spatial point distinction information, the image code stream is decoded, an inverse quantization precision parameter corresponding to the quantized residual value is constructed, and inverse quantization and synthetic transformation are performed to improve the quality of image reconstruction.
By setting quantization parameters based on the differences between feature channels and spatial points, the quantization loss is reduced and the encoding and decoding performance is improved.
Smart Images

Figure PI2025005301_DRAWING_001 
Figure PI2025005301_DRAWING_002 
Figure PI2025005301_DRAWING_003
Abstract
Description
Image decoding and encoding method, device, equipment and storage medium Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image decoding and encoding method, apparatus, device, and storage medium. Background Art
[0002] End-to-end image coding and decoding technology generally includes modules such as analysis transform network, synthesis transform network, context-based prediction, quantization, entropy coding, super coding network, and super-scale decoding network. Among them, quantization is a "many-to-one" mapping process, which will cause signal loss. Quantization acts on the residual and can change the value range of the signal, so that the encoder can give a good approximation of the original signal with a small number of symbols, thereby improving the compression rate. However, usually, the quantization module does not consider the differences in feature channels and image textures when performing quantization, which limits the quantization performance of the quantizer.
[0003] The above content is only used to assist in understanding the technical solution of the present disclosure and does not constitute an admission that the above content is prior art.
[0004] Summary of the Invention
[0005] The main purpose of the present disclosure is to provide an image decoding and encoding method, apparatus, device and storage medium, aiming to solve the technical problem of poor performance of quantizers in the image encoding and decoding process in the prior art.
[0006] To achieve the above object, the present disclosure provides an image decoding method, the method comprising the following steps:
[0007] Acquire characteristic channel distinguishing information and / or spatial point distinguishing information, and decode the image code stream to obtain a quantized residual value;
[0008] Constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information;
[0009] For any quantized residual value, dequantize the quantized residual value based on the dequantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value;
[0010] Performing a synthetic transformation on the reconstructed residual value to obtain a reconstructed image block.
[0011] In a possible implementation manner of the present disclosure, the step of decoding the image code stream to obtain a quantized residual value includes:
[0012] extracting coding profile parameters from the first image code stream;
[0013] Decoding the coded distribution parameters to obtain probability distribution parameters;
[0014] quantizing the probability distribution parameters to obtain distribution quantization parameters;
[0015] The second image code stream is decoded based on the distributed quantization parameter to obtain the quantized residual value.
[0016] In a possible implementation manner of the present disclosure, the step of quantizing the probability distribution parameters to obtain distribution quantization parameters includes:
[0017] Constructing a probability quantization parameter according to the characteristic channel distinction information and / or the spatial point distinction information;
[0018] The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
[0019] In a possible implementation of the present disclosure, the step of obtaining characteristic channel distinction information and / or spatial point distinction information includes:
[0020] The image code stream is decoded to obtain characteristic channel distinction information and / or spatial point distinction information.
[0021] In a possible implementation manner of the present disclosure, the image code stream further includes a third image code stream, and the third image code stream is used to transmit distinguishing information;
[0022] The step of obtaining characteristic channel distinguishing information and / or spatial point distinguishing information includes:
[0023] The third image code stream is decoded to obtain characteristic channel distinction information and / or spatial point distinction information.
[0024] In a possible implementation of the present disclosure, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information includes:
[0025] Extracting the quantization step size corresponding to each type of feature channel from the feature channel distinction information;
[0026] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0027] In a possible implementation of the present disclosure, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information includes:
[0028] Extracting the quantization step corresponding to each type of spatial point from the spatial point distinction information;
[0029] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0030] In a possible implementation of the present disclosure, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information includes:
[0031] Reading the various types of divided spatial points from the spatial point distinction information;
[0032] Reading the quantization step lengths corresponding to the various feature channels in various spatial points from the feature channel differentiation information;
[0033] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0034] In a possible implementation of the present disclosure, the step of constructing, according to the quantization step size, inverse quantization precision parameters corresponding to the respective quantized residual values includes:
[0035] Decode the image code stream to obtain probability distribution parameters;
[0036] Extracting parameter interval thresholds from the feature channel differentiation information;
[0037] The inverse quantization precision parameters corresponding to the respective quantization residual values are constructed according to the probability distribution parameter, the quantization step size and the parameter interval threshold.
[0038] In a possible implementation of the present disclosure, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information includes:
[0039] Extracting image segmentation information and parameter setting rules corresponding to each feature channel from the feature channel distinction information;
[0040] Decode the image code stream to obtain probability distribution parameters;
[0041] Calculate the distribution parameter mean corresponding to each image block information according to the probability distribution parameter;
[0042] Matching the distribution parameter mean with the parameter setting rule to obtain the quantization step size corresponding to each image block information;
[0043] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0044] In a possible implementation of the present disclosure, the step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information includes:
[0045] Parsing the feature channel distinction information, and obtaining, for any quantized residual value, a segmented step size set corresponding to each type of feature channel, wherein the segmented step size set corresponding to one type of feature channel includes a plurality of different quantization step sizes;
[0046] Extracting the quantization step size corresponding to the channel segment to which the quantized residual value belongs from the segmentation step size set corresponding to one type of feature channel;
[0047] The inverse quantization precision parameters corresponding to the quantization residual values are constructed according to the quantization step size.
[0048] In addition, to achieve the above-mentioned purpose, the present disclosure also proposes an image encoding method, which includes the following steps:
[0049] Perform analysis and transformation on the image blocks to be processed to obtain image features;
[0050] Performing residual calculation on the image features to obtain image residual values and probability distribution parameters;
[0051] Constructing quantization precision parameters corresponding to the residual values of each image according to the characteristic channel distinction information and / or the spatial point distinction information;
[0052] quantizing the image residual value based on the quantization precision parameter to obtain a quantized residual value;
[0053] The probability distribution parameter and the quantized residual value are written into an image code stream.
[0054] In a possible implementation manner of the present disclosure, the step of writing the probability distribution parameter and the quantized residual value into the image code stream includes:
[0055] Writing the probability distribution parameters into the first image code stream;
[0056] Constructing distribution quantization parameters according to the probability distribution parameters;
[0057] The quantized residual value is written into a second image code stream based on the distributed quantization parameter.
[0058] In a possible implementation manner of the present disclosure, the step of constructing a distribution quantization parameter according to the probability distribution parameter includes:
[0059] Constructing a probability quantization parameter according to the characteristic channel distinction information and / or the spatial point distinction information;
[0060] The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
[0061] In a possible implementation manner of the present disclosure, the image encoding method further includes:
[0062] The feature channel distinguishing information and / or the spatial point distinguishing information are written into the image code stream.
[0063] In a possible implementation manner of the present disclosure, the image code stream further includes a third image code stream, and the third image code stream is used to transmit distinguishing information;
[0064] After the step of writing the quantized residual value into the second image code stream based on the distributed quantization parameter, the method further includes:
[0065] The characteristic channel distinguishing information and / or the spatial point distinguishing information are written into the third image code stream.
[0066] In a possible implementation of the present disclosure, the step of constructing quantization precision parameters corresponding to respective image residual values according to feature channel distinction information and / or spatial point distinction information includes:
[0067] Extract the quantization step size corresponding to each feature channel from the feature channel distinction information;
[0068] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0069] In a possible implementation of the present disclosure, the step of constructing the quantization precision parameters corresponding to the respective image residual values according to the quantization step size includes:
[0070] Extracting parameter interval thresholds from the feature channel differentiation information;
[0071] The quantization precision parameters corresponding to the respective image residual values are constructed according to the probability distribution parameters, the parameter interval threshold and the quantization step size.
[0072] In a possible implementation of the present disclosure, the step of constructing quantization precision parameters corresponding to respective image residual values according to feature channel distinction information and / or spatial point distinction information includes:
[0073] Analyze the characteristic channel distinction information,
[0074] For any image residual value,
[0075] Obtaining a segmentation step size set corresponding to each type of feature channel, wherein the segmentation step size set corresponding to one type of feature channel includes multiple different quantization step sizes;
[0076] Extract the quantization step size corresponding to the channel segment to which the image residual value belongs from the segment step size set corresponding to one type of feature channel;
[0077] Quantization precision parameters corresponding to the image residual values are constructed according to the quantization step size.
[0078] In a possible implementation of the present disclosure, the step of constructing quantization precision parameters corresponding to respective image residual values according to feature channel distinction information and / or spatial point distinction information includes:
[0079] Extract the quantization step size corresponding to each type of spatial point from the spatial point distinction information;
[0080] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0081] In a possible implementation of the present disclosure, the step of constructing quantization precision parameters corresponding to respective image residual values according to feature channel distinction information and / or spatial point distinction information includes:
[0082] Read the various types of divided spatial points from the spatial point differentiation information;
[0083] Distinguish information from the feature channels and read the quantization step sizes corresponding to each feature channel in each spatial point;
[0084] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0085] In a possible implementation manner of the present disclosure, the step of writing the probability distribution parameter and the quantized residual value into the image code stream includes:
[0086] Perform image decoding according to the probability distribution parameter, the quantized residual value, and the characteristic channel distinction information to obtain a reconstructed image block and an image bit rate;
[0087] Adjusting the characteristic channel distinguishing information and / or the spatial point distinguishing information according to the reconstructed image block and the image bit rate;
[0088] If the current adjustment round is greater than or equal to the preset adjustment round, the probability distribution parameter and the quantized residual value are written into the image code stream.
[0089] In a possible implementation manner of the present disclosure, after the step of adjusting the characteristic channel distinction information and / or the spatial point distinction information according to the reconstructed image block and the image bit rate, the step further includes:
[0090] If the current adjustment round is less than the preset adjustment round, the process returns to the step of constructing the quantization precision parameters corresponding to each feature channel according to the feature channel distinction information.
[0091] In addition, to achieve the above-mentioned object, the present disclosure further proposes an image decoding device, the image decoding device comprising:
[0092] The entropy decoding module is used to obtain characteristic channel distinction information and / or spatial point distinction information, and decode the image code stream to obtain the quantized residual value;
[0093] a parameter construction module, configured to construct an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information;
[0094] an inverse quantization module, configured to inverse quantize any quantized residual value based on the inverse quantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value;
[0095] The synthesis transformation module is used to perform a synthesis transformation on the reconstructed residual value to obtain a reconstructed image block.
[0096] In addition, to achieve the above-mentioned objectives, the present disclosure further provides an image encoding device, the image encoding device comprising:
[0097] An analysis and transformation module is used to perform analysis and transformation on the image block to be processed to obtain image features;
[0098] A residual calculation module, used to perform residual calculation on the image features to obtain image residual values and probability distribution parameters;
[0099] A parameter construction module is used to construct quantization accuracy parameters corresponding to each feature channel according to feature channel distinction information and / or spatial point distinction information;
[0100] a quantization module, configured to perform quantization processing on the image residual value based on the quantization precision parameter to obtain a quantized residual value;
[0101] The entropy coding module is used to write the probability distribution parameters and the quantized residual value into the image code stream.
[0102] In addition, to achieve the above-mentioned purpose, the present disclosure also proposes a decoding device, which includes: a processor, a memory, and a decoding program stored on the memory and capable of running on the processor, and when the decoding program is executed by the processor, it implements the image decoding method described above.
[0103] In addition, to achieve the above-mentioned purpose, the present disclosure also proposes a coding device, which includes: a processor, a memory, and a coding program stored on the memory and runnable on the processor, and when the coding program is executed by the processor, the image coding method as described above is implemented.
[0104] In addition, to achieve the above-mentioned purpose, the present disclosure further proposes a computer-readable storage medium, on which an image decoding program and / or an image encoding program is stored;
[0105] When the image decoding program is executed, the image decoding method described above is implemented, or, when the image encoding program is executed, the image encoding method described above is implemented.
[0106] The present disclosure obtains characteristic channel distinguishing information and / or spatial point distinguishing information and decodes the image code stream to obtain quantized residual values; constructs dequantization precision parameters corresponding to each quantized residual value based on the characteristic channel distinguishing information and / or the spatial point distinguishing information; dequantizes the quantized residual value based on the dequantization precision parameters to obtain a reconstructed residual value; and performs a synthetic transformation on the reconstructed residual value to obtain a reconstructed image block. Because dequantization is performed by constructing corresponding dequantization parameters based on the characteristic channel distinguishing information and / or the spatial point distinguishing information, different quantization parameters can be set during the encoding process based on the differences in the characteristic channel and spatial point image textures, so that more important features suffer less quantization loss, thereby improving encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0107] FIG1 is a schematic diagram of the structure of an electronic device in a hardware operating environment according to an embodiment of the present disclosure;
[0108] FIG2 is a schematic diagram of a flow chart of a first embodiment of an image decoding method disclosed herein;
[0109] FIG3 is a schematic diagram of an image encoding and decoding process according to an embodiment of the present disclosure;
[0110] FIG4 is a schematic diagram of a characteristic channel according to an embodiment of the present disclosure;
[0111] FIG5 is a two-dimensional schematic diagram of the average value of characteristic channel probability distribution parameters according to an embodiment of the present disclosure;
[0112] FIG6 is a schematic flow chart of a second embodiment of an image decoding method disclosed herein;
[0113] FIG7 is a schematic diagram of a decoding process according to an embodiment of the present disclosure;
[0114] FIG8 is a schematic flow chart of a third embodiment of an image decoding method disclosed herein;
[0115] FIG9 is a schematic diagram of an identification matrix according to an embodiment of the present disclosure;
[0116] FIG10 is a schematic diagram of row and column division according to an embodiment of the present disclosure;
[0117] FIG11 is a schematic flow chart of a fourth embodiment of an image decoding method disclosed herein;
[0118] FIG12 is a schematic diagram of spatial point classification according to an embodiment of the present disclosure;
[0119] FIG13 is a schematic diagram of a flow chart of a first embodiment of an image encoding method disclosed herein;
[0120] FIG14 is a schematic diagram of an image encoding process according to an embodiment of the present disclosure;
[0121] FIG15 is a schematic flow chart of a second embodiment of the image encoding method disclosed herein;
[0122] FIG16 is a schematic flow chart of a third embodiment of an image encoding method disclosed herein;
[0123] FIG17 is a schematic diagram of the optimal threshold mode flag function process according to an embodiment of the present disclosure;
[0124] FIG18 is a flow chart of parameter optimization according to an embodiment of the present disclosure;
[0125] FIG19 is a structural block diagram of a first embodiment of an image decoding apparatus disclosed herein;
[0126] FIG20 is a block diagram showing the structure of the first embodiment of the image encoding device disclosed herein.
[0127] The realization of the objectives, functional features and advantages of the present disclosure will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION
[0128] It should be understood that the specific embodiments described herein are only used to illustrate the present disclosure and are not intended to limit the present disclosure.
[0129] 1 , which is a schematic diagram of the structure of an encoding device or decoding device in a hardware operating environment according to an embodiment of the present disclosure.
[0130] As shown in Figure 1, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0131] Those skilled in the art will appreciate that the structure shown in FIG1 does not limit the electronic device and may include more or fewer components than shown, or combine certain components, or arrange the components differently.
[0132] As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image encoding program and / or an image decoding program.
[0133] In the electronic device shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present disclosure can be set in an encoding device or a decoding device, and the electronic device calls the encoding program stored in the memory 1005 through the processor 1001 and executes the image encoding method provided by the embodiment of the present disclosure, or calls the image decoding program stored in the memory 1005 through the processor 1001 and executes the image decoding method provided by the embodiment of the present disclosure.
[0134] In some embodiments of the present disclosure, the differences in feature channels are taken into consideration to improve the quantization accuracy and thus enhance the performance of the quantizer. For example, the feature channels are classified according to their differences, and feature channels of different importance have different quantization accuracy. Features can also be classified according to the differences in texture complexity, and feature regions of different texture complexity have different quantization accuracy. In the present disclosure, all information used for feature channel classification belongs to the relevant information for distinguishing feature channels. The features classified by the above information will be quantized with different accuracy, that is, different quantization step sizes will be used, so that more important features will bear smaller quantization losses, making the prediction of the features to be encoded more accurate, and thus obtaining a better feature reconstruction value y_hat, and ultimately achieving the purpose of obtaining better encoding performance.
[0135] An embodiment of the present disclosure provides an image decoding method. Referring to FIG. 2 , FIG. 2 is a flow chart of a first embodiment of an image decoding method of the present disclosure.
[0136] In this embodiment, the image decoding method includes the following steps S10-S40.
[0137] Step S10: Acquire characteristic channel distinction information and / or spatial point distinction information, and decode the image code stream to obtain a quantized residual value.
[0138] It should be noted that the executor of this embodiment can be a decoding device for decoding image data. The decoding device can be an electronic device such as a camera, a mobile terminal, a personal computer, a server, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method disclosed in this disclosure is described using a decoding device as an example.
[0139] Among them, since during the image encoding process, the encoding device generally decodes the encoded image code stream after the encoding is completed, and determines whether the parameters used in the encoding need to be adjusted based on the image quality of the decoded image, the execution subject of this embodiment can also be the encoding device.
[0140] In the embodiments of the present disclosure, some technical terms that may be involved in the process of encoding or decoding an image include: quantization and dequantization (Quantization and Dequantization), scalar quantization (SQ, Scalar Quantization), entropy encoding (Entropy Encoding), convolutional neural network (CNN), feature channel (Feature Channel), etc., which are explained here.
[0141] Quantization is the process of mapping a signal's continuous values (or a large number of discrete values) into a finite number of discrete amplitudes, achieving a many-to-one mapping of signal values. In video coding, the transform coefficients of the residual signal after transformation often have a large dynamic range. Therefore, quantizing the transform coefficients can effectively reduce the signal value space and achieve better compression. However, due to the many-to-one mapping mechanism, the quantization process inevitably introduces distortion, which is the fundamental cause of distortion in video coding.
[0142] Inverse quantization is the inverse process of quantization, which maps the quantized coefficients to a reconstructed signal in the input signal space. The reconstructed signal is an approximation of the input signal.
[0143] Scalar quantization is the most basic quantization method, mapping a continuous signal (or a large number of discrete-valued signals) into a number of discrete signals. Specifically, scalar quantization takes a one-dimensional scalar signal as input. It first divides the input signal space into a series of mutually exclusive intervals, selecting a representative signal for each interval. Then, for each input signal, scalar quantization maps it to a representative signal for the interval in which it resides.
[0144] The simplest scalar quantization method is uniform scalar quantization, which divides the input signal space into equally spaced intervals. The representative signal of each interval is the interval midpoint. The interval length is called the quantization step, the interval index is called the level, and the parameter that represents the quantization step is the quantization parameter (QP).
[0145] The optimal scalar quantizer is the Lloyd-Max quantizer, which takes into account the distribution of the input signal. The interval division is non-uniform, the representative signal of each interval is the probability centroid of the interval, and the boundary point between two adjacent intervals is the midpoint of the representative signals of the two intervals.
[0146] Entropy coding is a method of encoding data without losing any information, based on the principle of entropy. Information entropy is the average amount of information in a source (a measure of uncertainty). Common entropy coding methods include Shannon coding, Huffman coding, and arithmetic coding.
[0147] Neural Network (NN):
[0148] The neural network mentioned here refers to artificial neural networks, not biological neural networks. A neural network is a computational model composed of a large number of interconnected nodes (or neurons). In an artificial neural network, neuronal processing units can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. Processing units in the network are divided into three types: input units, output units, and hidden units. Input units receive signals and data from the external world; output units output the system's processing results; and hidden units are located between input and output units and cannot be observed from outside the system. The connection weights between neurons reflect the strength of the connections between units, and the representation and processing of information is reflected in the connections between the network's processing units. Artificial neural networks are a non-programmed, brain-like information processing method. Their essence is to achieve parallel and distributed information processing capabilities through network transformations and dynamic behavior, mimicking the information processing functions of the human brain to varying degrees and levels. Currently, commonly used neural networks in the field of video processing include convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected networks.
[0149] Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. Its artificial neurons can respond to surrounding units within a certain coverage area and perform well in large-scale image processing.
[0150] Generally speaking, the basic structure of CNN includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer, and the local features are extracted. Once the local feature is extracted, the positional relationship between the local feature and other features is also determined; the second is the feature mapping layer (also called activation layer). Each computing layer of the network is composed of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since the neurons on a mapping surface share weights, the number of free parameters of the network is reduced.
[0151] One of the advantages of CNN over traditional image processing algorithms is that it avoids complex pre-processing of images (such as extracting artificial features) and can directly input raw images for end-to-end learning. One of the advantages of CNN over traditional neural networks is that traditional neural networks use a fully connected approach, meaning that all neurons from the input layer to the hidden layer are fully connected. This results in a huge number of parameters, making network training time-consuming or even difficult. CNN avoids this difficulty through methods such as local connections and weight sharing.
[0152] In a neural network, the input signal can obtain many feature maps after passing through the convolution layer. Each feature map is called a feature channel and contains a part of the information of the input signal.
[0153] It should be noted that the feature channel differentiation information may be differentiation information for dividing the feature channels of the image, and the feature channel differentiation information may include the rules for dividing the feature channels, as well as the quantization precision parameters when quantizing the residual values corresponding to each type of feature channel after the division. The spatial point differentiation information may be differentiation information for classifying each point in the image into categories, and the spatial point differentiation information may include the rules for classifying each spatial point in the image based on the image texture. If necessary, the spatial point differentiation information may also include the quantization precision parameters when quantizing the residual values corresponding to each type of spatial point after the division.
[0154] Among them, the feature channel or spatial point division rules can be set based on the probability distribution parameters, or based on the mean value or variance of the probability distribution parameters. For example, the corresponding threshold is set to obtain multiple value intervals, and the feature channels are divided into multiple categories according to the value intervals to which the corresponding probability distribution parameters belong.
[0155] Step S20: constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information.
[0156] It should be noted that constructing the inverse quantization precision parameters corresponding to each quantized residual value based on the characteristic channel distinction information and / or the spatial point distinction information can be to determine the characteristic channel category or spatial point category corresponding to each quantized residual value based on the characteristic channel distinction information and / or the spatial point information, thereby obtaining the inverse quantization precision parameters that should be used when performing inverse quantization processing on each quantized residual value.
[0157] Step S30: Dequantize the quantized residual value based on the dequantization precision parameter to obtain a reconstructed residual value.
[0158] It is understandable that after determining the inverse quantization precision parameter corresponding to each quantized residual value, the quantized residual value can be inversely quantized according to the inverse quantization precision parameter, that is, inverse quantization can be performed, and the value obtained after processing can be used as the reconstructed residual value.
[0159] Step S40: performing a synthesis transformation on the reconstructed residual value to obtain a reconstructed image block.
[0160] It should be noted that after obtaining the reconstructed residual value, the mean used to calculate the feature residual value can be obtained through context prediction. The reconstructed residual value can be reconstructed into image features through the mean, and then the image features can be reconstructed into image blocks through the synthetic transformation network to obtain the reconstructed image blocks.
[0161] It is understood that if the encoding device divides the image data into only one image block for processing during encoding, the reconstructed image block obtained at this time is the reconstructed image data corresponding to the image data. If the encoding device divides the image data into multiple image blocks for processing during encoding, the reconstructed image block obtained at this time is only the reconstructed image data corresponding to a specific image block in the image data.
[0162] For ease of understanding, the following description will be given in conjunction with Figures 3, 4, and 5, but this does not limit the present solution. Figure 3 is a schematic diagram of the image encoding and decoding process of this embodiment. As shown in Figure 3, the image encoding and decoding process mainly involves modules such as the analysis transform network, the synthesis transform network, context-based prediction, quantization, entropy coding, the super encoding network, and the super-scale decoding network, among which:
[0163] Analysis Transformation Network: transforms image x into feature y in the feature domain (latent domain), making it easier for all subsequent processes to operate in the latent domain;
[0164] Super coding network: Input the probability distribution parameter σ of the residual r to obtain the super prior information z_hat, which will be written to the image code stream biasstream#1, mainly transmitting the probability model parameter σ of the residual.
[0165] Context-based prediction module: The input of this module includes not only z_hat but also the decoded y_hat. The two are combined to obtain a more accurate mean mu. The mean mu is the predicted value of the original feature, and the reconstructed residual value r_hat is added to obtain the reconstructed y_hat.
[0166] Super-scale decoding network: The output z_hat of the super-coding network is used as the input of the super-scale decoding network to obtain the probability distribution parameter σ. After quantization, the distribution quantization parameter σ_hat is obtained. σ_hat will be used to obtain the quantized residual value r_q in conjunction with the second image code stream (Bitstream#2).
[0167] Synthetic transformation module: The reconstructed latent domain feature y_hat is obtained and the reconstructed image is obtained through the synthetic transformation module.
[0168] Entropy coding: A lossless coding method based on the principle of information entropy, which converts a series of element symbols used to represent a video sequence (such as transform coefficients and mode information) into a binary code stream, removing the statistical redundancy of these video element symbols.
[0169] Quantization: A "many-to-one" mapping process that results in signal loss. Quantization acts on the residual r, changing the signal's range of values. This allows the encoder to provide a good approximation of the original signal using a smaller number of symbols, thereby improving compression. Quantization also acts on σ to obtain σ_hat, which allows it to better estimate the probability distribution of the quantized residual r_q. Inverse quantization is the inverse of quantization. The quantization precision parameters or inverse quantization precision parameters used in both quantization and inverse quantization are constructed based on information distinguishing between characteristic channels and / or spatial points.
[0170] Figure 4 is a schematic diagram of the feature channels of this embodiment, and Figure 5 is a two-dimensional schematic diagram of the average values of the probability distribution parameters of the feature channels of this embodiment. As shown in Figure 3 above, after the image x is processed by the analysis and transformation network, the feature y obtained contains C feature channels (Channels). The number of C depends on the number of convolution kernels used in the analysis and transformation network. As shown in Figure 4, the amount of information contained in each feature channel is different. Some feature channels contain more information, which is almost the same as the input image, and some feature channels contain almost no information and are basically all noise. Therefore, using different precision quantization for different feature channels helps to obtain better coding performance.
[0171] In addition, a 2D image can be obtained by averaging the σ of all feature channels, as shown in Figure 5. In Figure 5, it can be determined that the texture values of some areas are higher, that is, these areas are more important. Allocating more codewords to the complex texture parts of the image will help improve the coding performance.
[0172] This embodiment obtains quantized residual values by acquiring characteristic channel distinguishing information and / or spatial point distinguishing information and decoding the image code stream; constructs inverse quantization precision parameters corresponding to each quantized residual value based on the characteristic channel distinguishing information and / or the spatial point distinguishing information; dequantizes the quantized residual values based on the inverse quantization precision parameters to obtain reconstructed residual values; and performs a synthetic transformation on the reconstructed residual values to obtain a reconstructed image block. Because dequantization is performed by constructing corresponding inverse quantization precision parameters based on characteristic channel distinguishing information and / or spatial point distinguishing information, different quantization parameters can be set during the encoding process based on the differences in characteristic channel and spatial point image textures, allowing more important features to incur less quantization loss, thereby improving encoding and decoding performance.
[0173] Refer to FIG6 , which is a flowchart of a second embodiment of an image decoding method disclosed herein.
[0174] In this embodiment, the image code stream may include a first image code stream and a second image code stream, wherein the first image code stream is used to transmit decoding auxiliary information, and the second image code stream is used to transmit residual data.
[0175] Based on the first embodiment, step S10 of the image decoding method of this embodiment includes steps S101 - S104 .
[0176] Step S101: Acquire characteristic channel distinguishing information and / or spatial point distinguishing information, and extract coding distribution parameters from the first image code stream.
[0177] It should be noted that extracting the coding distribution parameter from the first image code stream may be performing entropy decoding on the first image code stream and reading decoding auxiliary information contained in the first image code stream to obtain the coding distribution parameter.
[0178] Step S102: Decode the coded distribution parameters to obtain probability distribution parameters.
[0179] It should be noted that decoding the coding distribution parameters to obtain the probability distribution parameters may be performed by decoding the read decoding auxiliary information through a super-scale decoding network to obtain the probability distribution parameters.
[0180] Step S103: quantizing the probability distribution parameters to obtain distribution quantization parameters.
[0181] It should be noted that when the encoding device writes the residual data into the second image code stream, it does so based on the quantized probability distribution parameters. Therefore, when decoding the second image code stream, it is also necessary to obtain the quantized probability distribution parameters for decoding. At this time, in order to ensure that the second image code stream can be decoded normally, the probability distribution parameters can be quantized in the same way as when the encoding device is encoding to obtain the distribution quantization parameters.
[0182] In a specific implementation process, in order to ensure that the obtained distribution quantization parameters are consistent with those used by the encoding device during encoding, it is necessary to ensure that the parameters used when quantizing the probability distribution parameters are consistent with the parameters used by the encoding device during encoding. Step S103 in this embodiment may include:
[0183] Constructing a probability quantization parameter according to the characteristic channel distinction information and / or the spatial point distinction information;
[0184] The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
[0185] It should be noted that constructing probability quantization parameters based on characteristic channel distinction information and / or spatial point distinction information may be parameters used when quantizing probability distribution parameters extracted from characteristic channel distinction information and / or spatial point distinction information.
[0186] In a possible implementation of this embodiment, in order to facilitate decoding by the decoding device, the encoding device may write the characteristic channel distinguishing information and / or spatial point distinguishing information used in the encoding process into the first image code stream or the second image code stream. At this time, obtaining the characteristic channel distinguishing information and / or spatial point distinguishing information may be decoding the first image code stream or the second image code stream to extract the characteristic channel distinguishing information and / or spatial point distinguishing information therefrom.
[0187] In a possible implementation of this embodiment, in order to avoid code stream confusion, the encoding device may also set a third image code stream, and transmit the characteristic channel distinguishing information and / or spatial point distinguishing information used in the encoding process through the third image code stream. At this time, obtaining the characteristic channel distinguishing information and / or spatial point distinguishing information can be decoding the third image code stream to extract the characteristic channel distinguishing information and / or spatial point distinguishing information therefrom.
[0188] Step S104: decoding the second image code stream based on the distributed quantization parameter to obtain a quantized residual value.
[0189] It should be noted that when the encoding device writes the quantized residual value into the second image code stream, it performs entropy encoding on the quantized residual value based on the quantized distributed quantization parameter. At this time, after obtaining the distributed quantization parameter, the second image code stream can be subjected to an inverse operation, i.e., entropy decoding, according to the distributed quantization parameter to obtain the quantized residual value.
[0190] For ease of understanding, this embodiment will be described with reference to FIG7 , but this embodiment is not intended to limit the present invention. FIG7 is a schematic diagram of the decoding process of this embodiment. As shown in FIG7 , the decoding end receives at least two image streams (a first image stream Bitstream#1 and a second image stream Bitstream#2), and may also receive a third image stream Bitstream#3.
[0191] At this time, the decoding process includes: parsing the Bitstream#1 auxiliary bitstream to obtain a z_hat, through which the parameter σ of the probability distribution model of the residual can be obtained; obtaining relevant information for distinguishing feature channels or texture areas (feature channel distinguishing information and / or spatial point distinguishing information), and using this information to classify the feature channels or spatial points. The quantization precision of different categories of feature channels or different categories of spatial points is different. The relevant information for distinguishing feature channels can be transmitted from the encoding device to the decoding device through Bitstream#1 or Bitsstream#2, or can be transmitted through the new bitstream Bitsstream#3, or the relevant information can be written into the encoding and decoding device in advance without the need for bitstream transmission; entropy decoding is performed on Bitsstream#2, and the quantized residual r_q is obtained by combining σ with σ_hat obtained by quantization; the relevant information for distinguishing feature channels or spatial points will act on the inverse quantization of the quantized residual value r_q and make the quantized residual values r_q and σ corresponding to different categories of feature channels or different categories of spatial points have different quantization precisions.
[0192] Refer to FIG8 , which is a flowchart of a third embodiment of an image decoding method disclosed herein.
[0193] Based on the above first embodiment, step S20 of the image decoding method of this embodiment includes steps S201 - S202 .
[0194] Step S201: extracting the quantization step length corresponding to each type of feature channel from the feature channel distinction information.
[0195] It should be noted that when distinguishing feature channels, the feature channels can be divided into multiple categories, and different quantization steps can be assigned to feature channels of different categories. At this time, the feature channel distinction information will include the classification rules of the feature channels and the quantization steps corresponding to each type of feature channel.
[0196] Among them, the classification rules of the feature channels can be set in the following ways 1-4.
[0197] Method 1: Set according to the sequence number of the feature channel:
[0198] Set the corresponding quantization step size for each feature channel corresponding to the sequence number, and then classify the feature channels with the same quantization step size into one category. For example, assuming that the sequence number of the feature channel is 1-X, the feature channel quantization step size list can be: channel_list = [channel1, channel2, ..., channelX];
[0199] Furthermore, the same feature channel may also include components, such as brightness or chrominance. In this case, the brightness or chrominance can be further subdivided. For example: assuming that the brightness channel sequence is 1-X1 and the chrominance channel sequence is 1-X2, the brightness channel quantization step list is: channel_listY = [channel1, chanenel2, ..., channelX1], and the chrominance channel quantization step list is: channel_listUV = [channel1, chanenel2, ..., channelX2].
[0200] Method 2: Set the corresponding category based on the channel flag list;
[0201] A corresponding flag bit is set for the feature channel corresponding to each serial number, and the category of the feature channel corresponding to each serial number is identified by the flag bit.
[0202] For example, taking the brightness channel as an example, the brightness channel is numbered 1-128, and the flag list channel_listY = [flag0, flag1, ..., flag128], where the value of flag is the category corresponding to the brightness channel of the sequence number. If there are only two categories, the value of flag is 0 and 1.
[0203] Method 3: Classify feature channels by setting thresholds:
[0204] The category of the corresponding feature channel is set by the threshold calculated from the probability distribution parameters such as the mean value or variance of the probability distribution parameter σ, the sum of the absolute values of the quantized residual values or the bit rate and other related thresholds calculated from the quantized residual r_q, for example: thr_list = [thr1, thr2, ..., thrX].
[0205] Method 4: Classify feature channels by the number of channels: Sort the channels using a list of related thresholds such as the mean value or variance of σ calculated from the distribution parameter σ, or the sum of the absolute values of the quantization residual r_q or the bit rate, and set multiple number thresholds (multiple number thresholds can be represented by a list, such as: num_list = [num1, num2, ..., numX], where X is the length of the list, i.e., the number of number thresholds). Classify the channels according to the number threshold, for example, the first three categories are classified into one category, and the rest are classified into one category. Assuming that the channel number threshold can be set to 3 at this time, num_list = [3] can be set at this time.
[0206] Step S202: constructing inverse quantization precision parameters corresponding to each quantized residual value according to the quantization step size.
[0207] It should be noted that after obtaining the quantization step size corresponding to each type of feature channel, the parameters such as the quantization step size used to quantize each quantized residual value can be obtained according to the feature channel corresponding to each quantized residual value, thereby calculating the inverse quantization accuracy parameters required for inverse quantization of each quantized residual value.
[0208] For example: assuming that the feature channels are divided into X categories, the quantization step sizes corresponding to each category of feature channels in the feature channel distinction information can be represented by a list scale_list = [scale1, scale2, ..., scaleX], where X is the number of quantization steps. At this time, assuming that the quantization residual value corresponding to the j-th category feature channel is rqj, the feature residual is rj, and the reconstructed residual value obtained by inverse quantization is r_hatj, then the quantization process rqj = round(rj*scalej*scalei), and the inverse quantization process r_hatj = rqj / (scalej*scalei), where scalej is the quantization step size used when quantizing the feature residual corresponding to the j-th category feature channel according to the feature channel, and scalei is the quantization step size used when quantizing the feature residual corresponding to the j-th category feature channel in the original scheme. scalei can be set to 1, that is, the quantization step size of the original scheme can be retained or not. Similarly, in the following embodiments, you can choose whether to retain the quantization step size of the original scheme.
[0209] In a specific implementation process, different quantization step sizes may be further set for the same feature channel, such as by dividing multiple value intervals by setting a threshold, and further setting the value intervals to which the values calculated based on the probability distribution parameters belong. Step S202 in this embodiment may include:
[0210] Decode the image code stream to obtain probability distribution parameters;
[0211] Extracting parameter interval thresholds from the feature channel differentiation information;
[0212] The inverse quantization precision parameters corresponding to the respective quantization residual values are constructed according to the probability distribution parameter, the quantization step size and the parameter interval threshold.
[0213] It should be noted that when constructing the inverse quantization precision parameter corresponding to the quantization difference residual value, the characteristic channel category to which the quantization residual value belongs can be obtained first, and the quantization step size and parameter interval threshold corresponding to this type of characteristic channel can be obtained. Multiple value intervals are divided according to the parameter interval threshold. Then, the value interval to which the probability distribution parameter corresponding to the quantization residual value belongs is calculated (it can also be the value interval to which the mean value or variance of the probability distribution parameter belongs). According to the value interval, the corresponding quantization step size is selected from the multiple quantization step sizes corresponding to this type of characteristic channel. Then, the corresponding inverse quantization precision parameter when the quantization residual value is inversely quantized is calculated according to the quantization step size. Among them, the multiple quantization step sizes corresponding to a type of characteristic channel correspond to the divided value intervals respectively.
[0214] For example: assuming that the feature channel corresponding to the quantized residual value is class i, the quantization step size corresponding to this class of feature channels includes scale1, scale2, ..., scaleN, and the parameter interval thresholds include thr1, thr2, ..., thrN, which are different threshold values (thr1, thr2, ..., thrN are in descending order). Then, the value interval may include: (thr1, ∞), (thr2, thr1], ..., (thrN, thrN-1], and each value interval corresponds to the quantization step size one by one. At this time, if the probability distribution parameter σ corresponding to the quantized residual value is > thr1, then the corresponding quantization step size is scale1. If the probability distribution parameter thr1≥σ> thr2 corresponding to the quantized residual value is thr1≥σ> thr2, then the corresponding quantization step size is scale2, and so on.
[0215] In a specific implementation process, the same type of feature channels may be further classified. For example, a type of feature channel may be divided into multiple segments, and each segment may correspond to a different quantization step size. In this case, step S20 of this embodiment may include:
[0216] Parsing the feature channel distinction information to obtain a set of segmented step sizes corresponding to each type of feature channel;
[0217] Extracting the quantization step size corresponding to each channel segment from the segment step size set;
[0218] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0219] It should be noted that after further dividing a type of feature channel, each type of feature channel can correspond to multiple different quantization step sizes, which can be saved in the form of a set. At this time, the feature channel differentiation information can be parsed to obtain the segmentation step size set corresponding to each type of feature channel, and then the quantization step size corresponding to each channel segment can be extracted from the segmentation step size set. The segmentation method of different types of feature channels can be different or the same, and this embodiment does not limit this.
[0220] At this time, constructing the inverse quantization precision parameters corresponding to the quantized residual values according to the quantization step size can be to obtain the characteristic channel category corresponding to the quantized residual value, and determine the channel segment to which the quantized residual value belongs in this type of characteristic channel, and determine the quantization step size and other parameters used when calculating the quantized residual value according to the characteristic channel category and channel segmentation, so as to calculate the inverse quantization precision parameters required for inverse quantization of the quantized residual value.
[0221] For example, taking the i-th feature channel as an example, the i-th feature channel can be divided into X segments. At this time, the segmentation step size set corresponding to the i-th feature channel can be expressed as: scale_listi = [scalei1...scaleij...sscleiX], where scaleij is the quantization step size of the j-th segment in the i-th feature channel, and the value of j is [1, X].
[0222] In specific implementations, features within the same feature channel can also be divided into multiple categories, and the classification rules can be any of the following:
[0223] Rule 1, the list of relevant thresholds such as the mean value or variance of σ calculated by the distribution parameter σ, or the list of relevant thresholds such as the sum of the absolute values of the quantized residual r_q or the bit rate calculated by the quantized residual r_q, and the list length X, thr brightness and chrominance components can share one list, thr_list = [thr1, thr2, ..., thrX], or they can be passed separately.
[0224] Rule 2: A two-dimensional channel flag matrix, as shown in Figure 7, can be used to combine the luminance and chrominance components into a single matrix or pass them separately, similar to the channel flag list in Example 1. Different flags represent different categories.
[0225] Rule 3, row, column, oblique column numbers and list lengths are similar to channel numbers, as shown in Figure 7.
[0226] Rule 4: Pass row, column, and diagonal column flags, similar to the channel flag list.
[0227] Rule 5: The row, column, and diagonal column number table num_list = [num1, num2, ..., numX] and the list length X are similar to the channel number list.
[0228] Rule 6: Row, column, and diagonal column flags are similar to channel flags.
[0229] 9 and 10 are used for explanation. FIG9 is a schematic diagram of the identification matrix of this embodiment. As shown in FIG9 , the relevant data are divided into multiple categories, and the data corresponding to each category can correspond to different quantization steps. At this time, the relevant feature data in the channel can be classified and marked in a matrix manner. As shown in FIG9 , 0 and 1 in the matrix respectively represent the classification marks of the relevant feature data in the same channel.
[0230] Figure 10 is a schematic diagram of the row and column division of this embodiment. The specific row, column, and oblique column division can be as shown in Figure 8. From left to right, the relevant feature data in the channel can be divided into multiple rows, from top to bottom, the relevant feature data in the channel can be divided into multiple columns, and from the lower left to the upper right direction, the relevant feature data in the channel can be divided into multiple oblique columns.
[0231] In a possible implementation of this example, each feature channel may be divided into blocks and different quantization step sizes may be set. In this case, step S20 of this embodiment may include:
[0232] Extracting image segmentation information and parameter setting rules corresponding to each feature channel from the feature channel distinction information;
[0233] Decode the image code stream to obtain probability distribution parameters;
[0234] Calculate the distribution parameter mean corresponding to each image block information according to the probability distribution parameter;
[0235] Matching the distribution parameter mean with the parameter setting rule to obtain the quantization step size corresponding to each image block information;
[0236] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0237] It should be noted that each feature channel can be divided into blocks during encoding, and the block rules can be recorded in the feature channel distinction information. After that, different quantization step sizes are set according to the different means of the probability distribution parameters corresponding to these blocks, where the size of the blocks can be set freely.
[0238] Extracting the image segmentation information and parameter setting rules corresponding to each feature channel from the feature channel differentiation information may include extracting the parameter setting rules and the segmentation rules from the feature channel differentiation information, determining the segmentation corresponding to each feature channel according to the segmentation rules, and thereby obtaining the image segmentation information corresponding to each feature channel. The parameter setting rules include quantization step sizes corresponding to different distribution parameter means.
[0239] In actual use, calculating the mean value of the distribution parameters corresponding to each image block information according to the probability distribution parameters may be calculating the average value of the probability distribution parameters corresponding to each image block information, thereby obtaining the mean value of the distribution parameters.
[0240] During the implementation process, the distribution parameter mean is matched with the parameter setting rules to obtain the quantization step corresponding to each image block information. The distribution parameter mean can be matched with multiple value intervals included in the parameter setting rules, and the quantization step corresponding to the value interval to which the distribution parameter mean belongs is used as the quantization step corresponding to the image block information.
[0241] It can be understood that after obtaining the quantization step corresponding to the image block information, the quantization step and other parameters used to calculate the quantized residual value can be determined according to the image block information to which the quantized residual value belongs, thereby calculating the inverse quantization accuracy parameters required for inverse quantization of the quantized residual value.
[0242] This embodiment constructs a probability quantization parameter based on feature channel differentiation information and / or spatial point differentiation information; and quantizes the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter. Since the probability quantization parameter is constructed based on feature channel differentiation information and / or spatial point differentiation information, and the probability distribution parameter is quantized based on the probability quantization parameter to obtain the distribution quantization parameter, different quantization parameters can be set for the probability distribution parameter based on differences in feature channel and spatial point image texture, further improving the performance of the encoding method of this embodiment.
[0243] Refer to FIG11 , which is a flowchart of a fourth embodiment of an image decoding method disclosed herein.
[0244] Based on the first embodiment, step S20 of the image decoding method of this embodiment includes steps S201 ′-S202 ′.
[0245] Step S201 ′: extracting the quantization step length corresponding to each type of spatial point from the spatial point distinction information.
[0246] It should be noted that when setting the quantization step size, the spatial points in the image can be divided into multiple categories according to the image texture, and different quantization step sizes can be set for spatial points of different categories. For example: according to the image texture, the spatial points in the image are divided into X categories, and the quantization step size of the j-th category spatial point is scale_j, and the value range of j is [1, X].
[0247] Among them, when classifying spatial points, they can be distinguished according to the complexity of the image texture corresponding to the spatial points. For example, three complexity intervals are set, [0, F1], (F1, F2], (F2, ∞), and then the spatial points are divided into three categories according to the complexity interval to which the image texture complexity corresponding to the spatial points belongs.
[0248] Step S202 ′: constructing inverse quantization precision parameters corresponding to respective quantized residual values according to the quantization step size.
[0249] It can be understood that constructing the inverse quantization precision parameters corresponding to each quantized residual value according to the quantization step corresponding to each type of spatial point can be based on the spatial point category corresponding to the quantized residual value, obtaining the quantization step and other parameters used in calculating the quantized residual value, thereby calculating the inverse quantization precision parameters required for inverse quantization of each quantized residual value.
[0250] It should be noted that each spatial point actually contains C (C is the number of feature channels) points. After dividing the spatial points into different categories, the multiple points contained in the spatial points can actually be further divided so that different feature channels in a type of spatial point can correspond to different quantization step sizes. In this case, the spatial point distinction information and the feature channel distinction information can be used in conjunction. In this case, step S20 in this embodiment may include:
[0251] Reading the various types of divided spatial points from the spatial point distinction information;
[0252] Reading the quantization step lengths corresponding to the various feature channels in various spatial points from the feature channel differentiation information;
[0253] The inverse quantization precision parameters corresponding to the respective quantized residual values are constructed according to the quantization step size.
[0254] It should be noted that, when the multiple points contained in each type of spatial point are further divided according to the different characteristic channels, the points of multiple different characteristic channels corresponding to one type of spatial point can correspond to different quantization step sizes. At this time, the multiple types of spatial points that can be divided can be read from the spatial point distinction information first, and then the quantization step sizes corresponding to each characteristic channel in each type of spatial point can be read.
[0255] For example: divide the spatial points into two categories a and b. At this time, the feature channels contained in a and b are both 1-X. Then the quantization step size corresponding to each feature channel in category a spatial points can be expressed as scale_lista=[scale1, scale2, ..., scaleX], and the quantization step size corresponding to each feature channel in category b spatial points can be expressed as scale_listb=[scale1, scale2, ..., scaleX].
[0256] In a specific implementation, in order to facilitate the decoding device to clarify the specific quantization step setting method, a corresponding flag bit can also be set in the image code stream to determine how to obtain the quantization step size. For example: if the flag bit is set to 0, the quantization step size is determined based on the characteristic channel distinction information. If the flag bit is set to 1, the quantization step size is determined based on the spatial point distinction information. If the flag bit is set to 2, the quantization step size is determined based on the characteristic channel distinction information and the spatial point distinction information.
[0257] It should be noted that this embodiment only describes a method of using spatial point distinction information in conjunction with characteristic channel distinction information. Further division of multiple points contained in the spatial points can also utilize the implementation method for further division of characteristic channels provided in any of the above embodiments, and this embodiment does not limit this.
[0258] For ease of understanding, this is explained in conjunction with Figure 12, but this solution is not limited thereto. Figure 12 is a schematic diagram of the spatial point classification of this embodiment. According to the image texture complexity corresponding to the spatial point, the spatial point can be divided into the first category, the second category, and the third category (as shown by the arrows in the figure). The specific spatial points after division are shown in Figure 12 (the color depth of each point in Figure 12 is used to represent the image texture complexity. If the color depth is consistent, it belongs to the same category, and the darker the color, the higher the image texture complexity).
[0259] This embodiment extracts the quantization step sizes corresponding to each type of spatial point from the spatial point differentiation information; and constructs inverse quantization precision parameters corresponding to each quantized residual value based on the quantization step sizes. Since inverse quantization precision parameters corresponding to each quantized residual value are constructed based on the quantization step sizes corresponding to each type of spatial point, different quantization step sizes can be set for spatial points with different image textures, allowing differentiated settings to be made for quantization and inverse quantization based on differences in image textures.
[0260] An embodiment of the present disclosure provides an image encoding method. Referring to FIG. 13 , FIG. 13 is a flow chart of a first embodiment of an image encoding method of the present disclosure.
[0261] In this embodiment, the image encoding method includes the following steps S100-S500.
[0262] Step S100: Analyze and transform the image block to be processed to obtain image features.
[0263] It should be noted that the executor of this embodiment may be an encoding device for encoding image data. The encoding device may be an electronic device such as a personal computer or a server, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image encoding method disclosed herein is described using an encoding device as an example.
[0264] It should be noted that performing analysis and change processing on the image block to be processed to obtain image features can be performed by performing analysis and change processing on the image block to be processed through an analysis transformation network to extract feature information, thereby obtaining image features.
[0265] Step S200: performing residual calculation on the image features to obtain image residual values and probability distribution parameters.
[0266] It should be noted that the residual calculation of the image features to obtain the image residual value and probability distribution parameters can be a prediction value generated by a context-based prediction module, and then the residual calculation is performed based on the image features and the prediction value to obtain the graphic residual value, and the image residual value is analyzed through the super coding network to obtain the probability distribution parameters.
[0267] Step S300: constructing quantization precision parameters corresponding to respective image residual values according to feature channel distinction information and / or spatial point distinction information.
[0268] It should be noted that the feature channel distinguishing information can be distinguishing information for classifying image feature channels, and the spatial point distinguishing information can be distinguishing information for classifying each point in the image into categories. Both the feature channel distinguishing information and the spatial point distinguishing information can be pre-configured by the encoding device administrator based on actual needs.
[0269] It should be noted that constructing each quantized residual value based on the characteristic channel distinguishing information and / or the spatial point distinguishing information may correspond to determining the characteristic channel category or spatial point category corresponding to each image residual value based on the characteristic channel distinguishing information and / or the spatial point information, and reading the quantization precision parameter required for quantizing each image residual value from the characteristic channel distinguishing information and / or the spatial point distinguishing information based on the characteristic channel category or the spatial point category. The quantization precision parameter may be a parameter that can limit the quantization precision, such as a quantization step size.
[0270] Step S400: quantizing the image residual value based on the quantization precision parameter to obtain a quantized residual value.
[0271] It should be noted that, the image residual value is quantized based on the quantization precision parameter to obtain the quantized residual value, which may be performed by quantizing the image residual value according to the quantization precision parameter and using the calculated value as the quantized residual value.
[0272] Step S500: writing the probability distribution parameter and the quantized residual value into an image code stream.
[0273] It should be noted that writing the probability distribution parameters and the quantized residual values into the image code stream can be performed by processing the probability distribution parameters through a super coding network and then writing them into the image code stream through entropy coding, and entropy coding the quantized residual values based on the quantized probability distribution parameters and writing them into the image code stream.
[0274] In a specific implementation, the image code stream may include a first image code stream and a second image code stream, the first image code stream is used to transmit decoding auxiliary information, and the second image code stream is used to transmit residual data. Then, step S500 in this embodiment may include:
[0275] Writing the probability distribution parameters into the first image code stream;
[0276] Constructing distribution quantization parameters according to the probability distribution parameters;
[0277] The quantized residual value is written into the second image code stream based on the distributed quantization parameter.
[0278] It should be noted that writing the probability distribution parameters into the first image codestream may involve processing the probability distribution parameters through a super coding network and then writing them into the first image codestream through entropy coding. Constructing the distribution quantization parameters based on the probability distribution parameters may involve quantizing the probability distribution parameters to obtain the distribution quantization parameters. Writing the quantized residual values into the second image codestream based on the distribution quantization parameters may involve entropy coding the quantized residual values based on the distribution quantization parameters and writing them into the second image codestream.
[0279] In order to ensure the performance of the quantizer, when quantizing the probability distribution parameters, the quantization parameters used may be constructed based on the characteristic channel distinction information and / or the spatial point distinction information. In this case, the step of constructing the distribution quantization parameters based on the probability distribution parameters in this embodiment may include:
[0280] Constructing a probability quantization parameter according to the characteristic channel distinction information and / or the spatial point distinction information;
[0281] The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
[0282] It should be noted that by constructing probabilistic quantization parameters based on feature channel differentiation information and / or spatial point differentiation information, different quantization parameters can be set according to the different feature channels or spatial points corresponding to the probability distribution parameters, so that the quantization can fully adapt to the differences in feature channels and the complexity of image textures, thereby ensuring the performance of the quantizer as much as possible.
[0283] In actual use, in order to ensure that the decoding device can normally decode the generated image code stream, the characteristic channel distinction information and / or the spatial point distinction information can also be encoded into the image code stream.
[0284] It can be understood that since the parameters used for quantization are adjusted according to the characteristic channel distinguishing information and / or the spatial point distinguishing information during encoding, if the decoding device is to be able to decode normally, it is also necessary to obtain the characteristic channel distinguishing information and / or the spatial point distinguishing information. At this time, the characteristic channel distinguishing information and / or the spatial point distinguishing information are written into the first image code stream or the second image code stream, which can ensure that the decoding device can obtain the characteristic channel distinguishing information and / or the spatial point distinguishing information, thereby ensuring that the decoding device can decode normally.
[0285] In actual use, in order to avoid code stream data confusion or excessive complexity, a third image code stream may be provided to transmit the distinguishing information through the third image code stream. In this case, after the step of writing the quantized residual value into the second image code stream based on the distributed quantization parameter in this embodiment, the following step may be further included:
[0286] The characteristic channel distinguishing information and / or the spatial point distinguishing information are written into the third image code stream.
[0287] For ease of understanding, the present invention will be described in conjunction with Figure 14, but this solution is not limited thereto. Figure 14 is a schematic diagram of the image encoding process of this embodiment. As shown in Figure 14, the encoding device can write the quantized residual value into the second image code stream, write the encoding distribution parameter z_hat calculated by the probability distribution parameter σ into the first image code stream, write the relevant information for distinguishing feature channels or spatial points (feature channel distinguishing information and / or the spatial point distinguishing information) into the first image code stream or the second image code stream, or write it into the third image code stream. If necessary, the relevant information for distinguishing feature channels or spatial points can also be directly written into the encoding device in advance as a local parameter without the need for code stream transmission, and the encoding device end and the decoding device end can be kept consistent.
[0288] This embodiment analyzes and transforms the image blocks to be processed to obtain image features; performs residual calculation on the image features to obtain image residual values and probability distribution parameters; constructs quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information; quantizes the image residual values based on the quantization precision parameters to obtain quantized residual values; and writes the probability distribution parameters and quantized residual values into the image bitstream. Because different quantization parameters can be set during the encoding process based on differences in feature channel and spatial point image texture, more important features incur less quantization loss, thereby improving encoding and decoding performance.
[0289] Refer to FIG15 , which is a flowchart of a second embodiment of an image encoding method disclosed herein.
[0290] Based on the first embodiment of the image coding method described above, step S300 of the image coding method of this embodiment includes steps S3001 and S3002.
[0291] Step S3001: extracting the quantization step size corresponding to each type of feature channel from the feature channel distinction information.
[0292] It should be noted that when distinguishing feature channels, the feature channels can be divided into multiple categories, and different quantization step sizes can be assigned to feature channels of different categories. At this time, the feature channel distinction information will include the classification rules of the feature channels and the quantization step sizes corresponding to each type of feature channel. At this time, the quantization step sizes corresponding to each type of feature channel can be directly extracted from the feature channel distinction information.
[0293] Among them, the classification rule setting method of the feature channel is similar to the above setting method and will not be repeated here.
[0294] Step S3002: constructing quantization precision parameters corresponding to each image residual value according to the quantization step size.
[0295] It can be understood that after determining the quantization step size corresponding to each type of feature channel, the quantization step size and other parameters when quantizing the image residual value can be determined according to the feature channel category corresponding to each image residual value, thereby determining the quantization accuracy parameters corresponding to each image residual value.
[0296] In a specific implementation, different quantization step sizes may be further set for the same feature channel. For example, a threshold may be set to divide the value intervals into multiple value intervals, and the value intervals may be further set based on the probability distribution parameters or the value calculated based on the probability distribution parameters. Step S3002 in this embodiment may include:
[0297] Extracting parameter interval thresholds from the feature channel differentiation information;
[0298] The quantization precision parameters corresponding to the respective image residual values are constructed according to the probability distribution parameters, the parameter interval threshold and the quantization step size.
[0299] It should be noted that when constructing the quantization precision parameters corresponding to each feature channel according to the probability distribution parameters, parameter interval thresholds and quantization step sizes, you can first obtain the feature channel category to which the quantized residual value belongs, obtain the quantization step size and parameter interval threshold size corresponding to this type of feature channel, divide it into multiple value intervals according to the parameter interval threshold value, and then calculate the value interval to which the probability distribution parameter corresponding to the quantized residual value belongs (it can also be the value interval to which the mean value or variance of the probability distribution parameter belongs), and select the corresponding quantization step size from the multiple quantization step sizes corresponding to this type of feature channel according to the value interval, so as to obtain the quantization precision parameters corresponding to each image residual value.
[0300] In a specific implementation process, the same type of feature channels may be further classified. For example, a type of feature channel may be divided into multiple segments, each segment corresponding to a different quantization step size. In this case, step S3002 of this embodiment may include:
[0301] Parsing the feature channel distinction information to obtain a set of segmented step sizes corresponding to each type of feature channel;
[0302] Extracting the quantization step size corresponding to each channel segment from the segment step size set;
[0303] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0304] It should be noted that after further dividing a type of feature channel, each type of feature channel can correspond to multiple different quantization step sizes, which can be saved in the form of a set. At this time, the feature channel differentiation information can be parsed to obtain the segmentation step size set corresponding to each type of feature channel, and then the quantization step size corresponding to each channel segment can be extracted from the segmentation step size set. The segmentation method of different types of feature channels can be different or the same, and this embodiment does not limit this.
[0305] At this time, constructing the quantization precision parameters corresponding to each image residual value according to the quantization step size can be to obtain the feature channel category corresponding to the image residual value, and determine the channel segment to which the image residual value belongs in this type of feature channel, and determine the quantization step size and other parameters used for quantizing the image residual value according to the feature channel category and channel segmentation, so as to obtain the quantization precision parameters corresponding to each image residual value.
[0306] In a specific implementation, when setting the quantization step size, it can be set according to the image texture. In this case, step S3002 of this embodiment may include:
[0307] Extract the quantization step size corresponding to each type of spatial point from the spatial point distinction information;
[0308] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0309] It should be noted that when setting the quantization step size, the spatial points in the image can be divided into multiple categories according to the image texture, and different quantization step sizes can be set for spatial points of different categories. For example: according to the image texture, the spatial points in the image are divided into X categories, and the quantization step size of the j-th category spatial point is scale_j, and the value range of j is [1, X].
[0310] Among them, when classifying spatial points, they can be distinguished according to the complexity of the image texture corresponding to the spatial points. For example, three complexity intervals are set, [0, F1], (F1, F2], (F2, ∞), and then the spatial points are divided into three categories according to the complexity interval to which the image texture complexity corresponding to the spatial points belongs.
[0311] It can be understood that constructing the quantization precision parameters corresponding to each image residual value according to the quantization step corresponding to each type of spatial point can be based on the spatial point category corresponding to the image residual value, obtaining the parameters such as the quantization step used when quantizing the image residual value, thereby obtaining the quantization precision parameters corresponding to each image residual value.
[0312] It should be noted that each spatial point actually contains C (C is the number of feature channels) points. After dividing the spatial points into different categories, the multiple points contained in the spatial points can actually be further divided so that different feature channels in a category of spatial points can correspond to different quantization step sizes. In this case, the spatial point distinction information and the feature channel distinction information can be used in conjunction. In this case, step S3002 of this embodiment may include:
[0313] Read the various types of divided spatial points from the spatial point differentiation information;
[0314] Distinguish information from the feature channels and read the quantization step sizes corresponding to each feature channel in each spatial point;
[0315] The quantization precision parameters corresponding to the respective image residual values are constructed according to the quantization step size.
[0316] It should be noted that, when the multiple points contained in each type of spatial point are further divided according to the different characteristic channels, the points of multiple different characteristic channels corresponding to one type of spatial point can correspond to different quantization step sizes. At this time, the multiple types of spatial points that can be divided can be read from the spatial point distinction information first, and then the quantization step sizes corresponding to each characteristic channel in each type of spatial point can be read.
[0317] For example: divide the spatial points into two categories a and b. At this time, the feature channels contained in a and b are both 1-X. Then the quantization step size corresponding to each feature channel in category a spatial points can be expressed as scale_lista=[scale1, scale2, ..., scaleX], and the quantization step size corresponding to each feature channel in category b spatial points can be expressed as scale_listb=[scale1, scale2, ..., scaleX].
[0318] In a specific implementation, in order to facilitate the encoding device to clarify the specific quantization step setting method, the corresponding flag bit can also be pre-set in the encoding device, and the flag bit is used to determine how to obtain the quantization step length. For example: if the flag bit is set to 0, the quantization step length is determined based on the characteristic channel distinction information; if the flag bit is set to 1, the quantization step length is determined based on the spatial point distinction information; if the flag bit is set to 2, the quantization step length is determined based on the characteristic channel distinction information and the spatial point distinction information.
[0319] It should be noted that this embodiment only describes a method of using spatial point distinction information in conjunction with characteristic channel distinction information. Further division of multiple points contained in the spatial points can also utilize the implementation method for further division of characteristic channels provided in any of the above embodiments, and this embodiment does not limit this.
[0320] Refer to FIG16 , which is a flowchart of a third embodiment of an image encoding method disclosed herein.
[0321] Based on the first embodiment of the image coding method described above, step S500 of the image coding method of this embodiment includes steps S5001 to S5003.
[0322] Step S5001: performing image decoding according to the probability distribution parameters, the quantized residual value and the characteristic channel distinction information to obtain a reconstructed image block and an image bit rate.
[0323] It should be noted that, performing image decoding according to the probability distribution parameters, the quantized residual value and the characteristic channel distinguishing information to obtain the reconstructed image block and the image bit rate can be performed according to the image decoding method provided in any of the above embodiments, performing image decoding according to the probability distribution parameters, the quantized residual value and the characteristic channel distinguishing information to obtain the reconstructed image block, and calculating the image bit rate according to the probability distribution parameters.
[0324] Step S5002: adjusting the characteristic channel distinction information and / or the spatial point distinction information according to the reconstructed image block and the image bit rate.
[0325] It should be noted that the encoding device can also optimize and adjust the parameters used in the encoding process. At this time, adjusting the characteristic channel distinction information according to the reconstructed image block and the image bit rate can be optimizing and adjusting the quantization-related parameters in the characteristic channel distinction information and / or spatial point distinction information according to the reconstructed image block and the image bit rate.
[0326] In a specific implementation, the administrator of the encoding device can determine whether to perform parameter optimization adjustment by modifying the optimal threshold mode flag in the encoding device. If the optimal threshold mode flag value is 0, it indicates that parameter optimization adjustment is not performed; if the optimal threshold mode flag value is 1, it indicates that parameter optimization adjustment is performed.
[0327] For ease of understanding, the present invention will be described in conjunction with FIG17, but this solution is not limited thereto. FIG17 is a schematic diagram of the optimal threshold mode flag. As shown in FIG17, the user of the encoding device can set the optimal threshold mode flag Flag_useRDAQ, and pass the set flag and the relevant threshold or parameter for distinguishing the characteristic channel to the encoding device. The encoding device will then detect whether the optimal threshold mode flag value is 1. If it is 1, the parameter optimization adjustment will be turned on, and the input relevant threshold or parameter will be used as the initial value. Then, multiple iterations will be performed, and the relevant threshold or parameter for distinguishing the characteristic channel obtained after the iteration will be encoded, and the generated image code stream will be transmitted to the decoding device. If it is not 1, the encoding will be directly performed based on the relevant threshold or parameter for distinguishing the characteristic channel passed in, and the encoded image code stream will be transmitted to the decoding device.
[0328] Step S5003: If the current adjustment round is greater than or equal to the preset adjustment round, the probability distribution parameter and the quantized residual value are written into the image code stream.
[0329] It should be noted that the preset adjustment round may be the number of times the characteristic channel distinguishing information is adjusted in advance, and the current adjustment round may be the number of times the characteristic channel distinguishing information has been adjusted.
[0330] It is understood that if the current adjustment round is greater than or equal to the preset adjustment round, the parameter optimization adjustment is complete. At this point, the probability distribution parameters and the quantized residual value can be written into the image bitstream and transmitted to the decoding device. The adjusted feature channel distinction information and / or spatial point distinction information can also be written into the image bitstream.
[0331] If the current adjustment round is less than the preset adjustment round, it means that the parameter optimization adjustment has not yet been completed. Therefore, it is possible to return to the step of constructing the quantization precision parameters corresponding to each feature channel according to the feature channel distinction information.
[0332] For ease of understanding, the present invention is described in conjunction with FIG18 , but this solution is not limited thereto. FIG18 is a parameter optimization flowchart of this embodiment. In FIG18 , the indicator representing the distortion D can be a combination of one or more of multiple indicators such as msssim, vif, fsim, nlpd, iw-ssim, vmaf, and psnr_HVS. X is the relevant threshold or parameter for distinguishing the input feature channel, res_q is the quantized residual value, sigmma is the probability distribution parameter σ, and step_X can be pre-set by the administrator of the encoding device according to actual needs. Meeting the iteration limit can be comparing the current adjustment round with the preset adjustment round. If the current adjustment round is greater than or equal to the preset adjustment round, it is determined that the iteration limit is met; if the current adjustment round is less than the preset adjustment round, it is determined that the iteration limit is not met.
[0333] This embodiment decodes the image based on the probability distribution parameters, the quantized residual values, and the characteristic channel distinguishing information to obtain a reconstructed image block and an image bit rate. The characteristic channel distinguishing information and / or spatial point distinguishing information are adjusted based on the reconstructed image block and the image bit rate. If the current adjustment round is greater than or equal to a preset adjustment round, the probability distribution parameters and the quantized residual values are written into the image bitstream. By performing image decoding and adjusting the characteristic channel distinguishing information and / or spatial point distinguishing information based on the reconstructed image block and the image bit rate, the parameters used in the final encoding are guaranteed to be high-quality solutions, thereby further improving image encoding performance.
[0334] In addition, an embodiment of the present disclosure also proposes a storage medium, on which an image encoding or image decoding program is stored. When the image encoding program is executed by a processor, the steps of the image encoding method described above are implemented; when the image decoding program is executed by a processor, the steps of the image decoding method described above are implemented.
[0335] Refer to FIG. 19 , which is a structural block diagram of the first embodiment of the image decoding apparatus disclosed herein.
[0336] As shown in FIG19 , the image decoding apparatus proposed in the embodiment of the present disclosure includes:
[0337] An entropy decoding module 10 is used to obtain characteristic channel distinguishing information and / or spatial point distinguishing information, and decode the image code stream to obtain a quantized residual value;
[0338] A parameter construction module 20 is configured to construct an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information;
[0339] a dequantization module 30, configured to dequantize the quantized residual value based on the dequantization precision parameter to obtain a reconstructed residual value;
[0340] The synthesis transformation module 40 is used to perform synthesis transformation on the reconstructed residual value to obtain a reconstructed image block.
[0341] For ease of understanding, this is explained here in conjunction with Figure 3 above, but this solution is not limited. In this embodiment, the entropy decoding module can execute the entropy decoding process shown in Figure 3, the parameter construction module and the inverse quantization module execute the inverse quantization process shown in Figure 3, and the synthesis transformation module executes the synthesis transformation process in Figure 3 through the synthesis transformation network.
[0342] This embodiment obtains quantized residual values by acquiring characteristic channel distinguishing information and / or spatial point distinguishing information and decoding the image code stream; constructs inverse quantization precision parameters corresponding to each quantized residual value based on the characteristic channel distinguishing information and / or the spatial point distinguishing information; dequantizes the quantized residual values based on the inverse quantization precision parameters to obtain reconstructed residual values; and performs a synthetic transformation on the reconstructed residual values to obtain a reconstructed image block. Because dequantization is performed by constructing corresponding inverse quantization precision parameters based on characteristic channel distinguishing information and / or spatial point distinguishing information, different quantization parameters can be set during the encoding process based on the differences in characteristic channel and spatial point image textures, allowing more important features to incur less quantization loss, thereby improving encoding and decoding performance.
[0343] In a possible implementation of this embodiment, the image code stream includes a first image code stream and a second image code stream, the first image code stream is used to transmit decoding auxiliary information, and the second image code stream is used to transmit residual data;
[0344] The entropy decoding module 10 is further configured to extract coding distribution parameters from the first image code stream; decode the coding distribution parameters to obtain probability distribution parameters; quantize the probability distribution parameters to obtain distribution quantization parameters; and decode the second image code stream based on the distribution quantization parameters to obtain a quantized residual value.
[0345] In a possible implementation of this embodiment, the entropy decoding module 10 is further used to construct a probability quantization parameter based on the feature channel distinction information and / or the spatial point distinction information; and quantize the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.
[0346] In a possible implementation of this embodiment, the entropy decoding module 10 is further configured to decode the first image code stream or the second image code stream to obtain feature channel distinction information and / or spatial point distinction information.
[0347] In a possible implementation of this embodiment, the image code stream further includes a third image code stream, and the third image code stream is used to transmit distinguishing information;
[0348] The entropy decoding module 10 is further configured to decode the third image code stream to obtain feature channel distinction information and / or spatial point distinction information.
[0349] In a possible implementation of this embodiment, the parameter construction module 20 is further used to extract the quantization step corresponding to each type of feature channel from the feature channel distinction information; and construct the inverse quantization precision parameters corresponding to each quantization residual value according to the quantization step.
[0350] In a possible implementation of this embodiment, the parameter construction module 20 is further used to extract the quantization step corresponding to each type of spatial point from the spatial point distinction information; and construct the inverse quantization precision parameters corresponding to each quantized residual value according to the quantization step.
[0351] In a possible implementation of this embodiment, the parameter construction module 20 is further used to read the various types of divided spatial points from the spatial point distinction information; read the quantization step corresponding to each type of characteristic channel in each type of spatial point from the characteristic channel distinction information; and construct the inverse quantization precision parameters corresponding to each quantized residual value according to the quantization step.
[0352] In a possible implementation of this embodiment, the parameter construction module 20 is also used to decode the image code stream to obtain probability distribution parameters; extract parameter interval thresholds from the feature channel distinction information; and construct inverse quantization precision parameters corresponding to each quantization residual value based on the probability distribution parameters, the quantization step size, and the parameter interval threshold.
[0353] In a possible implementation of this embodiment, the parameter construction module 20 is further used to extract the image block information and parameter setting rules corresponding to each feature channel from the feature channel distinction information; decode the image code stream to obtain probability distribution parameters; calculate the mean value of the distribution parameters corresponding to each image block information based on the probability distribution parameters; match the mean value of the distribution parameters with the parameter setting rules to obtain the quantization step size corresponding to each image block information; and construct the inverse quantization precision parameters corresponding to each quantization residual value based on the quantization step size.
[0354] In a possible implementation of this embodiment, the parameter construction module 20 is also used to parse the feature channel distinction information to obtain a set of segmented step sizes corresponding to each type of feature channel; extract the quantization step size corresponding to each channel segment from the segmented step size set; and construct an inverse quantization precision parameter corresponding to each quantized residual value based on the quantization step size.
[0355] Refer to FIG. 20 , which is a structural block diagram of the first embodiment of the image encoding device disclosed herein.
[0356] As shown in FIG20 , the image encoding apparatus proposed in the embodiment of the present disclosure includes:
[0357] An analysis and transformation module 100 is used to perform analysis and transformation processing on the image block to be processed to obtain image features;
[0358] The residual calculation module 200 is used to perform residual calculation on the image features to obtain image residual values and probability distribution parameters;
[0359] A parameter construction module 300 is used to construct quantization precision parameters corresponding to each characteristic channel according to characteristic channel distinction information and / or spatial point distinction information;
[0360] A quantization module 400 is configured to perform quantization processing on the image residual value based on the quantization precision parameter to obtain a quantized residual value;
[0361] The entropy coding module 500 is configured to write the probability distribution parameters and the quantized residual value into an image code stream.
[0362] For ease of understanding, this is explained here in conjunction with Figure 3 above, but this solution is not limited. In this embodiment, the analysis transformation module can execute the analysis transformation process shown in Figure 3 through the analysis transformation network, the residual calculation module, the parameter construction module and the quantization module execute the quantization process shown in Figure 3, and the entropy coding module executes the entropy coding process as shown in Figure 3.
[0363] This embodiment analyzes and transforms the image blocks to be processed to obtain image features; performs residual calculation on the image features to obtain image residual values and probability distribution parameters; constructs quantization precision parameters corresponding to each image residual value based on feature channel differentiation information and / or spatial point differentiation information; quantizes the image residual values based on the quantization precision parameters to obtain quantized residual values; and writes the probability distribution parameters and quantized residual values into the image bitstream. Because different quantization parameters can be set during the encoding process based on differences in feature channel and spatial point image texture, more important features incur less quantization loss, thereby improving encoding and decoding performance.
[0364] In a possible implementation of this embodiment, the image code stream includes a first image code stream and a second image code stream, the first image code stream is used to transmit decoding auxiliary information, and the second image code stream is used to transmit residual data;
[0365] The entropy coding module 500 is further configured to write the probability distribution parameters into the first image code stream; construct a distribution quantization parameter according to the probability distribution parameters; and write the quantized residual value into the second image code stream based on the distribution quantization parameter.
[0366] In a possible implementation of this embodiment, the entropy coding module 500 is further used to construct a probability quantization parameter based on the feature channel distinction information and / or spatial point distinction information; and quantize the probability distribution parameter based on the probability quantization parameter to obtain a distribution quantization parameter.
[0367] In a possible implementation of this embodiment, the entropy coding module 500 is further configured to write the feature channel distinguishing information and / or the spatial point distinguishing information into the first image code stream or the second image code stream.
[0368] In a possible implementation of this embodiment, the image code stream further includes a third image code stream, and the third image code stream is used to transmit distinguishing information;
[0369] The entropy coding module 500 is further configured to write the feature channel distinguishing information and / or the spatial point distinguishing information into the third image code stream.
[0370] In a possible implementation of this embodiment, the parameter construction module 300 is further used to extract the quantization step corresponding to each type of feature channel from the feature channel distinction information; and construct the quantization precision parameters corresponding to each image residual value according to the quantization step.
[0371] In a possible implementation of this embodiment, the parameter construction module 300 is also used to extract a parameter interval threshold from the feature channel distinction information; and construct quantization accuracy parameters corresponding to each image residual value based on the probability distribution parameter, the parameter interval threshold and the quantization step size.
[0372] In a possible implementation of this embodiment, the parameter construction module 300 is also used to parse the feature channel distinction information to obtain a set of segmented step sizes corresponding to each type of feature channel; extract the quantization step size corresponding to each channel segment from the segmented step size set; and construct the quantization accuracy parameters corresponding to each image residual value according to the quantization step size.
[0373] In a possible implementation of this embodiment, the parameter construction module 300 is further used to extract the quantization step corresponding to each type of spatial point from the spatial point distinction information; and construct the quantization precision parameters corresponding to each image residual value according to the quantization step.
[0374] In a possible implementation of this embodiment, the parameter construction module 300 is also used to read the various types of divided spatial points from the spatial point distinction information; read the quantization step corresponding to each type of feature channel in each type of spatial point from the feature channel distinction information; and construct the quantization accuracy parameters corresponding to each image residual value according to the quantization step.
[0375] In a possible implementation of this embodiment, the entropy coding module 500 is further used to perform image decoding based on the probability distribution parameters, the quantized residual value and the characteristic channel distinguishing information to obtain a reconstructed image block and an image bit rate; adjust the characteristic channel distinguishing information and / or the spatial point distinguishing information based on the reconstructed image block and the image bit rate; if the current adjustment round is greater than or equal to the preset adjustment round, write the probability distribution parameters and the quantized residual value into the image code stream.
[0376] In a possible implementation of this embodiment, the entropy coding module 500 is further configured to return to the step of constructing quantization precision parameters corresponding to each feature channel according to the feature channel distinction information if the current adjustment round is less than the preset adjustment round.
[0377] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present disclosure. In specific applications, those skilled in the art can make settings as needed, and the present disclosure does not impose any restrictions on this.
[0378] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present disclosure. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no restrictions are imposed here.
[0379] In addition, for technical details not fully described in this embodiment, please refer to the image encoding or image decoding method provided in any embodiment of the present disclosure, and will not be repeated here.
[0380] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.
[0381] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.
[0382] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, or by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.
[0383] The above are only preferred embodiments of the present disclosure and are not intended to limit the patent scope of the present disclosure. Any equivalent structure or equivalent process transformation made using the contents of the present disclosure and the drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present disclosure.
Claims
1. An image decoding method, It is characterized in that The image decoding method comprises the following steps: Acquire characteristic channel distinguishing information and / or spatial point distinguishing information, and decode the image code stream to obtain a quantized residual value; Constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information; For any quantized residual value, dequantize the quantized residual value based on the dequantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; The reconstructed residual value is subjected to a synthetic transformation to obtain a reconstructed image block.
2. The image decoding method according to claim 1, It is characterized in that The step of decoding the image code stream to obtain a quantized residual value includes: Extracting coding profile parameters from the first image code stream; Decoding the coded distribution parameters to obtain probability distribution parameters; quantizing the probability distribution parameters to obtain distribution quantization parameters; The second image code stream is decoded based on the distributed quantization parameter to obtain the quantized residual value.
3. The image decoding method according to claim 2, It is characterized in that The step of quantizing the probability distribution parameter to obtain the distribution quantization parameter comprises: Constructing a probability quantization parameter according to the characteristic channel distinguishing information and / or the spatial point distinguishing information; The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
4. The image decoding method according to claim 3, It is characterized in that The step of constructing a probability quantization parameter according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Constructing the probability quantization parameter according to the characteristic channel or spatial point corresponding to the probability distribution parameter; Among them, different feature channels or spatial points correspond to different probability quantization parameters.
5. The image decoding method according to claim 1, It is characterized in that The step of obtaining characteristic channel distinguishing information and / or spatial point distinguishing information comprises: The image code stream is decoded to obtain feature channel distinction information and / or space point distinction information.
6. The image decoding method according to claim 1, It is characterized in that The step of constructing the inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Extracting the quantization step length corresponding to each type of feature channel from the feature channel distinction information; The inverse quantization precision parameters corresponding to the quantized residual values are constructed according to the quantization step size.
7. The image decoding method according to claim 6, It is characterized in that Different categories of feature channels correspond to different quantization steps.
8. The image decoding method according to claim 1, It is characterized in that The step of constructing the inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Extracting the quantization step length corresponding to each type of spatial point from the spatial point distinction information; The inverse quantization precision parameters corresponding to the quantized residual values are constructed according to the quantization step size.
9. The image decoding method according to claim 8, It is characterized in that Different categories of spatial points correspond to different quantization steps.
10. The image decoding method according to claim 1, It is characterized in that The step of constructing the inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Reading the various types of divided spatial points from the spatial point distinction information; Reading the quantization step lengths corresponding to each type of characteristic channel in each type of spatial point from the characteristic channel distinction information; The inverse quantization precision parameters corresponding to the quantized residual values are constructed according to the quantization step size.
11. The image decoding method according to claim 6, It is characterized in that The step of constructing the inverse quantization precision parameters corresponding to the quantized residual values according to the quantization step size comprises: Decode the image code stream to obtain probability distribution parameters; Extracting parameter interval thresholds from the characteristic channel distinction information; The inverse quantization precision parameters corresponding to each quantization residual value are constructed according to the probability distribution parameter, the quantization step size and the parameter interval threshold.
12. The image decoding method according to claim 1, It is characterized in that The step of constructing the inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Parsing the characteristic channel distinction information, For any quantized residual value, a segmented step size set corresponding to each type of feature channel is obtained, wherein the segmented step size set corresponding to one type of feature channel includes a plurality of different quantized step sizes; Extracting the quantization step corresponding to the channel segment to which the quantized residual value belongs from the segment step set corresponding to one type of feature channel; The inverse quantization precision parameters corresponding to the quantization residual values are constructed according to the quantization step size.
13. The image decoding method according to claim 1, It is characterized in that The method further comprises: For any quantized residual value, obtain the characteristic channel category corresponding to the quantized residual value, and determine the channel segment to which the quantized residual value belongs in this type of characteristic channel, determine the quantization step size corresponding to the quantized residual value according to the characteristic channel category and the channel segment, and determine the inverse quantization precision parameter corresponding to the quantized residual value according to the quantization step size; wherein, a segmentation step size set corresponding to a type of characteristic channel includes multiple different quantization step sizes, a type of characteristic channel is divided into multiple channel segments, and one channel segment corresponds to one quantization step size.
14. The image decoding method according to claim 1, It is characterized in that The step of constructing an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Determine the characteristic channel category and / or the spatial point category corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point information; Determining, according to the feature channel category and / or the spatial point category, an inverse quantization precision parameter corresponding to each quantized residual value; Different categories of feature channels and / or spatial point categories are quantized with different precisions.
15. A method for encoding an image, It is characterized in that The image encoding method comprises the following steps: Perform analysis and transformation on the image block to be processed to obtain image features; Performing residual calculation on the image features to obtain image residual values and probability distribution parameters; Constructing quantization precision parameters corresponding to the residual values of each image according to the feature channel distinction information and / or the spatial point distinction information; Quantizing the image residual value based on the quantization precision parameter to obtain a quantized residual value; The probability distribution parameter and the quantized residual value are written into an image code stream.
16. The image encoding method according to claim 15, It is characterized in that The step of writing the probability distribution parameter and the quantized residual value into the image code stream comprises: Writing the probability distribution parameters into the first image code stream; Constructing distribution quantization parameters according to the probability distribution parameters; The quantized residual value is written into a second image code stream based on the distributed quantization parameter.
17. The image encoding method according to claim 15, It is characterized in that The step of constructing a distribution quantization parameter according to the probability distribution parameter comprises: Constructing a probability quantization parameter according to the characteristic channel distinguishing information and / or the spatial point distinguishing information; The probability distribution parameter is quantized based on the probability quantization parameter to obtain a distribution quantization parameter.
18. The image encoding method according to claim 17, It is characterized in that The step of constructing a probability quantization parameter according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Constructing the probability quantization parameter according to the characteristic channel or spatial point corresponding to the probability distribution parameter; Among them, different feature channels or spatial points correspond to different probability quantization parameters.
19. The image encoding method according to claim 15, It is characterized in that The image encoding method further comprises: The feature channel distinguishing information and / or the space point distinguishing information are written into the image code stream.
20. The image encoding method according to claim 15, It is characterized in that The step of constructing the quantization precision parameters corresponding to the residual values of each image respectively according to the characteristic channel distinction information and / or the spatial point distinction information comprises: Extract the quantization step size corresponding to each type of feature channel from the feature channel distinction information; The quantization precision parameters corresponding to the residual values of each image are constructed according to the quantization step size.
21. The image encoding method according to claim 20, It is characterized in that The image encoding method further comprises: Divide the feature channels into multiple categories; For each type of feature channel, determine the quantization step size corresponding to the feature channel; Among them, different categories of feature channels correspond to different quantization steps.
22. The image encoding method according to claim 20, It is characterized in that The step of constructing the quantization precision parameters corresponding to the image residual values according to the quantization step size comprises: Extracting parameter interval thresholds from the characteristic channel distinction information; The quantization precision parameters corresponding to the respective image residual values are constructed according to the probability distribution parameters, the parameter interval threshold and the quantization step size.
23. The image encoding method according to claim 15, It is characterized in that The step of constructing the quantization precision parameters corresponding to the residual values of each image respectively according to the characteristic channel distinction information and / or the spatial point distinction information comprises: Parsing the characteristic channel distinction information, For any image residual value, Obtaining a segmented step size set corresponding to each type of feature channel, wherein the segmented step size set corresponding to one type of feature channel includes a plurality of different quantization step sizes; Extract the quantization step size corresponding to the channel segment to which the image residual value belongs from the segment step size set corresponding to one type of feature channel; The quantization precision parameters corresponding to the image residual values are constructed according to the quantization step size.
24. The image encoding method according to claim 15, It is characterized in that The step of constructing the quantization precision parameters corresponding to the residual values of each image respectively according to the characteristic channel distinction information and / or the spatial point distinction information comprises: Extract the quantization step size corresponding to each type of spatial point from the spatial point distinction information; The quantization precision parameters corresponding to the residual values of each image are constructed according to the quantization step size.
25. The image encoding method according to claim 24, It is characterized in that The image encoding method further comprises: Divide the spatial points in the image into multiple categories according to the image texture; For each type of spatial point, determine the quantization step size of the type of spatial point; Different categories of spatial points correspond to different quantization steps.
26. The image encoding method according to claim 15, It is characterized in that The step of constructing the quantization precision parameters corresponding to the residual values of each image respectively according to the characteristic channel distinction information and / or the spatial point distinction information comprises: Read various types of divided spatial points from the spatial point distinction information; Distinguish information from the feature channels and read the quantization step lengths corresponding to each feature channel in each spatial point; The quantization precision parameters corresponding to the residual values of each image are constructed according to the quantization step size.
27. The image encoding method according to claim 15, It is characterized in that The method further comprises: For any image residual value, obtain the feature channel category corresponding to the image residual value, and determine the channel segment to which the image residual value belongs in this type of feature channel, determine the quantization step used for quantizing the image residual value according to the feature channel category and the channel segment, and determine the quantization accuracy parameter corresponding to the image residual value according to the quantization step; wherein, a segmentation step set corresponding to a type of feature channel includes multiple different quantization step sizes, a type of feature channel is divided into multiple channel segments, and one channel segment corresponds to one quantization step size.
28. The image encoding method according to claim 15, It is characterized in that The step of constructing quantization precision parameters corresponding to the residual values of each image respectively according to the characteristic channel distinguishing information and / or the spatial point distinguishing information comprises: Determine the feature channel category and / or the space point category corresponding to each image residual value according to the feature channel distinction information and / or the space point information; Determine the quantization precision parameter corresponding to each image residual value according to the feature channel category and / or the space point category. number; Among them, different categories of feature channels and / or quantizations with different precisions are used.
29. An image decoding device, It is characterized in that The image decoding device comprises: An entropy decoding module is used to obtain feature channel distinguishing information and / or spatial point distinguishing information, and decode the image code stream to obtain a quantized residual value; A parameter construction module, used to construct an inverse quantization precision parameter corresponding to each quantized residual value according to the characteristic channel distinction information and / or the spatial point distinction information; A dequantization module, configured to dequantize any quantized residual value based on the dequantization precision parameter corresponding to the quantized residual value to obtain a reconstructed residual value; The synthesis transformation module is used to perform synthesis transformation on the reconstructed residual value to obtain a reconstructed image block.
30. An image encoding device, It is characterized in that The image encoding device comprises: An analysis and transformation module is used to perform analysis and transformation processing on the image block to be processed to obtain image features; A residual calculation module, used to perform residual calculation on the image features to obtain image residual values and probability distribution parameters; A parameter construction module, used to construct quantization precision parameters corresponding to each characteristic channel according to characteristic channel distinction information and / or spatial point distinction information; A quantization module, used for performing quantization processing on the image residual value based on the quantization precision parameter to obtain a quantized residual value; The entropy coding module is used to write the probability distribution parameter and the quantized residual value into the image code stream.
31. A decoding device, It is characterized in that The image decoding device comprises: a processor, a memory, and a decoding program stored in the memory and executable on the processor, wherein the decoding program implements the image decoding method according to any one of claims 1 to 14 when executed by the processor.
32. A coding device, It is characterized in that The encoding device comprises: a processor, a memory, and an encoding program stored in the memory and executable on the processor, wherein the encoding program implements the image encoding method according to any one of claims 15 to 28 when executed by the processor.
33. A computer readable storage medium, It is characterized in that The computer-readable storage medium stores an image decoding program and / or an image encoding program; When the image decoding program is executed, the image decoding method according to any one of claims 1 to 14 is implemented; or, when the image encoding program is executed, the image encoding method according to any one of claims 15 to 28 is implemented.
34. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the image decoding method according to any one of claims 1 to 14 is implemented; or, when the image encoding program is executed, the image encoding method according to any one of claims 15 to 28 is implemented.
