Image coding method, image reconstruction method and device
By setting different quantization degree parameters for different feature channels of image encoding and determining the quantization boundary with nonlinear functions, the problem of code rate and quality imbalance caused by uniform quantization is solved, and efficient image encoding effect is achieved.
Patent Information
- Application Number
- CN202411373344.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2024-11-29
AI Technical Summary
In the existing image encoding technology, uniform quantization makes it difficult to balance the code rate and reconstructed image quality, and it is impossible to effectively realize efficient image encoding.
By setting different quantization degree parameters for different feature channels, non-uniform quantization is achieved, the quantization degree is adjusted to balance the code rate and reconstruct the image quality, and a non-linear function is used to determine the quantization boundary to reduce the computational complexity.
While reducing the bit rate, the image quality is reconstructed, which improves the effect of image encoding and compression efficiency, and avoids high-complexity calculations of non-uniform quantization.
Smart Images

Figure CN119052478A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present specification relate to the field of image processing technology, and in particular to an image encoding method, an image reconstruction method and a device. Background Art
[0002] With the rapid development of computer technology, communication technology and network technology, various videos and images are widely used as important information. Image coding is of great significance for data storage and transmission. Uncompressed images will take up a huge storage space and bring great pressure to transmission. High-quality compressed images play a vital role in network transmission and storage on mobile platforms.
[0003] In the prior art, image coding technology is widely used in various information systems. Image coding technology based on deep learning has developed rapidly in recent years, which can greatly save bandwidth and storage. Specifically, the coding features of the image to be coded can be quantized and encoded to obtain a bitstream file. At present, the coding features are generally quantized by uniform quantization. However, in the above method, the greater the quantization degree, the smaller the bit rate, but the worse the image reconstruction quality. Uniform quantization cannot achieve a balance between the bit rate and the quality of the reconstructed image, resulting in poor image coding effect. Therefore, a better image coding scheme is urgently needed. Summary of the invention
[0004] In view of this, an embodiment of the present specification provides an image encoding method. One or more embodiments of the present specification also relate to an image reconstruction method, an image encoding device, an image reconstruction device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.
[0005] According to a first aspect of an embodiment of this specification, there is provided an image encoding method, including: Acquire an image to be compressed, and input the image to be compressed into an encoder to obtain a coding feature, wherein the coding feature includes at least one feature channel; Respectively determine a quantization degree parameter corresponding to the at least one characteristic channel, wherein the at least one characteristic channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees; The encoding feature is quantized and encoded according to a quantization degree parameter corresponding to the at least one feature channel.
[0006] According to a second aspect of an embodiment of this specification, there is provided an image reconstruction method, comprising: Obtaining a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantizing and encoding based on the above-mentioned image encoding method; Dequantizing the compressed code stream to obtain coding features of the image to be reconstructed; The encoding feature is input into a decoder to obtain a reconstructed image corresponding to the compressed code stream.
[0007] According to a third aspect of an embodiment of this specification, there is provided an image encoding device, including: A first acquisition module is configured to acquire an image to be compressed, and input the image to be compressed into an encoder to obtain a coding feature, wherein the coding feature includes at least one feature channel; A determination module is configured to respectively determine a quantization degree parameter corresponding to the at least one characteristic channel, wherein the at least one characteristic channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees; The encoding module is configured to quantize and encode the encoding feature according to a quantization degree parameter corresponding to the at least one feature channel.
[0008] According to a fourth aspect of an embodiment of this specification, there is provided an image reconstruction device, including: An acquisition module is configured to acquire a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantization encoding based on the above-mentioned image encoding method; A dequantization module, configured to dequantize the compressed code stream to obtain coding features of the image to be reconstructed; The second acquisition module is configured to input the encoding feature into a decoder to obtain a reconstructed image corresponding to the compressed code stream.
[0009] According to a fifth aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned image encoding method or image reconstruction method are implemented.
[0010] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned image encoding method or image reconstruction method are implemented.
[0011] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned image encoding method or image reconstruction method when executed by a processor.
[0012] The embodiments of the present specification provide an image encoding method, which includes obtaining an image to be compressed, inputting the image to be compressed into an encoder to obtain encoding features, wherein the encoding features include at least one feature channel; respectively determining quantization degree parameters corresponding to the at least one feature channel, wherein the at least one feature channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees; and quantizing and encoding the encoding features according to the quantization degree parameters corresponding to the at least one feature channel.
[0013] An embodiment of the present specification realizes that at least one feature channel of the coding feature corresponds to a different quantization degree parameter, and the quantization degree of the corresponding feature channel is adjusted based on the quantization degree parameter to achieve different quantization degrees for different feature channels, thereby balancing the compression bit rate and the reconstructed image quality, so that the coding feature can take into account the reconstructed image quality while reducing the bit rate, thereby ensuring a better image coding effect; and, based on at least one feature channel corresponding to different quantization degree parameters, non-uniform quantization is achieved, thereby avoiding high-complexity calculations of non-uniform quantization, saving the amount of calculation in the image coding process, and improving compression efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1a is a schematic diagram of an AI image encoding provided by an embodiment of this specification; Figure 1b It is a schematic diagram of a processing process of an image compression method based on deep learning provided by an embodiment of this specification; Figure 2 is a flowchart of an image encoding method provided by an embodiment of this specification; Figure 3 is a schematic diagram of quantization degree parameters of multiple feature channels provided by an embodiment of this specification; Figure 4 is a flow chart of an image reconstruction method provided by an embodiment of this specification; Figure 5 is a processing flow chart of an image encoding method provided by an embodiment of this specification; Figure 6 is a structural schematic diagram of an image encoding device provided by an embodiment of this specification; Figure 7 is a structural schematic diagram of an image reconstruction device provided by an embodiment of this specification; Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION
[0015] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.
[0016] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0017] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0018] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0019] First, the terms involved in one or more embodiments of this specification are explained.
[0020] Image coding: is a data processing technology that aims to reduce the size of image files while preserving their visual quality as much as possible. This technology is critical for storing and transmitting digital images because it can save storage space, speed up network transmission and reduce bandwidth usage. Image coding is mainly divided into two types: lossless coding and lossy coding.
[0021] Deep learning: It is a subfield of machine learning. It imitates the working method of the human brain and uses a multi-layer neural network model to learn high-level abstract representations of data. In recent years, this technology has achieved remarkable success in many fields such as image recognition, speech recognition, natural language processing, and recommendation systems, and has become an important driving force for the development of artificial intelligence.
[0022] Quantization: It is an important step in image coding and other signal processing fields. It is the process of converting a continuous signal or value range into a discrete representation for storage or transmission. Quantization involves converting continuous values or high-precision values into discrete values or lower-precision values. In this process, the loss of precision of the original data will lead to loss of information, but it will also reduce the amount of data, which helps compression.
[0023] Uniform quantization: The entire range of the input signal is divided into equally spaced intervals, each interval corresponds to a quantization level, which means that all quantization intervals are the same. This quantization method is simple and easy to implement, but it may not be the best choice in some cases.
[0024] Non-uniform quantization: It is a quantization technique that is widely used in image coding in the field of digital signal processing. The main difference between non-uniform quantization and uniform quantization is the selection of the quantization interval. Non-uniform quantization allows the quantization interval to vary with the signal value, which means that in areas where the signal value is densely distributed, the quantization interval is small; while in areas where the signal value is sparsely distributed, the quantization interval is large. The purpose of this is to better match the probability density function of the signal, thereby reducing the quantization error and improving the compression efficiency.
[0025] Quantization boundary: In the quantization process, it refers to the threshold that determines whether the quantization value is rounded up or down. This threshold is usually set to 0.5, which means whether the decimal part of the quotient of the eigenvalue of the coding feature divided by the quantization step is greater than or equal to 0.5, which determines the final quantization value. Specifically, the quantization boundary determines how the eigenvalue of the coding feature is mapped to the nearest quantization value during the quantization process. For a given quantization step, if the decimal part of the eigenvalue divided by the quantization step is less than 0.5, the quantization value will be closer to 0, the degree of quantization will be greater, and the bit rate will be smaller; if the decimal part is greater than or equal to 0.5, the quantization value will be away from 0, the degree of quantization will be smaller, and the bit rate will be larger.
[0026] Entropy coding: It is an important technology in image coding, which is used to allocate code length according to the probability of feature occurrence, so as to effectively compress data. The goal of entropy coding is to minimize the code length while maintaining the integrity and decodability of information. In image coding, entropy coding is usually performed after other preprocessing steps, such as transform coding and quantization, in order to further reduce redundancy and improve compression efficiency. In information theory, "entropy" refers to the uncertainty measure of information in a message. If the probability of occurrence of various features of a message is unequal, the overall code length can be reduced by assigning shorter codes to features with high probability of occurrence. Entropy coding is designed based on this principle. Common entropy coding methods can include Huffman Coding, Arithmetic Coding, Run-Length Encoding (RLE), etc. In image coding, entropy coding usually occurs after transform coding and quantization coding steps. Entropy coding achieves efficient compression by minimizing the average code length while maintaining the integrity of information. Therefore, it is an important part of image coding technology. In practical applications, the above entropy coding methods can usually be combined to obtain better compression effects.
[0027] It should be noted that in recent years, with the rapid development of information technology, the amount of image data has continued to grow. In order to effectively store and transmit these image data, image coding technology has become particularly important. The rise of artificial intelligence (AI) technology has brought new solutions to the field of image coding. Artificial intelligence (AI) technology, especially deep learning, has made significant progress in the field of image coding. Traditional image coding standards such as JPEG, JPEG 2000, WebP, etc. mainly rely on mathematical transformation (such as discrete cosine transform DCT), quantization and entropy coding technologies, while AI-based coding technology can use deep learning, generative adversarial networks (GAN), convolutional neural networks (CNN) and other algorithms to more intelligently identify redundant information in images by learning the intrinsic structure of data, and achieve more efficient and higher-quality image coding.
[0028] AI-based image coding methods mainly include end-to-end image coding models, such as encoders, which use deep learning models to perform the coding process for feature extraction and quantization. The image coding model can be connected to an image decoding model, such as a decoder, which uses a deep learning model to perform the decoding process for image reconstruction, which can greatly save bandwidth and storage.
[0029] Figure 1aThis is a schematic diagram of an AI image coding provided by an embodiment of this specification. AI image coding technology is a technology that uses artificial intelligence, especially deep learning methods, to improve traditional image coding algorithms. Traditional image coding methods (such as JPEG, PNG, etc.) are mainly based on manually designed transformation and quantization processes, while AI image coding uses a neural network model to adaptively learn how to represent image data more effectively, thereby achieving higher compression efficiency or better reconstruction quality. Figure 1a As shown in the figure, the original image is input into the encoder, and the image features are extracted by the feature extraction layer in the encoder. The feature extraction layer can use a convolutional neural network (CNN) or other types of neural networks, usually involving multiple convolutional layers and pooling layers, gradually reducing the spatial resolution while increasing the depth of the feature map; then the latent representation (that is, the encoded features) of the original image is obtained by the feature representation layer in the encoder. The feature representation layer obtains a low-dimensional latent representation through a series of convolution operations. This representation should contain enough information to reconstruct the original image.
[0030] Quantize the continuous potential representation (i.e., coding features) into discrete values to obtain data distribution, so that the data can be effectively encoded into a bit stream. Common quantization methods include uniform quantization and non-uniform quantization. According to the distribution of quantized data, establish a suitable probability model. For example, an autoregressive model can be used to predict the probability of each symbol. Use entropy coding algorithms (such as arithmetic coding, Huffman coding, range coding, etc.) to further compress and encode the quantized data to generate the final encoded bit stream output.
[0031] Figure 1b is a schematic diagram of a processing process of an image compression method based on deep learning provided by an embodiment of this specification, the image compression method is also an image encoding and decoding method, such as Figure 1b As shown in the figure, the deep learning model includes an encoder, a statistical model and a decoder. The input of the encoder is the image to be compressed, and the output is the coding feature. The coding feature will be quantized to obtain the quantized feature, and then the probability distribution of the quantized features of different feature channels will be obtained through the statistical model for entropy coding to obtain the compressed code stream file, which will be stored. When decoding is required, the compressed code stream file can be entropy decoded first to obtain the quantized feature, and then dequantized to obtain the coding feature, and then the coding feature can be input into the decoder for decoding to obtain the reconstructed image.
[0032] Among them, the dimension of the encoded feature is (C, H, W), where C represents the number of channels, and H and W represent the height and width respectively. In a statistical model, a conditional probability p(y_i | y_{i - 1}) is usually constructed. The conditional probability p(y_i | y_{i - 1}) can indicate the probability distribution of the feature value y_i of the feature channel i given the feature value y_{i - 1} of the feature channel i - 1, where 0 <= i < C, representing the serial number of the feature channel under the C dimension. It describes how the feature value y_i of the current feature channel i depends on the feature value y_{i - 1} of the previous feature channel i - 1, that is, the feature value y of the later feature channel depends on the feature value y of the earlier feature channel. This dependence relationship reflects the information correlation between feature channels. In image data, the information between adjacent feature channels often has a certain correlation, and this correlation can be used to improve probability estimation. The obtained probability p is used for entropy encoding the encoded feature to output the final compressed bitstream file.
[0033] The conditional probability p(y_i | y_{i - 1}) can indicate the probability distribution of the feature value y_i of the feature channel i given the feature value y_{i - 1} of the feature channel i - 1.
[0034] Suppose the encoded feature is a tensor of (C, H, W), where C = 3. Traverse this tensor in the order of feature channels, that is, from the 0th feature channel to the 2nd feature channel. For each feature channel i, first calculate the conditional probability p(y_i | y_{i - 1}) of its feature value y_i, where y_{i - 1} is the feature value of the previous feature channel. These probabilities are used to entropy encode the feature values of each channel to generate the final compressed bitstream.
[0035] Among them, the statistical model can be a context model, which is used to estimate the probability distribution of the current feature given the previous information. In image compression, the context model can help predict the probability of the current feature more accurately, so as to achieve more effective entropy encoding.
[0036] For the image coding method based on deep learning, the key technical points include: quantization, which is used to reduce the redundant information in the image representation and is a key step in reducing the bit rate during the compression process; entropy coding, including Huffman coding, arithmetic coding, etc., which is used to further reduce the bit rate of the coding; adaptive learning, which learns the coding strategy through the training data set so that the model can make adaptive adjustments according to the content of the input image.
[0037] The AI-based image coding technology has been widely applied in many fields such as online video streaming services, mobile communications, satellite imaging, and medical imaging.
[0038] From the above, we can see that since the coding features need to be quantized and entropy encoded to obtain a compressed bitstream, and at the same time the coding features need to be input into the decoder to obtain the reconstructed image, there is a trade-off for the coding features. If the degree of quantization is large, the bitstream is small, but the reconstruction quality is poor. Conversely, the bitstream is large and the reconstruction quality is very good.
[0039] Conventional quantization schemes uniformly quantize the coding features of all feature channels, that is, rounding operations. However, the coding features of the front feature channels need to provide dependencies for the coding features of the back feature channels, so the coding features of the front feature channels are more important. If the coding features of the front feature channels are all quantized to 0, they cannot effectively provide dependencies for the back feature channels, and the function of the statistical model is greatly weakened, resulting in inaccurate probability estimates, resulting in a larger final compressed bitstream file.
[0040] Conventional non-uniform quantization can adaptively determine the quantization degree of the coding features of different feature channels, but it needs to use different quantization steps or matrices to quantize according to the importance of the coefficients; or, generally a large number of iterative calculations are required to determine the quantization boundaries of the coding features, which makes the entire quantization complexity very high. For example, conventional non-uniform quantization can be implemented using the Lloyd-Max algorithm (optimal scalar quantization algorithm). The Lloyd-Max algorithm is a design method for optimal scalar quantization, which aims to find a set of quantization thresholds (that is, quantization boundaries) and a set of quantization values to minimize the mean square error (MSE) of the quantization error. The algorithm requires a large number of iterative calculations, has high complexity, large amount of calculation, and low compression efficiency.
[0041] Therefore, the embodiments of this specification provide an image coding method, an image reconstruction method and an apparatus, which use a non-uniform quantization scheme to distinguish the importance of coding features of different feature channels, that is, the coding features of more important features have a smaller quantization degree, and the coding features of unimportant feature channels have an appropriately increased quantization degree; that is, the coding features of the front feature channels are maintained at a small quantization degree, and the back feature channels are maintained at a larger quantization degree, thereby maintaining the quality of the reconstructed image obtained by inputting the coding features to the decoder while reducing the bit rate. Specifically, a quantization degree parameter is introduced to adjust the quantization degree of different feature channels, thereby achieving a more detailed trade-off between the compression bit rate and the reconstructed image quality, and effectively improving the performance of the end-to-end image coding scheme.
[0042] In this specification, an image encoding method is provided. This specification also relates to an image reconstruction method, an image encoding device, an image reconstruction device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.
[0043] See also Figure 2 , Figure 2 It is a flowchart of an image encoding method provided by an embodiment of this specification, which specifically includes the following steps.
[0044] Step 202: Acquire an image to be compressed, and input the image to be compressed into an encoder to obtain encoding features, wherein the encoding features include at least one feature channel.
[0045] It should be noted that the image encoding method provided in the embodiments of this specification can be applied to the client or to the cloud server. If applied to the client, the image to be compressed can be obtained through the cloud server or other devices to compress the image to be compressed on the client; if applied to the cloud server, the image to be compressed can be obtained locally from the client, other servers or the cloud server to compress the image to be compressed on the cloud server. Among them, the cloud server can be connected to one or more clients through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client here can include but is not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, car-mounted devices, etc. The client can interact with the user through a graphical user interface to upload and manage the image to be compressed, thereby realizing the image encoding method provided in the embodiments of this specification.
[0046] In actual implementation, the image coding method provided in the embodiment of this specification can be a plug-and-play coding module, that is, the embodiment of this specification can be applied to any end-to-end image (or video) coding scheme. Among them, end-to-end image coding is an image coding method implemented by using machine learning, especially deep learning technology. Traditional image coding methods (such as JPEG or PNG) usually include multiple manually designed steps, such as discrete cosine transform (DCT), quantization and entropy coding, etc., while end-to-end image coding uses a neural network model to learn these coding steps, thereby realizing the entire process from the original image to the coded representation and decompressing the image. In end-to-end image coding, there are usually two main neural network components: an encoder and a decoder. The encoder is used to convert the input image to be compressed into a coding feature, and the coding feature is quantized and encoded to obtain a compressed code stream file; the decoder is used to reconstruct the image and restore the content of the original image as much as possible.
[0047] Specifically, the image to be compressed may refer to an acquired image to be compressed. The image to be compressed may be an acquired single image or a video frame image acquired from a video, which is not limited in the embodiments of the present specification.
[0048] In end-to-end image coding, the encoded features output by the encoder can be a feature map, which is usually a three-dimensional tensor whose dimensions are expressed as (C, H, W), where dimension C represents the number of feature channels (Channels), dimension H represents the height (Height), and dimension W represents the width (Width).
[0049] Dimension C can indicate the number of different feature channels in the encoded features. Each feature channel represents different feature information, which can be understood as the result of different types of feature extraction on the input image to be compressed.
[0050] For grayscale images, each pixel has only one value, so it can be considered a single-channel image, in which case C=1.
[0051] For color images (RGB images), there are three feature channels, corresponding to the three color components of red, green, and blue. In this case, C=3.
[0052] For other types of images, such as HSV (Hue, Saturation, Value) color space, there are also three feature channels.
[0053] For encoding features in deep learning, especially in convolutional neural networks (CNNs), after a series of operations such as convolutional layers and pooling layers, the obtained encoding features usually have multiple feature channels, and each feature channel can be regarded as different feature information extracted. For example, an encoding feature may have 64 feature channels (C=64), each of which represents a specific feature pattern.
[0054] For example, assume that the dimension of the encoded feature of an image to be compressed is (C, H, W) = (64, 28, 28), which means that the feature to be encoded has 64 different feature channels, a height of 28 pixels, and a width of 28 pixels.
[0055] In an embodiment of the present specification, an image to be compressed can be obtained, and the image to be compressed can be input into an encoder to obtain an output coding feature. The dimension C of the coding feature can refer to the number of feature channels included. Each feature channel represents a specific feature information. The more feature channels there are, the richer the information contained in the coding feature.
[0056] Step 204: respectively determine the quantization degree parameters corresponding to at least one characteristic channel, wherein at least one characteristic channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees.
[0057] The quantization degree parameter is information corresponding to the quantization degree of the corresponding feature channel. Different quantization degree parameters correspond to different quantization degrees of the corresponding feature channels. The quantization degree parameter can be a function relationship generated based on a nonlinear function, a nonlinear curve, etc. For example, the quantization degree parameter can be a quantization adjustment curve.
[0058] In actual implementation, at least one feature channel corresponds to different quantization degree parameters. Each feature channel can correspond to a quantization degree parameter, and the quantization degree parameters corresponding to any two feature channels are different; or, when at least one feature channel is more than 2, multiple feature channels can also correspond to the same quantization degree parameter. For example, when 3 feature channels are included, the 0th feature channel corresponds to a quantization degree parameter, and the 1st and 2nd feature channels correspond to another quantization degree parameter.
[0059] It should be noted that the coding features of the front feature channel need to provide dependence for the coding features of the back feature channel, so the coding features of the front feature channel are more important, so the quantization degree of the front feature channel can be made smaller, and sufficient feature information can be retained to provide dependence for the back feature channel, so that the quantization degree of the back feature channel is larger, and the code stream of the compressed code stream file is minimized, so as to achieve a balance between the compression code rate and the reconstructed image quality, and improve the image coding effect. Therefore, in the embodiment of the present specification, at least one feature channel corresponds to different quantization degree parameters, and the quantization degree of the back feature channel is controlled to be greater than the quantization degree of the front feature channel.
[0060] In an optional implementation of this embodiment, determining the quantization degree parameter corresponding to at least one feature channel respectively includes: Selecting a target adjustment parameter corresponding to the i-th feature channel from the candidate adjustment parameter set, wherein the target adjustment parameter corresponding to the i-th feature channel is less than the target adjustment parameter corresponding to the i-1-th feature channel; According to the end value of the quantization error and the target adjustment parameter corresponding to the ith feature channel, a quantization degree parameter corresponding to the ith feature channel is generated based on the objective function, wherein the quantization degree parameter corresponding to the ith feature channel corresponds to controlling the quantization degree of the ith feature channel to be greater than the quantization degree of the i-1th feature channel, and the objective function is any set nonlinear function.
[0061] The candidate adjustment parameter set includes a set value of candidate adjustment parameters, and the set value is greater than or equal to the number of feature channels.
[0062] In one possible implementation, the set numerical candidate adjustment parameters can be multiple numerical values that are smaller than the standard quantization boundary. In the implementation, the target adjustment parameter corresponding to the i-th feature channel can be selected from the candidate adjustment parameter set, and the target adjustment parameter corresponding to the i-th feature channel is smaller than the target adjustment parameter corresponding to the i-1-th feature channel, so that the quantization degree parameter corresponding to the i-th feature channel generated based on the target adjustment parameter corresponding to the i-th feature channel can control the quantization degree of the i-th feature channel to be greater than the quantization degree of the i-1-th feature channel.
[0063] Specifically, the set numerical candidate adjustment parameters can be arranged in order from large to small to obtain a candidate adjustment parameter set. At this time, for the 0th feature channel to the i-th feature channel, the corresponding candidate adjustment parameters are selected in order from front to back in the candidate adjustment parameter set as the target adjustment parameters of the corresponding channel.
[0064] Of course, in practical applications, each candidate adjustment parameter in the candidate adjustment parameter set can also be arranged from small to large. In this case, for the 0th feature channel to the i-th feature channel, the corresponding candidate adjustment parameters can be selected in order from back to front in the candidate adjustment parameter set. Alternatively, each candidate adjustment parameter in the candidate adjustment parameter set can also be arranged randomly, as long as the target adjustment parameter corresponding to the i-th feature channel is less than the target adjustment parameter corresponding to the i-1th feature channel. This specification embodiment does not limit this.
[0065] In actual implementation, the quantization error refers to the decimal part of the characteristic value of a certain characteristic channel divided by the quantization step size, which can be used to describe the deviation or gap between the original value and the most recent quantized value. Specifically, this decimal part reflects the position information of the original value in the quantization interval. The end value of the quantization error is also the range of values that the decimal part can take. The end value of the quantization error is generally 0 and 1. Of course, it can also be configured as a range of values between {0, 1} based on the actual business scenario, and the embodiments of this specification do not limit this.
[0066] In addition, the objective function is any set nonlinear function, where a nonlinear function refers to a function that does not conform to a linear relationship. Specifically, if a function f(x) cannot be expressed in the form of f(x)=ax+b, where a and b are constants, then it is nonlinear. Nonlinear functions are a type of function widely used in mathematics and engineering disciplines. The graphs of such functions are not straight lines, and their behaviors and properties are more complex and diverse than linear functions. Nonlinear functions have important applications in many fields, including signal processing, control systems, machine learning, economics, etc. For example, the objective function can be an exponential function, a ln logarithmic function, an advanced expansion of the exponential function to the first or second order, and a polynomial function, etc.
[0067] As an example, the nonlinear function may be an exponential function as shown in the following formula (1): (1) in, a, b, c is the parameter to be determined; t is the decimal point precision of the current feature, t ∈{0,1}, {0,1} is the end value of the quantization error.
[0068] In actual implementation, the quantization degree parameter is used to map the relationship between the adjusted quantization boundary and the standard quantization boundary. The adjusted quantization boundary is the quantization boundary adjusted based on the target adjustment parameter of the current feature channel, and the standard quantization boundary is the quantization boundary generally used in uniform quantization. Among them, the quantization boundary refers to the threshold that determines whether the quantization value is rounded up or down. This threshold is usually set to 0.5, that is, the standard quantization boundary can be 0.5, which means that whether the decimal part of the quotient after the eigenvalue is divided by the quantization step size is greater than or equal to 0.5 determines the final quantization value.
[0069] Specifically, the objective function can be respectively set to be the end value of the quantization error and the target adjustment parameter corresponding to the i-th characteristic channel, and the variable parameters in the objective function can be solved to obtain the quantization degree parameter corresponding to the i-th characteristic channel.
[0070] Using the above example, the quantization degree parameter corresponding to the i-th feature channel can be generated by the following formula (2): (2) Among them, 0 and 1 are the upper and lower values of the quantization error, respectively. Represents the target adjustment parameter corresponding to the i-th feature channel. Through the three equations in the above formula (2), the parameters of the above formula (1) can be solved: a, b, c , obtain the quantization degree parameter corresponding to the i-th feature channel.
[0071] For example, the quantization degree parameter is the quantization adjustment curve. Assuming that the candidate adjustment parameter set is {0.45, 0.4, 0.35, 0.3, 0.25, 0.2, 0.15, 0.1}, for the 0th feature channel, select 0.45 0.4, 0.35 is 0.3, 0.25, is 0.2, 0.15, is 0.1, and the quantization adjustment curves corresponding to the 0th feature channel to the 7th feature channel are obtained respectively, such as Figure 3 As shown, Figure 3It is a schematic diagram of quantization degree parameters of multiple feature channels provided by an embodiment of this specification.
[0072] In the present embodiment, the target adjustment parameter corresponding to the later feature channel is smaller than the target adjustment parameter corresponding to the earlier feature channel, and a nonlinear function is introduced. Based on the quantization degree parameter determined by the target adjustment parameter, the quantization degree corresponding to the later feature channel can be controlled to be greater than that corresponding to the earlier feature channel, so that the earlier feature channel has a smaller quantization degree and a better reconstruction quality, and the later feature channel has a larger quantization degree and a lower bit rate. Introducing a nonlinear function and introducing different quantization degrees for different feature channels can more carefully balance the bit rate and the quality of the reconstructed image.
[0073] In an optional implementation of this embodiment, determining the quantization degree parameter corresponding to at least one feature channel respectively includes: Grouping at least one feature channel to obtain at least one group of feature channels; For each group of feature channels, the corresponding quantization degree parameters are determined.
[0074] It should be noted that some coding features may include a large number of feature channels. If the corresponding quantization degree parameters are determined one by one, the amount of calculation will be large. Therefore, in actual implementation, at least one feature channel can be grouped to obtain at least one group of feature channels, and then for a group of feature channels, a corresponding quantization degree parameter is determined according to the above method, that is, a group of feature channels share a quantization degree parameter and have the same quantization degree. Specifically, the rules for grouping at least one feature channel can be configured based on actual business needs, such as uniform grouping according to set intervals, or non-uniform grouping according to different intervals, etc.
[0075] For example, assume that the dimension C of the coding feature is 320, that is, the coding feature includes 320 feature channels. Assume that the grouping rule is to use a set interval of 31 to obtain the first group of feature channels, and each subsequent group of feature channels is the starting feature channel plus the number of feature channels included in the previous group of feature channels, that is, the feature channels of the coding feature are grouped, and y_{c1}, y_{c2}, y_{c3}, y_{c4}, y_{c5} can be obtained, where y_{c1}={1,…,31}, y_{c2}={32,…,63}, y_{c3}={64,…,127}, y_{c4}={128,…,255}, y_{c5}={256,…,320}. y_{c1}-y_{c5} can select 0.45, 0.4, 0.3, 0.2, and 0.1 as the target adjustment parameters respectively to obtain the corresponding quantization degree parameters so that the quantization degree of y_{c5} is maximized, that is, the features in y_{c5} are more easily quantized to 0.
[0076] In an embodiment of the present specification, at least one feature channel can be grouped, and each feature channel in each group of feature channels shares a quantization degree parameter, that is, the quantization degree of a group of feature channels is the same, and the quantization degree of the feature channels of the current group is greater than the quantization degree of the feature channels of the previous group. This not only introduces different quantization degrees for different feature channels, but also can more finely balance the bit rate and the quality of the reconstructed image, and saves the amount of calculation and improves the compression efficiency.
[0077] Step 206: quantize and encode the encoding feature according to the quantization degree parameter corresponding to at least one feature channel.
[0078] It should be noted that different quantization degree parameters can achieve different quantization degrees. Based on the quantization degree parameter corresponding to at least one feature channel, the coding feature can be quantized and encoded to obtain the compressed code stream of the image to be compressed, so as to achieve a balance between the code rate and the quality of the reconstructed image. In one implementation, the adjustment quantization boundary corresponding to each feature channel can be determined based on the quantization degree parameter, and the feature value of the corresponding feature channel can be quantized based on the adjustment quantization boundary; in another implementation, the decimal part obtained by dividing the feature value by the quantization step size can be mapped and adjusted to the standard quantization boundary to achieve quantization.
[0079] In an optional implementation of this embodiment, quantizing and encoding the encoding feature according to a quantization degree parameter corresponding to at least one feature channel to obtain a compressed code stream of the image to be compressed includes: determining a characteristic value of at least one characteristic channel; quantizing at least one feature channel according to a quantization degree parameter and a feature value corresponding to at least one feature channel to obtain a quantized feature of the at least one feature channel; The quantized features of at least one feature channel are encoded to obtain a compressed code stream of the image to be compressed.
[0080] In actual implementation, the characteristic value of each characteristic channel can be determined from the coding feature output by the encoder, and then at least one characteristic channel is quantized according to the characteristic value and the corresponding quantization degree parameter to obtain the quantized feature of at least one characteristic channel. Then, the quantized feature of at least one characteristic channel is encoded to obtain the compressed code stream of the image to be compressed. Specifically, entropy coding can be used to encode the quantized feature into the compressed code stream of the image to be compressed. Entropy coding is a lossless coding technology used to allocate the coding length according to the probability of the feature appearing. Its purpose is to further compress the quantized data and reduce redundancy.
[0081] It should be noted that quantization is a pre-step of entropy coding. Quantization can discretize continuous signal values to generate a set of quantized features, namely quantization features. Quantization features usually have certain statistical characteristics. For example, the probability of occurrence of some features is relatively high, while the probability of occurrence of other features is relatively low. A conditional probability p(y_i|y_{i - 1}) can be constructed, where 0 <= i < C, for entropy coding of the encoded features to output the final compressed bitstream file.
[0082] In the embodiments of this specification, the quantization features can be used as the input of entropy coding. Entropy coding allocates the coding length according to the probability distribution of the quantization features. Generally, features with a high probability of occurrence will be allocated shorter codes, while features with a low probability of occurrence will be allocated longer codes, which can further reduce the number of bits required for storage or transmission. Inevitably, there will be information loss during the quantization process, but entropy coding can make full use of the statistical characteristics in the remaining information to further compress the data. Even if the quantization features have lost some information, entropy coding can still minimize redundancy by optimizing the coding length. Quantization and entropy coding are two complementary steps in the image coding process, so as to achieve more efficient compression, and can greatly reduce the storage or transmission requirements of data while maintaining a certain image quality.
[0083] In addition, in the fields of image coding and other signal processing, the feature values of each feature channel can usually be divided into two parts: positive numbers and negative numbers. In one possible implementation, a unified quantization method can be adopted without considering the positive or negative of the feature values, and the feature values are quantized according to the same quantization step and quantization boundary. In another possible implementation, considering the statistical characteristics and distribution of positive and negative numbers in the actual application scenarios, the quantization and coding methods for positive and negative numbers may be different. Therefore, they can also be processed separately for positive and negative numbers. For example, for the feature values of positive numbers, the quantization step of positive numbers can be adopted to achieve quantization; for the feature values of negative numbers, the quantization step of negative numbers can be adopted to achieve quantization, etc.; or, the absolute value of the feature value can be processed first, and then the sign of the feature value can be superimposed for quantization operation.
[0084] In an optional implementation manner of this embodiment, at least one feature channel is quantized according to the quantization degree parameter and the feature value corresponding to at least one feature channel to obtain the quantization features of at least one feature channel, including: Determine the quantization error of the i-th feature channel according to the quantization step and the feature value of the i-th feature channel, where i is initially 0; Quantize the i-th feature channel according to the quantization degree parameter corresponding to the i-th feature channel and the quantization error to obtain the quantization features of the i-th feature channel; Let i be incremented by 1, and return to execute the step of determining the quantization error of the i-th feature channel according to the quantization step size and the eigenvalue of the i-th feature channel, until i is equal to the number of feature channels minus one, and obtain the quantization feature corresponding to at least one feature channel.
[0085] The quantization step size is a pre-configured value, such as 2, 6, 9, etc., and can be an integer or a decimal.
[0086] In actual implementation, the decimal part obtained by dividing the characteristic value of the 0th characteristic channel by the quantization step size can be determined as the quantization error of the 0th characteristic channel. Then, according to the quantization degree parameter corresponding to the 0th characteristic channel and the quantization error, it is determined whether the 0th characteristic channel is quantized upward or downward to obtain the quantization feature of the 0th characteristic channel. Then, the quantization feature of the first characteristic channel is determined continuously until the quantization features of each characteristic channel are obtained.
[0087] Among them, down-quantization and up-quantization usually refer to the way of performing quantization operations based on the relationship between the quantization step and the eigenvalue. Specifically, these two quantization methods can determine whether to down-quantize or up-quantize based on the decimal part of the quotient after the eigenvalue is divided by the quantization step. Down-quantization refers to mapping the eigenvalue to the nearest quantization value less than or equal to the eigenvalue, that is, rounding down; up-quantization refers to mapping the eigenvalue to the nearest quantization value greater than or equal to the eigenvalue, that is, rounding up.
[0088] For example, taking the standard quantization boundary of 0.5 as an example, if the decimal part of the eigenvalue divided by the quantization step is greater than or equal to 0.5, then quantize upward. If the decimal part of the eigenvalue divided by the quantization step is less than 0.5, then quantize downward. Assuming the eigenvalue y=7.3, the quantization step s=2, 7.3 / 2=3.65, the quotient is 3, the decimal part is 0.65, and the decimal part is greater than the standard quantization boundary of 0.5, then quantize upward to obtain the quantized feature =(3+1)*2=8.
[0089] In the embodiments of the present specification, whether to perform upward quantization or downward quantization is determined based on the quantization degree parameter corresponding to the i-th feature channel and the quantization error, and the quantization operation on the i-th feature channel is implemented to obtain the quantization characteristics of the i-th feature channel. The quantization degree parameter corresponding to the i-th feature channel determines whether to perform upward quantization or downward quantization, so that different quantization degree parameters can control different quantization degrees, realize adaptive quantization, and effectively control the bit rate.
[0090] In an optional implementation of this embodiment, quantizing the i-th feature channel according to the quantization degree parameter and the quantization error corresponding to the i-th feature channel to obtain the quantized feature of the i-th feature channel includes: Determine an adjusted quantization boundary of the i-th feature channel according to a quantization degree parameter and a standard quantization boundary corresponding to the i-th feature channel, wherein the adjusted quantization boundary of the i-th feature channel is greater than the adjusted quantization boundary of the i-1-th feature channel; The quantization error is rounded according to the adjusted quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
[0091] In actual implementation, the mapping value of the standard quantization boundary on the quantization degree parameter corresponding to the i-th feature channel is determined as the adjusted quantization boundary of the i-th feature channel. Based on the determined adjusted quantization boundary, rounding up or rounding down is performed to obtain the quantization feature of the i-th feature channel.
[0092] Specifically, when the quantization error of the i-th feature channel is greater than the adjusted quantization boundary of the i-th feature channel, it is rounded up, that is, the eigenvalue of the i-th feature channel is mapped to a quantization value greater than or equal to the nearest eigenvalue; when the quantization error of the i-th feature channel is less than or equal to the adjusted quantization boundary of the i-th feature channel, it is rounded down, that is, the eigenvalue of the i-th feature channel is mapped to a quantization value less than or equal to the nearest eigenvalue.
[0093] Using the above example, Figure 3For example, the quantization degree parameter is the quantization adjustment curve. For the 0th feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 0th feature channel is determined to be 0.55, that is, the adjusted quantization boundary corresponding to the 0th feature channel is 0.55; for the 1st feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 1st feature channel is determined to be 0.599, that is, the adjusted quantization boundary corresponding to the 0th feature channel is 0.599; for the 2nd feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 2nd feature channel is determined to be 0.646, that is, the adjusted quantization boundary corresponding to the 2nd feature channel is 0.646; for the 3rd feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 3rd feature channel is determined to be 0.69, that is, the adjusted quantization boundary corresponding to the 3rd feature channel is The boundary is 0.69; for the 4th feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 4th feature channel is determined to be 0.732, that is, the adjusted quantization boundary corresponding to the 4th feature channel is 0.732; for the 5th feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 5th feature channel is determined to be 0.772, that is, the adjusted quantization boundary corresponding to the 5th feature channel is 0.772; for the 6th feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 6th feature channel is determined to be 0.809, that is, the adjusted quantization boundary corresponding to the 6th feature channel is 0.809; for the 7th feature channel, the mapping value of the standard quantization boundary 0.5 on the quantization adjustment curve corresponding to the 7th feature channel is determined to be 0.845, that is, the adjusted quantization boundary corresponding to the 7th feature channel is 0.845.
[0094] From the above, we can see that by adjusting the target adjustment parameters ϵ∈{0.45,…,0.1} of each feature channel, we can get the corresponding adjustment quantization boundaries of each feature channel to be {0.55,…,0.845} respectively. Among them, the later the feature channel, the larger the quantization boundary. In the quantization process, the probability that the quantized value is close to 0 is greater, because 0 occupies fewer bits, which reduces the absolute value of the characters entering the entropy coding, thereby further reducing the bit rate.
[0095] As an example, assuming that for the 7th feature channel, the quantization error calculated based on the eigenvalue and the quantization step size is 0.8, and the standard quantization boundary in uniform quantization is 0.5, since 0.8 is greater than 0.5, the eigenvalue of the feature channel is quantized to 1 in a uniform quantization manner. However, using the solution provided in this embodiment, the adjusted quantization boundary corresponding to the 7th feature channel is 0.845, that is, the quantization boundary obtained by nonlinear mapping is 0.845. Since 0.8 is less than 0.845, the eigenvalue of the 7th feature channel will be quantized to 0 at this time, which increases the probability of quantization to 0, effectively saving bit rate.
[0096] In the embodiment of the present specification, based on the quantization degree parameter and the standard quantization boundary corresponding to the i-th feature channel, the adjustment quantization boundary of the i-th feature channel is determined, and the adjustment quantization boundary of each feature channel is gradually increased, so that the probability that the quantization feature of the later feature channel is close to 0 is greater, that is, the quantization degree of the later feature channel is greater, thereby achieving the purpose of further saving the bit rate. The quantization boundary of different feature channels is changed by introducing the quantization degree parameter, and the operation process of uniform quantization is reused to achieve non-uniform quantization, that is, the quantization degree parameter is introduced to change the quantization degree of different feature channels to achieve adaptive quantization, thereby effectively controlling the bit rate.
[0097] In an optional implementation of this embodiment, quantizing the i-th feature channel according to the quantization degree parameter and the quantization error corresponding to the i-th feature channel to obtain the quantized feature of the i-th feature channel includes: Determining an adjusted quantization error of the i-th feature channel according to a quantization degree parameter and a quantization error corresponding to the i-th feature channel; The adjusted quantization error is rounded according to the standard quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
[0098] It should be noted that in addition to the above-mentioned adjustment of the quantization boundary of each feature channel, the quantization error calculated for the feature channel can also be adjusted based on the quantization degree parameter corresponding to each feature channel, that is, the decimal part obtained by dividing the channel value of the feature channel by the quantization step size is adjusted, and it is mapped to the standard quantization boundary, rounded up or down, and the quantization feature of the i-th feature channel is obtained.
[0099] In actual implementation, the mapping value of the quantization error of the i-th feature channel on the corresponding quantization degree parameter can be determined as the adjusted quantization error of the i-th feature channel. Based on the comparison between the adjusted quantization error and the standard quantization boundary, when the adjusted quantization error of the i-th feature channel is greater than the standard quantization boundary, round up, that is, map the eigenvalue of the i-th feature channel to a quantization value greater than or equal to the nearest eigenvalue; when the adjusted quantization error of the i-th feature channel is less than or equal to the standard quantization boundary, round down, that is, map the eigenvalue of the i-th feature channel to a quantization value less than or equal to the nearest eigenvalue.
[0100] Using the above example, Figure 3 For example, assuming that for the 7th feature channel, the quantization error calculated based on the eigenvalue and the quantization step size is 0.8, and the mapping value corresponding to 0.8 is determined to be 0.382 on the quantization degree parameter corresponding to the 7th feature channel. The adjusted quantization error corresponding to the 7th feature channel is 0.382, which is less than the standard quantization boundary of 0.5. At this time, the eigenvalue of the 7th feature channel is quantized down to 0, which increases the probability of quantization to 0, effectively saving bit rate.
[0101] It should be noted that by introducing a quantization degree parameter to adjust the decimal part of each eigenvalue, uniform quantization is reused to achieve the effect of non-uniform quantization, avoiding the high complexity of conventional non-uniform quantization, and taking into account the quality of reconstructed image while reducing the bit rate.
[0102] The embodiment of this specification provides an image coding method, in which at least one feature channel of the coding feature output by the encoder corresponds to a different quantization degree parameter, and the quantization degree of the corresponding feature channel is adjusted based on the quantization degree parameter to achieve different quantization degrees for different feature channels, thereby balancing the compression code rate and the quality of the reconstructed image, so that the coding feature can take into account the quality of the reconstructed image while reducing the code rate, thereby ensuring a better image coding effect; and, based on at least one feature channel corresponding to different quantization degree parameters, non-uniform quantization is achieved, thereby avoiding the high complexity calculation of non-uniform quantization, saving the amount of calculation in the image coding process, and improving the compression efficiency. The image coding method can be used as a plug-and-play compression module, applicable to any end-to-end image coding scheme, thereby effectively improving the performance of end-to-end image coding.
[0103] See also Figure 4 , Figure 4 It is a flowchart of an image reconstruction method provided by an embodiment of this specification, which specifically includes the following steps.
[0104] Step 402: Obtain a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantizing and encoding based on the above-mentioned image encoding method.
[0105] It should be noted that in the process of quantization encoding based on the fractional part of the eigenvalue of each eigenchannel divided by the quantization step, the decompression step of the decompression usually does not need to directly use the fractional part to restore the original eigenvalue. In the dequantization process, it is necessary to know the quantization step and the quantized features after quantization to restore the original eigenvalue as accurately as possible and obtain the reconstructed image.
[0106] Step 404: Dequantize the compressed code stream to obtain coding features of the image to be reconstructed.
[0107] In actual implementation, in the decompression process, an inverse quantization step is required to restore the original feature value. Specifically, the compressed bitstream can be entropy decoded to decode the quantized quantized features from the compressed bitstream. Then, based on the quantized features and the quantization step size used in the compression process, inverse quantization is performed to obtain the coding features of the target image.
[0108] Among them, the dequantization method can usually adopt center quantization, integer (rounding) dequantization, precise dequantization, etc. Center quantization assumes that the values in each quantization interval are uniformly distributed, and uses the center value of the interval as an approximation; integer (rounding) dequantization assumes that the values in each quantization interval tend to be close to the center of the interval; if the quantization error (that is, the decimal part) is recorded during the quantization process, this information can be used during dequantization to more accurately restore the eigenvalue. Step 406: Input the encoding features into the decoder to obtain a reconstructed image corresponding to the compressed code stream.
[0109] The embodiment of this specification provides an image reconstruction method. In the obtained compressed code stream, different feature channels have different quantization degrees. The feature channels closer to the front have lower quantization degrees, retaining more detail information, and can provide more accurate dependencies for the feature channels closer to the back, so that the reconstructed image obtained by the decoder is more accurate and closer to the original image. Different feature channels have different quantization degrees, which balances the compression bit rate and the quality of the reconstructed image, so that the coding features can take into account the quality of the reconstructed image while reducing the bit rate, ensuring better image coding effects. In addition, the image reconstruction method can be used as a plug-and-play reconstruction module, suitable for any end-to-end image reconstruction scheme, thereby effectively improving the performance of end-to-end image reconstruction.
[0110] The following combination Figure 5 , taking the application of the image encoding method provided in this specification in the content interaction scenario as an example, the image encoding method is further described. Figure 5 This is a processing flow chart of an image encoding method provided by an embodiment of this specification, which specifically includes the following steps.
[0111] Step 502: Acquire an interactive image in the content interaction platform.
[0112] Step 504: Input the interactive image to an encoder to obtain a coding feature, where the coding feature includes at least one feature channel.
[0113] Step 506: Select a target adjustment parameter corresponding to the ith feature channel from the candidate adjustment parameter set, the target adjustment parameter corresponding to the ith feature channel is smaller than the target adjustment parameter corresponding to the i-1th feature channel; generate a quantization degree parameter corresponding to the ith feature channel based on a nonlinear function according to the rounded end value and the target adjustment parameter corresponding to the ith feature channel, the quantization degree parameter corresponding to the ith feature channel is used to control the quantization degree of the ith feature channel to be greater than the quantization degree of the i-1th feature channel.
[0114] Step 508: Determine the eigenvalue of the ith feature channel, and divide the eigenvalue of the ith feature channel by the quantization step size to obtain a decimal part as the quantization error of the ith feature channel.
[0115] Step 510: Determine the adjusted quantization boundary of the i-th feature channel according to the quantization degree parameter and the standard quantization boundary corresponding to the i-th feature channel, and the adjusted quantization boundary of the i-th feature channel is greater than the adjusted quantization boundary of the i-1-th feature channel; round the quantization error according to the adjusted quantization boundary to obtain the quantization feature of the i-th feature channel, and the quantization degree of the i-th feature channel is greater than the quantization degree of the i-th feature channel.
[0116] Step 512: Determine the adjusted quantization error of the i-th feature channel according to the quantization degree parameter and quantization error corresponding to the i-th feature channel; round the adjusted quantization error according to the standard quantization boundary to obtain the quantization feature of the i-th feature channel, and the quantization degree of the i-th feature channel is greater than the quantization degree of the i-th feature channel.
[0117] Step 514: increment i by 1, and return to step 508 until the quantitative features corresponding to each feature channel are obtained.
[0118] Step 516: Perform entropy coding based on the quantized features corresponding to each feature channel to obtain a compressed bit stream.
[0119] Step 518: Obtain a compressed code stream, dequantize the compressed code stream, and obtain the coding features of the target image; input the coding features into a decoder to obtain a reconstructed image corresponding to the compressed code stream.
[0120] The embodiment of this specification provides an image coding method, in which at least one feature channel of the coding feature output by the encoder corresponds to a different quantization degree parameter, and the quantization degree of the corresponding feature channel is adjusted based on the quantization degree parameter to achieve different quantization degrees for different feature channels. The feature channel closer to the front has a lower quantization degree, and more detail information is retained, which can provide more accurate dependence for the feature channel closer to the back, so that the reconstructed image obtained by the decoder is more accurate and closer to the original image. The different quantization degrees of different feature channels balance the compression bit rate and the reconstructed image quality, so that the coding feature can take into account the reconstructed image quality while reducing the bit rate, ensuring a better image coding effect; and, based on at least one feature channel corresponding to different quantization degree parameters, non-uniform quantization is achieved, avoiding the high complexity calculation of non-uniform quantization, saving the calculation amount of the image coding process, and improving the compression efficiency. The image coding and reconstruction scheme can be used as a plug-and-play module, suitable for any end-to-end image coding and reconstruction scheme, thereby effectively improving the performance of end-to-end image coding and reconstruction.
[0121] Corresponding to the above method embodiment, this specification also provides an image encoding device embodiment, Figure 6 FIG. 1 is a schematic diagram of the structure of an image encoding device provided by an embodiment of this specification. Figure 6 As shown, the device comprises: A first obtaining module 602 is configured to obtain an image to be compressed, and input the image to be compressed into an encoder to obtain a coding feature, wherein the coding feature includes at least one feature channel; The determination module 604 is configured to respectively determine a quantization degree parameter corresponding to at least one characteristic channel, wherein at least one characteristic channel corresponds to a different quantization degree parameter, and different quantization degree parameters correspond to controlling different quantization degrees; The encoding module 606 is configured to perform quantization encoding on the encoding feature according to a quantization degree parameter corresponding to at least one feature channel to obtain a compressed code stream of the image to be compressed.
[0122] Optionally, the determination module 604 is further configured to: Selecting a target adjustment parameter corresponding to the i-th feature channel from the candidate adjustment parameter set, wherein the target adjustment parameter corresponding to the i-th feature channel is less than the target adjustment parameter corresponding to the i-1-th feature channel; According to the end value of the quantization error and the target adjustment parameter corresponding to the ith feature channel, a quantization degree parameter corresponding to the ith feature channel is generated based on the objective function, wherein the quantization degree parameter corresponding to the ith feature channel corresponds to controlling the quantization degree of the ith feature channel to be greater than the quantization degree of the i-1th feature channel, and the objective function is any set nonlinear function.
[0123] Optionally, the encoding module 606 is further configured to: determining a characteristic value of at least one characteristic channel; quantizing at least one feature channel according to a quantization degree parameter and a feature value corresponding to at least one feature channel to obtain a quantized feature of the at least one feature channel; The quantized features of at least one feature channel are encoded to obtain a compressed code stream of the image to be compressed.
[0124] Optionally, the encoding module 606 is further configured to: Determine the quantization error of the i-th feature channel according to the quantization step size and the feature value of the i-th feature channel, where i is initially 0; Quantize the i-th feature channel according to the quantization degree parameter and quantization error corresponding to the i-th feature channel to obtain the quantization feature of the i-th feature channel; Let i be incremented by 1, and return to execute the step of determining the quantization error of the i-th feature channel according to the quantization step size and the eigenvalue of the i-th feature channel, until i is equal to the number of feature channels minus one, and obtain the quantization feature corresponding to at least one feature channel.
[0125] Optionally, the encoding module 606 is further configured to: Determine an adjusted quantization boundary of the i-th feature channel according to a quantization degree parameter and a standard quantization boundary corresponding to the i-th feature channel, wherein the adjusted quantization boundary of the i-th feature channel is greater than the adjusted quantization boundary of the i-1-th feature channel; The quantization error is rounded according to the adjusted quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
[0126] Optionally, the encoding module 606 is further configured to: Determining an adjusted quantization error of the i-th feature channel according to a quantization degree parameter and a quantization error corresponding to the i-th feature channel; The adjusted quantization error is rounded according to the standard quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
[0127] Optionally, the determination module 604 is further configured to: Grouping at least one feature channel to obtain at least one group of feature channels; For each group of feature channels, the corresponding quantization degree parameters are determined.
[0128] The embodiment of the present specification provides an image encoding device, including a first acquisition module, a determination module and an encoding module, which controls at least one feature channel of the encoding feature to correspond to different quantization degree parameters, and adjusts the quantization degree of the corresponding feature channel based on the quantization degree parameter to achieve different quantization degrees for different feature channels, thereby balancing the compression bit rate and the reconstructed image quality, so that the encoding feature can take into account the reconstructed image quality while reducing the bit rate, thereby ensuring a better image encoding effect; and, based on at least one feature channel corresponding to different quantization degree parameters, non-uniform quantization is achieved, thereby avoiding high-complexity calculations of non-uniform quantization, saving the amount of calculation in the image encoding process, and improving compression efficiency.
[0129] The above is a schematic scheme of an image coding device of this embodiment. It should be noted that the technical scheme of the image coding device and the technical scheme of the above-mentioned image coding method belong to the same concept, and the details not described in detail in the technical scheme of the image coding device can be referred to the description of the technical scheme of the above-mentioned image coding method.
[0130] Corresponding to the above method embodiment, this specification also provides an image reconstruction device embodiment, Figure 7 FIG. 1 is a schematic diagram of the structure of an image reconstruction device provided by an embodiment of this specification. Figure 7 As shown, the device comprises: The acquisition module 702 is configured to acquire a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantization encoding based on the above-mentioned image encoding method; The dequantization module 704 is configured to dequantize the compressed code stream to obtain the coding features of the image to be reconstructed; The second obtaining module 706 is configured to input the encoding feature into the decoder to obtain a reconstructed image corresponding to the compressed code stream.
[0131] The embodiment of this specification provides an image reconstruction device, including an acquisition module, an inverse quantization module and a second acquisition module. In the compressed code stream obtained, different feature channels have different quantization degrees. The more forward feature channels have lower quantization degrees, and more detail information is retained, which can provide more accurate dependencies for the later feature channels, so that the reconstructed image obtained by the decoder is more accurate and closer to the original image. Different feature channels have different quantization degrees, which balances the compression code rate and the quality of the reconstructed image, so that the coding features can take into account the quality of the reconstructed image while reducing the code rate, ensuring better image coding effects. In addition, the image reconstruction method can be used as a plug-and-play reconstruction module, which is suitable for any end-to-end image reconstruction scheme, thereby effectively improving the performance of end-to-end image reconstruction.
[0132] The above is a schematic scheme of an image reconstruction device of this embodiment. It should be noted that the technical scheme of the image reconstruction device and the technical scheme of the above-mentioned image encoding method and image reconstruction method belong to the same concept, and the details not described in detail in the technical scheme of the image reconstruction device can be referred to the description of the technical scheme of the above-mentioned image encoding method and image reconstruction method.
[0133] Figure 8 The block diagram of a computing device according to an embodiment of the present specification is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and the database 850 is used to store data.
[0134] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).
[0135] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.
[0136] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.
[0137] The processor 820 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned image encoding method or image reconstruction method.
[0138] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned image encoding method or image reconstruction method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned image encoding method or image reconstruction method.
[0139] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned image encoding method or image reconstruction method.
[0140] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned image encoding method or image reconstruction method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned image encoding method or image reconstruction method.
[0141] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned image encoding method or image reconstruction method.
[0142] The above is an illustrative solution of a computer program of this embodiment. It should be noted that the technical solution of the computer program and the technical solution of the above-mentioned image encoding method or image reconstruction method belong to the same concept, and the details not described in detail in the technical solution of the computer program can be referred to the description of the technical solution of the above-mentioned image encoding method or image reconstruction method.
[0143] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0144] Computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. Computer readable media may include: any entity or device capable of carrying computer program codes, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROM), random access memories (RAM), electric carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of computer readable media may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0145] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.
[0146] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0147] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well. This specification is limited only by the claims and their full scope and equivalents.
Claims
1. An image coding method, characterized in that: include: Acquire an image to be compressed, and input the image to be compressed into an encoder to obtain a coding feature, wherein the coding feature includes at least one feature channel; Respectively determine a quantization degree parameter corresponding to the at least one characteristic channel, wherein the at least one characteristic channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees; The encoding feature is quantized and encoded according to a quantization degree parameter corresponding to the at least one feature channel.
2. The image encoding method according to claim 1, characterized in that The respectively determining the quantization degree parameter corresponding to the at least one feature channel comprises: Selecting a target adjustment parameter corresponding to the i-th feature channel from the candidate adjustment parameter set, wherein the target adjustment parameter corresponding to the i-th feature channel is less than the target adjustment parameter corresponding to the i-1-th feature channel; According to the end value of the quantization error and the target adjustment parameter corresponding to the i-th feature channel, a quantization degree parameter corresponding to the i-th feature channel is generated based on the objective function, wherein the quantization degree parameter corresponding to the i-th feature channel corresponds to controlling the quantization degree of the i-th feature channel to be greater than the quantization degree of the i-1-th feature channel, and the objective function is any set nonlinear function.
3. The image encoding method according to claim 1, characterized in that: The step of quantizing and encoding the coding feature according to the quantization degree parameter corresponding to the at least one feature channel includes: determining a characteristic value of the at least one characteristic channel; quantizing the at least one feature channel according to the quantization degree parameter corresponding to the at least one feature channel and the feature value to obtain a quantized feature of the at least one feature channel; The quantized features of the at least one feature channel are encoded.
4. The image encoding method according to claim 3, characterized in that: The step of quantizing the at least one feature channel according to the quantization degree parameter corresponding to the at least one feature channel and the feature value to obtain the quantitative feature of the at least one feature channel includes: Determine a quantization error of the i-th feature channel according to a quantization step size and a feature value of the i-th feature channel, wherein i is initially 0; quantizing the i-th feature channel according to the quantization degree parameter corresponding to the i-th feature channel and the quantization error to obtain the quantization feature of the i-th feature channel; Let i be incremented by 1, and return to execute the step of determining the quantization error of the i-th feature channel according to the quantization step size and the feature value of the i-th feature channel, until i is equal to the number of the feature channels minus one, and obtain the quantitative feature corresponding to the at least one feature channel.
5. The image encoding method according to claim 4, characterized in that: The step of quantizing the i-th feature channel according to the quantization degree parameter corresponding to the i-th feature channel and the quantization error to obtain the quantization feature of the i-th feature channel includes: Determining an adjusted quantization boundary of the i-th feature channel according to a quantization degree parameter and a standard quantization boundary corresponding to the i-th feature channel, wherein the adjusted quantization boundary of the i-th feature channel is greater than the adjusted quantization boundary of the i-1-th feature channel; The quantization error is rounded according to the adjusted quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
6. The image encoding method according to claim 4, characterized in that: The step of quantizing the i-th feature channel according to the quantization degree parameter corresponding to the i-th feature channel and the quantization error to obtain the quantization feature of the i-th feature channel includes: Determining an adjusted quantization error of the i-th feature channel according to a quantization degree parameter corresponding to the i-th feature channel and the quantization error; The adjusted quantization error is rounded according to a standard quantization boundary to obtain the quantization feature of the i-th feature channel, wherein the quantization degree of the i-th feature channel is greater than the quantization degree of the i-1-th feature channel.
7. The image encoding method according to any one of claims 1 to 6, characterized in that: The respectively determining the quantization degree parameter corresponding to the at least one feature channel comprises: Grouping the at least one characteristic channel to obtain at least one group of characteristic channels; For each group of feature channels, the corresponding quantization degree parameters are determined.
8. An image reconstruction method, characterized in that: include: Acquire a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantizing and encoding the image encoding method according to any one of claims 1 to 7; Dequantizing the compressed code stream to obtain coding features of the image to be reconstructed; The encoding feature is input into a decoder to obtain a reconstructed image corresponding to the compressed code stream.
9. An image encoding device, characterized in that: include: A first acquisition module is configured to acquire an image to be compressed, and input the image to be compressed into an encoder to obtain a coding feature, wherein the coding feature includes at least one feature channel; A determination module is configured to respectively determine a quantization degree parameter corresponding to the at least one characteristic channel, wherein the at least one characteristic channel corresponds to different quantization degree parameters, and different quantization degree parameters correspond to controlling different quantization degrees; The encoding module is configured to quantize and encode the encoding feature according to a quantization degree parameter corresponding to the at least one feature channel.
10. An image reconstruction device, characterized in that: include: An acquisition module, configured to acquire a compressed code stream corresponding to the image to be reconstructed, wherein the compressed code stream is obtained by quantizing and encoding the image encoding method according to any one of claims 1 to 7; A dequantization module is configured to dequantize the compressed code stream to obtain coding features of the image to be reconstructed; The second acquisition module is configured to input the encoding feature into a decoder to obtain a reconstructed image corresponding to the compressed code stream.
11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the image encoding method according to any one of claims 1 to 7 or the image reconstruction method according to claim 8 are implemented.
12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, can implement the steps of the image encoding method according to any one of claims 1 to 7 or the image reconstruction method according to claim 8.
13. A computer program product, characterized in that The invention comprises a computer program / instruction, which, when executed by a processor, implements the steps of the image encoding method according to any one of claims 1 to 7 or the image reconstruction method according to claim 8.
Citation Information
Cited By
Image compression method and system based on variable code rate allocation, and storage medium
CN119767013A
Method, device and equipment for real-time compression and reconstruction of electroencephalogram signals in attention glasses
CN120066283A
