Deep learning image compression methods and content-adaptive context extraction methods

Abstract

This invention provides a deep learning image compression method and its content-adaptive context extraction method and system. The content-adaptive context extraction method employs a two-pass encoding approach. After obtaining the latent variables, content-adaptive anchor points are extracted in the first encoding pass. In the second encoding pass, context information is extracted from these anchor points as auxiliary prediction information for encoding the remaining latent variables. The complexity of different regions of the latent variables is assessed using the distribution prediction standard deviation. The anchor point positions of the content-adaptive distribution are extracted through masking operations, and context information is calculated from them for the second encoding pass. The results of the two encoding passes are merged and input into the decoder to obtain the reconstructed image. This invention achieves a more efficient reference anchor point selection scheme, improving overall compression performance with negligible increases in computational complexity.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based end-to-end image compression technology, specifically to a deep learning image compression method and its content adaptive context extraction method, and also provides a corresponding system, computer terminal, and computer-readable storage medium. Background Technology

[0002] Image compression is a fundamental task in signal processing and computer vision. The growth of image communication over the Internet has led to a search for methods with higher compression efficiency and lower coding latency. The goal of image compression is to transform an image from the pixel domain into a compact representation by reducing spatial and statistical redundancy. Traditional image compression methods, such as JPEG, JPEG2000, and BPG, typically use hand-designed modules, including linear transforms, quantization, and entropy coding.

[0003] Today, deep learning-based image processing technologies are demonstrating enormous potential. Increasingly, attempts are emerging to apply deep learning to lossy image compression. The Variational Autoencoder (VAE) architecture, which non-linearly transforms pixels into latent variables and then compresses the quantized latent variables within an entropy model, has been adopted by many subsequent learning-based compression methods. Much research aims to improve compression efficiency by using more complex entropy models or more powerful transform networks. Some studies have even achieved compression performance superior to the current highest-efficiency traditional image compression method, VVC intra-frame coding.

[0004] One of the key aspects of deep learning-based end-to-end image compression is modeling the probability distribution at each location in the latent variables using entropy models. If the probability distribution modeling is accurate enough, the cross-entropy between the predicted and actual latent variable distributions can be further reduced, thereby reducing the number of bits required for entropy encoding. Entropy models typically use a priori models and causal context to assist in the probability prediction and modeling of latent variables.

[0005] For spatial context information, many methods extract it from previously decoded latent variables using a serial autoregressive approach. This serial operation involves frequent data exchanges and multiple calls to the entropy encoding process, resulting in significant time consumption. To address this issue, the two-pass coding checkerboard context model (CCM) solves the problem by using only two parallel encoding passes to reduce the time-consuming nature of the numerous serial operations. In the checkerboard context model, latent variables are spatially divided into two groups in a checkerboard pattern, with one half representing anchor points and the other half representing non-anchor points. The anchor points decoded in the first pass are used to provide contextual information for the non-anchor point encoding in the second pass.

[0006] In CCM, the anchor point selection strategy for latent variables in simple and complex regions is fixed. However, the latent variable distributions in simple regions are similar. Compared to complex regions, non-anchor points in simple regions require fewer anchor points to extract context. The original checkerboard context model cannot well adapt to the different content of latent variables. There is redundancy in anchor point selection in simple regions. Since anchor point encoding can only refer to prior information and cannot use spatial context information, redundant anchor points reduce the overall compression efficiency.

[0007] The search revealed:

[0008] Chinese invention patent application CN114501034A, entitled "Image Compression Method and Medium Based on Discrete Gaussian Mixture Super-Prior and Mask," describes a method for preprocessing an image to be compressed to obtain a preprocessed image; extracting feature maps from the preprocessed image and generating Mask values ​​based on the spatial feature information of the preprocessed image; performing a dot product between the feature maps and the Mask values ​​to obtain latent variable representations; extracting the distribution of the latent variable representations using multiple Gaussian distributions to generate discrete Gaussian mixture super-prior values; quantizing the latent variable representations; and performing entropy coding compression on the quantized latent variable representations based on the super-prior values ​​to obtain the encoded information of the compressed image; and finally, decoding based on the encoded information of the compressed image to obtain a reconstructed image. This method has the following technical problems:

[0009] This image compression method uses an autoregressive context model, which requires extracting the context from the decoded latent variables in the upper left corner. Therefore, each encoding pass can only decode the latent variables at the current position, necessitating a serial decoding process that multiplies the width and height of the latent variables, resulting in very high latency and complexity. Summary of the Invention

[0010] To address the aforementioned shortcomings in the prior art, this invention provides a deep learning image compression method and its content adaptive context extraction method, along with a corresponding system, computer terminal, and computer-readable storage medium.

[0011] According to one aspect of the present invention, a content-adaptive context extraction method for deep learning image compression is provided, characterized in that it includes: a first encoding process and a second encoding process; wherein:

[0012] The first encoding process includes:

[0013] For the input image, after nonlinear transformation to latent variables, the latent variables are divided into simple and complex regions, and a mask M is obtained to record the location of the simple regions. simple Then, the latent variables are quantized to obtain the quantized latent variables.

[0014] Using the mask M simple From quantified latent variables Extracting non-anchor points from simple regions The position, and according to the non-anchor point Find the top-left position (non-anchor point) of each simple region using the index of the location, and set that position as the pseudo-anchor point. Using another mask M pa For the pseudo anchor point Record the location;

[0015] Mask M using a checkerboard pattern ckbd Subtract the mask M from it pa This yields a mask M that adapts to the anchor point positions of the current input image. a Using the mask M a From the quantized latent variables Extracting anchor points The anchor point obtained by entropy decoding in this process The context information is saved for extraction during the second pass of encoding.

[0016] The second encoding process includes:

[0017] Based on the anchor points obtained during the first encoding process Extract contextual information to complete content adaptive context extraction in deep learning image compression.

[0018] Preferably, the latent variable is divided into simple and complex regions, and a mask M is obtained to record the location of the simple regions. simple ,include:

[0019] Obtain the distribution prediction parameters of the latent variables, the distribution prediction parameters including: distribution prediction mean and distribution prediction standard deviation;

[0020] Based on the predicted standard deviation of the distribution, the latent variables are divided into simple and complex regions:

[0021] When the predicted standard deviation is less than a set threshold, the region is determined to be a simple region, and a mask M is used. simple Recording is performed, where the mask M simple A value of 1 indicates that the location belongs to a simple region.

[0022] Otherwise, this area is a complex area.

[0023] Preferably, the step based on the non-anchor point The position index is used to find the top-left position of each simple region (excluding anchor points), including:

[0024] According to the mask M simple Find the non-anchor points in the location of the simple area recorded in the middle. The location distribution, and the use of mask M na Save it, and according to the mask M na The non-anchor points recorded in The index of the position is used to set the top-left position of the non-anchor point in each simple region as the pseudo-anchor point. and the pseudo anchor point The position is set to 1, and a mask M is used. pa Record it;

[0025] and / or

[0026] The checkerboard-patterned mask M ckbd The structure uses 0s and 1s arranged in a checkerboard pattern; the checkerboard pattern mask M... ckbd Subtract the record pseudo anchor point Position mask M pa This yields an anchor point distribution mask that adapts to the anchor point positions of the current input image, which is called mask M. a For the removed pseudo-anchor points For the missing positions, extract the values ​​of the non-anchor positions from the distribution prediction mean of the latent variables and fill them into the missing positions;

[0027] The mask M a With the quantized latent variables By performing element-wise multiplication, the adaptive anchor points required for the second encoding process can be extracted.

[0028] Preferably, in the second encoding process, the anchor points obtained from entropy decoding in the first encoding process are... Extract contextual information, including:

[0029] The context model is constructed using convolutional networks as follows:

[0030]

[0031] Among them, g cm (·) represents the context model. The anchor point is obtained from entropy decoding in the first pass of encoding, μ1 is the predicted mean of the latent variable distribution, and M is the anchor point. simple For mask M simple , records the position of non-anchor points in a simple region, and ⊙ represents the multiplication operation of corresponding elements;

[0032] From the decoded anchor point Content adaptive context information is extracted from the distribution prediction mean of latent variables in simple regions, thus completing the content adaptive context extraction in deep learning image compression.

[0033] According to another aspect of the present invention, a content-adaptive context extraction system for deep learning image compression is provided, comprising: a first-pass encoding module and a second-pass encoding module; wherein:

[0034] The first encoding module includes:

[0035] The region segmentation unit is used to nonlinearly transform the input image to latent variables, then segment the latent variables into simple and complex regions, and thereby obtain a mask M that records the locations of the simple regions. simole Then, the latent variables are quantized to obtain the quantized latent variables.

[0036] Non-anchor point and pseudo-anchor point setting unit, which uses the mask M simple From quantified latent variables Extracting non-anchor points from simple regions The position, and according to the non-anchor point Find the top-left position (non-anchor point) of each simple region using the index of the location, and set that position as the pseudo-anchor point. Using another mask M pa For the pseudo anchor point Record the location;

[0037] Anchor point setting unit, which uses a checkerboard-patterned mask M ckbd Subtract the mask M from it pa This yields a mask M that adapts to the anchor point positions of the current input image. a Using the mask M a From the quantized latent variables Extracting anchor points The anchor point obtained by entropy decoding in this process The context information is saved for extraction during the second pass of encoding.

[0038] The second encoding module includes:

[0039] The context extraction unit is based on the anchor points obtained during the first encoding pass. Extract contextual information to complete content adaptive context extraction in deep learning image compression.

[0040] According to a third aspect of the present invention, a deep learning image compression method is provided, comprising:

[0041] The deep learning-based encoder reads in the input image x and nonlinearly transforms the input image x into latent variables y; the latent variables y are input into the super-prior encoder to obtain the super-prior code stream to be transmitted and then pass through the super-prior decoder to obtain super-prior information containing latent variable structure information.

[0042] The obtained latent variable y is quantized to obtain the quantized latent variable.

[0043] Extract content adaptive context information, and obtain the quantized latent variables based on the content adaptive context information. Calculate the distribution prediction parameters, and apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the quantized latent variable bitstream;

[0044] Entropy decoding is performed based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For the quantized latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression;

[0045] Wherein, the extraction of content adaptive context information adopts the content adaptive context extraction method according to any one of claims 1-5, wherein:

[0046] The first encoding process for the quantized latent variables The regions are divided into simple and complex regions, and the quantized latent variables are... Divided into anchor points pseudo anchor point Non-anchor point Among them, the anchor point The context information extracted during the second encoding process is used to perform probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

[0047] Preferably, any end-to-end image encoder based on deep learning is used to nonlinearly transform the input image x into a latent variable y. The latent variable y is input into a super-prior encoder of any implementation to obtain super-prior information containing latent variable structure information. The latent variable structure information includes parameters (μ, σ) for preliminary prediction of the latent variable, namely the mean μ and standard deviation σ of the Gaussian distribution.

[0048] Preferably, scalar quantization or vector quantization is used to convert the latent variable y from a floating-point number into an integer that can be encoded without loss of entropy, thus completing the quantization operation and obtaining the quantized latent variable.

[0049] Preferably, during the first encoding process, spatial context information is not used; instead, the distribution prediction parameters of the latent variables are obtained through prior information or channel-dimensional context information.

[0050] Preferably, during the second encoding process, based on the anchor point The extracted content adaptive context information is used as the pseudo-anchor point. Non-anchor point Reference information for the probability distribution prediction process.

[0051] Preferably, the anchor point is obtained from the bitstream through entropy decoding based on the distribution prediction parameters. pseudo anchor point and non-anchor points These three parts are added together to obtain the quantified latent variables. Input it into the decoder to obtain the final reconstructed image. Complete image compression.

[0052] According to a fourth aspect of the present invention, a deep learning image compression system is provided, comprising:

[0053] The input image processing module reads in the input image x based on a deep learning encoder, nonlinearly transforms the input image x into latent variables y, and inputs the latent variables y into a super-prior encoder to obtain the super-prior bitstream to be transmitted and then passes it through a super-prior decoder to obtain super-prior information containing latent variable structure information.

[0054] The quantization module is used to perform quantization operations on the obtained latent variable y to obtain the quantized latent variable.

[0055] The context extraction module is used to extract content adaptive context information and obtain the quantized latent variables based on the content adaptive context information. Calculate the distribution prediction parameters, and apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the bitstream of the quantized latent variables; entropy decoding is performed based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For the quantized latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression;

[0056] The context extraction module employs the content adaptive context extraction method described in any one of the above descriptions; wherein:

[0057] The first encoding process for the quantized latent variables The regions are divided into simple and complex regions, and the quantized latent variables are... Divided into anchor points pseudo anchor point Non-anchor point Among them, the anchor point The context information extracted during the second encoding process is used to perform probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

[0058] According to a fifth aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform the content adaptive context extraction method described in any one of the preceding descriptions, or to perform the deep learning image compression method described in any one of the preceding descriptions.

[0059] According to a sixth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the content adaptive context extraction method described in any one of the preceding descriptions, or to perform the deep learning image compression method described in any one of the preceding descriptions.

[0060] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0061] The deep learning image compression method and its content adaptive context extraction method and system provided by this invention employ content adaptive context extraction technology, which can improve the compression performance of deep learning image compression models and only brings a small amount of additional computation, with virtually no increase in encoding and decoding latency.

[0062] The deep learning image compression method and its content adaptive context extraction method and system provided by this invention can use the distribution of latent variables to predict the standard deviation, determine the complexity of the latent variables after the image is transformed by the deep learning encoder, and determine the number of anchor points in simple and complex regions accordingly, thereby eliminating redundant anchor points, improving the utilization rate of context information, and thus improving the overall compression performance.

[0063] The deep learning image compression method and its content adaptive context extraction method and system provided by this invention eliminate redundant anchor points based on the content characteristics of latent variables, thereby improving the position of latent variables that use context information to assist in encoding and enhancing the overall compression efficiency. Attached Figure Description

[0064] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0065] Figure 1 This is a flowchart illustrating the content adaptive context extraction method in one embodiment of the present invention.

[0066] Figure 2 This is a flowchart illustrating the process of extracting adaptive contextual information in a preferred embodiment of the present invention.

[0067] Figure 3 This is a schematic diagram of the decoding process in adaptive context extraction in a preferred embodiment of the present invention.

[0068] Figure 4 This is a schematic diagram of the components of a content adaptive context extraction system in one embodiment of the present invention.

[0069] Figure 5 This is a flowchart illustrating the workflow of a deep learning image compression method in one embodiment of the present invention.

[0070] Figure 6 This is a flowchart illustrating a deep learning image compression method that applies a content-adaptive context extraction method in a preferred embodiment of the present invention.

[0071] Figure 7 This is a schematic diagram of a compression scheme for a deep learning image compression method that applies an adaptive contextual information extraction method in a preferred embodiment of the present invention.

[0072] Figure 8 This is a schematic diagram of the constituent modules of a deep learning image compression system in one embodiment of the present invention.

[0073] Figure 9 A comparison of the bitrate-distortion curves of the end-to-end image compression model using the content adaptive context method provided in the preferred embodiment of the present invention and the original checkerboard context model method; wherein, (a) is the test result on the Kodak dataset, and (b) is the test result on the CLIC dataset. Detailed Implementation

[0074] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0075] One embodiment of the present invention provides a content adaptive context extraction method for end-to-end image compression using deep learning.

[0076] like Figure 1 As shown, the method includes: a first encoding process and a second encoding process; wherein:

[0077] S1, the first pass of encoding, includes:

[0078] S11, For the input image, after nonlinear transformation to latent variables, the latent variables are divided into simple and complex regions, and a mask M is obtained to record the location of the simple regions. simple Then, the latent variables are quantized to obtain the quantized latent variables.

[0079] S12, using mask M simple From quantified latent variables Extracting non-anchor points from simple regions The position, and based on the non-anchor point Find the top-left position (non-anchor point) of each simple region using the index of the location, and set that position as the pseudo-anchor point. Using another mask M pa For pseudo anchor points Record the location;

[0080] S13, using a checkerboard-patterned mask M ckbd Subtract the mask M from it pa This yields the mask M for the anchor point positions of the current input image (i.e., adaptive to the current input image). a Using mask M a From quantified latent variables Extracting anchor points The anchor point obtained by entropy decoding in this process The context information is saved for extraction during the second pass of encoding.

[0081] S2, the second encoding process, includes:

[0082] S21, based on the anchor point obtained during the first encoding process. Extract contextual information to complete content adaptive context extraction in deep learning image compression.

[0083] The technical solution provided by the above embodiments of the present invention will be further described below with reference to a preferred embodiment.

[0084] like Figure 2 As shown, the content-adaptive context extraction method for deep learning image compression provided in this preferred embodiment includes:

[0085] S1, First Encoding Pass:

[0086] S11: For the latent variables to be encoded in the deep learning end-to-end image compression method, a mask M for recording the location of simple regions is obtained based on the prediction of the entropy of the latent variable distribution. simple Then, the latent variables are quantized to obtain the quantized latent variables. Used for subsequent context extraction processes;

[0087] S12: Using mask M simple From quantified latent variables Extracting non-anchor points from simple regions The position, and based on the non-anchor point Find the top-left position of each simple region (excluding anchor points) using the index of the location, and set it as a pseudo-anchor point. And then use another mask M pa For pseudo anchor points Record the location;

[0088] S13: Mask M using a checkerboard pattern ckbd Subtract the mask M from it pa This yields a mask M that adapts to the anchor point positions of the current input image. a ; based on the mask M at the anchor point position a From latent variables Extracting anchor points And it does not use spatial context to entropy encode it, anchor point This will be saved for context information extraction in the subsequent second pass of encoding;

[0089] S2, the second encoding process:

[0090] S21: Using a simple convolutional network to create content-adaptive anchors Extracting context information completes content adaptive context extraction in deep learning image compression; since the distribution of anchor points is not fixed at this time, in a preferred embodiment, ordinary convolution operations can be used for context extraction.

[0091] In a preferred embodiment of S11, the latent variables are divided into simple and complex regions (prediction of complexity distribution), and a mask M recording the location of the simple regions is obtained accordingly. simple ,include:

[0092] (can be based on prior information) The distribution prediction parameters of the latent variables are obtained, including the distribution prediction mean and the distribution prediction standard deviation.

[0093] When the standard deviation of the distribution prediction is less than the set threshold, the region is determined to be a simple region, and a mask M is used. simple Recording is performed, where the mask M simple A value of 1 indicates that the location belongs to a simple region.

[0094] In a preferred embodiment of S12, according to the mask M simple Find the non-anchor points in the location of the simple area recorded in the middle. The location distribution is determined using a mask M. na Save it, and according to the mask M na Simple regions without anchor points recorded in the middle The position of the index is used to set the top-left position of each simple region (excluding the anchor point) as a pseudo-anchor point. The position of the pseudo anchor point is set to 1, and a mask M is used. pa Record it.

[0095] In a preferred embodiment of S13, the checkerboard-patterned mask M ckbd The structure uses 0s and 1s arranged in a checkerboard pattern; the checkerboard-patterned mask M... ckbd Subtract the mask M that records the pseudo-anchor point position pa This yields an anchor point distribution mask that adapts to the anchor point positions of the current input image, which is called mask M. a ; and fill the gaps in the removed pseudo-anchor points with the predicted mean of the distribution obtained during the first encoding pass (the predicted mean information of neighboring non-anchor points), mask M. a With quantified latent variables Element-wise multiplication yields content-adaptive anchor points. These are then entropy-encoded in the first pass and used in the context extraction process of the second pass.

[0096] The decoding process in adaptive context extraction, such as Figure 3 As shown.

[0097] In a preferred embodiment of S21, the context model is constructed using a convolutional network as follows:

[0098]

[0099] Among them, g cm (·) represents the context model. The anchor point is obtained from entropy decoding in the first pass of encoding, μ1 is the predicted mean of the latent variable distribution, and M is the anchor point. simple For mask M simple , records the position of non-anchor points in a simple region, and ⊙ represents the multiplication operation of corresponding elements;

[0100] From the decoded anchor point Contextual information is extracted from the mean of the distribution predictions of non-anchor points in simple regions during the first encoding pass, completing the content-adaptive context extraction in deep learning image compression. This content-adaptive context is then used for the probability distribution prediction process during the second encoding pass.

[0101] One embodiment of the present invention provides a content-adaptive context extraction system for deep learning image compression.

[0102] like Figure 4 As shown, the content adaptive context extraction system provided in this embodiment includes: a first-pass encoding module and a second-pass encoding module; wherein:

[0103] The first pass of the encoding module includes:

[0104] The region segmentation unit is used to nonlinearly transform the input image to latent variables, then segment the latent variables into simple and complex regions, and thereby obtain a mask M that records the locations of the simple regions. simple Then, the latent variables are quantized to obtain the quantized latent variables.

[0105] Non-anchor point and pseudo-anchor point setting unit, which uses mask M simple From quantified latent variables Extracting non-anchor points from simple regions The position, and based on the non-anchor point Find the top-left position (non-anchor point) of each simple region using the index of the location, and set that position as the pseudo-anchor point. Using another mask M pa For pseudo anchor points Record the location;

[0106] Anchor point setting unit, which uses a checkerboard-patterned mask M ckbd Subtract the mask M from it pa This yields the mask M for the anchor point positions of the current input image (i.e., adaptive to the current input image). a Using mask M a From quantified latent variables Extracting anchor points The anchor point obtained by entropy decoding in this process The context information is saved for extraction during the second pass of encoding.

[0107] The second encoding module includes:

[0108] The context extraction unit is based on the anchor points obtained during the first encoding pass. Extract contextual information to complete content adaptive context extraction in deep learning image compression.

[0109] It should be noted that the steps in the content adaptive context extraction method provided by the present invention can be implemented using the corresponding modules, devices, units, etc. in the content adaptive context extraction system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0110] One embodiment of the present invention provides a deep learning end-to-end image compression method.

[0111] like Figure 5 As shown, this deep learning end-to-end image compression method includes:

[0112] M1, a deep learning-based encoder, reads in the input image x and nonlinearly transforms the input image x into latent variables y; the latent variables y are input into the super-prior encoder to obtain the super-prior code stream to be transmitted and then pass through the super-prior decoder to obtain the super-prior information containing the latent variable structure information.

[0113] M2, performs a quantization operation on the obtained latent variable y to obtain the quantized latent variable. The quantified latent variable The floating-point number is converted into an integer, which is then used as the data that needs to be entropy encoded.

[0114] M3 extracts content-adaptive context information and obtains quantized latent variables based on this information. Calculate the distribution prediction parameters, and then apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the quantized latent variable bitstream;

[0115] M4 performs entropy decoding based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For quantified latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression;

[0116] The extraction of content adaptive context information employs any one of the content adaptive context extraction methods in the above embodiments of the present invention, or any one of the content adaptive context extraction systems in the above embodiments of the present invention, wherein:

[0117] The first encoding process involves the quantized latent variables. Divide the region into simple and complex regions, and then quantify the latent variables. Divided into anchor points pseudo anchor point Non-anchor point Among them, anchor points This information is used to extract context information during the second encoding pass. The extracted context information is then used for probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

[0118] The technical solution provided by the above embodiments of the present invention will be further described below with reference to a preferred embodiment.

[0119] like Figure 6 As shown, the deep learning end-to-end image compression method provided in this preferred embodiment includes:

[0120] M1: A deep learning-based encoder reads in the input image x and nonlinearly transforms it into latent variables y. The latent variables y are then input into a super-prior encoder to obtain the super-prior bitstream to be transmitted, and then reconstructed by a super-prior decoder to obtain super-prior information containing the latent variable structure information.

[0121] M2: Quantize the latent variable y obtained from the nonlinear transformation to obtain the quantized latent variable. The quantized latent variables are changed from floating-point numbers to integers and are used as data that needs to be entropy encoded.

[0122] M3: Applies a two-pass encoding method based on quantized latent variables. Methods for extracting contextual information, which adaptively obtain quantized latent variables based on content and contextual information. Calculate the distribution prediction parameters, and then apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the quantized latent variable bitstream. The first encoding pass encodes the quantized latent variables... Divide the region into simple and complex regions, and then quantify the latent variables. Divided into anchor points pseudo anchor point Non-anchor point Among them, anchor points This information is used to extract context information during the second encoding pass. The extracted context information is then used for probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

[0123] M4: Based on the distribution prediction parameters, the anchor point can be obtained from the bitstream through entropy decoding. non-anchor point and pseudo-anchors Combining these three parts yields the quantified latent variables. This is input into the decoder to obtain the final reconstructed image.

[0124] In a preferred embodiment of M1, any deep learning-based end-to-end image encoder can be used to nonlinearly transform the input image into latent variables y. The latent variables y are input into a hyperprior encoder / decoder of any implementation to obtain hyperprior information. This hyperprior information includes the parameters (μ, σ) for the initial prediction of the latent variables, namely the mean μ and standard deviation σ of a Gaussian distribution.

[0125] In a preferred embodiment of M2, different quantization methods such as scalar quantization or vector quantization can be used to convert the latent variables from floating-point numbers into integers that can be encoded without loss of entropy, thus completing the quantization operation and obtaining the quantized latent variables.

[0126] In a preferred embodiment of M3, during the first encoding pass, spatial context information is not used; instead, the distribution prediction parameters of the latent variables are obtained through prior information or channel-dimensional context information.

[0127] In a preferred embodiment of M3, during the second encoding process, based on anchor points... The extracted content uses adaptive contextual information as pseudo-anchor points. Non-anchor point Reference information for the probability distribution prediction process.

[0128] Furthermore, in M3, during the probability prediction distribution prediction process in entropy coding, the aforementioned adaptive context extraction method is used to calculate the spatial context. Latent variables are divided into anchor points. pseudo anchor point and non-anchor points Three parts. Contextual information is extracted from the decoded latent variables and used as anchor points. This information serves as a reference for the probability distribution prediction process of the remaining latent variables. Using prior information and contextual information from a non-spatial dimension (an optional input to the distribution prediction parameters obtained in the first encoding pass), the latent variables are divided into simple and complex regions. Redundant anchor points are removed from the simple regions and treated as pseudo-anchor points. This allows for the use of fewer reference points in simple areas, while a denser distribution of reference points with a checkerboard pattern is used in complex areas.

[0129] In a preferred embodiment of M4, both the encoder and decoder can perform the above-described probability distribution prediction process, and obtain the anchor point from the bitstream through entropy decoding based on the predicted probability distribution parameters. pseudo anchor point and non-anchor points These three parts. Adding and combining these three parts yields the quantified latent variables. Input it into the decoder to obtain the final reconstructed image.

[0130] like Figure 7 The diagram illustrates an end-to-end deep learning image compression scheme based on the content-adaptive context information extraction method of this invention. In this scheme, for the image to be encoded and compressed, after nonlinear transformation into latent variables by the encoder, the latent variables that require entropy encoding transmission after quantization are encoded in two passes. The result of the first pass is used to extract content-adaptive context information to assist in the entropy encoding process of the remaining latent variables. Finally, the different parts obtained from entropy decoding of the bitstream are combined and input into the decoder to obtain the reconstructed image.

[0131] One embodiment of the present invention provides a deep learning image compression system.

[0132] like Figure 8 As shown, the deep learning image compression system provided in this embodiment includes:

[0133] The input image processing module reads the input image x based on a deep learning encoder, transforms the input image x into a latent variable y nonlinearly, and inputs the latent variable y into a super-prior encoder to obtain the super-prior bitstream to be transmitted. The super-prior decoder then obtains the super-prior information containing the latent variable structure information.

[0134] The quantization module is used to quantize the obtained latent variable y to obtain the quantized latent variable.

[0135] The context extraction module is used to extract content-adaptive context information and obtain quantized latent variables based on this information. Calculate the distribution prediction parameters, and then apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the bitstream of the quantized latent variables; entropy decoding is then performed based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For quantified latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression;

[0136] The context extraction module employs any one of the content adaptive context extraction methods described in the above embodiments of the present invention; wherein:

[0137] The first encoding process involves the quantized latent variables. Divide the region into simple and complex regions, and then quantify the latent variables. Divided into anchor points pseudo anchor point Non-anchor point Among them, anchor points This information is used to extract context information during the second encoding pass. The extracted context information is then used for probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

[0138] It should be noted that the steps in the deep learning image compression method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the deep learning image compression system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0139] In the technical solution provided by the above embodiments of the present invention, for the input image, the present invention can use any codec based on a deep learning-based end-to-end image compression method to transform x into latent variable y. During the entropy encoding of the latent variable, a two-pass encoding approach can balance compression performance and decoding latency. In the first pass of encoding, anchor points are processed as reference points for extracting contextual information; during the second pass, contextual information is extracted from the anchor points already encoded in the first pass to predict the probability distribution of latent variables at remaining positions, improving the encoding efficiency of non-anchor points. The internal calculations of the two passes of encoding can be performed in parallel, therefore the impact on encoding and decoding latency is acceptable.

[0140] The technical solution provided in the above embodiments of the present invention improves the process of extracting contextual information. Specifically, based on the distribution prediction parameters obtained in the first encoding pass, the simple and complex regions of the latent variable are divided. The input image x is encoded by an image encoder to obtain the latent variable y, which is then encoded by a super-prior encoder to obtain super-prior information h. Through the super-prior information h or the non-spatial dimension context (i.e., the optional input of the distribution prediction parameters), estimates of the mean and variance of the latent variable distribution can be obtained.

[0141] The entropy of the latent variable, i.e., the bit rate required during entropy coding, can effectively reflect the complexity of the encoded content. Typically, when using a Gaussian distribution as the prior distribution of the latent variable, the magnitude of the prediction standard deviation directly reflects the entropy, i.e., the complexity of the content. By comparing the prediction standard deviation with a pre-set threshold, the latent variable is divided into simple and complex regions, and the locations of the simple regions are masked using a mask M. simple Record it.

[0142] Depending on the location of the simple or complex regions, different densities of reference points can be assigned. For complex regions, a checkerboard-like anchor point distribution is used, with each non-anchor point's four positions (up, down, left, and right) serving as anchor points. For simple regions, each non-anchor point has only its upper-right position as an anchor point, and its upper-left position as a pseudo-anchor point. The encoding of pseudo-anchor points can also utilize contextual information to assist in probability distribution prediction. In simple regions, the anchor point distribution without contextual information is sparser compared to complex regions.

[0143] Targeting latent variables after quantization In the entropy encoding process, a two-pass parallel encoding method is used. The first pass first encodes the anchor point. Encode the remaining pseudo-anchors in the second pass. non-anchor point Encoding is then performed. During the first encoding pass, the probability distribution prediction process cannot use spatial context information; it can only use the decoded prior information and optional channel context to encode and decode the anchor points. At this time, the distribution parameters φ of the non-anchor points... a It is calculated using the following formula:

[0144]

[0145] In the formula, g ep For the distributed parameter prediction model, This represents the decoded prior information, where 0 indicates that the input to the spatial context is empty.

[0146] In the second encoding pass, the anchor point where decoding is complete. It is already available, using convolutional networks to extract contextual information from anchor points and applying it to distribution prediction in the entropy encoding of residual latent variables. (Non-anchor points) With pseudo anchors The calculation process for the distribution prediction parameters is shown in the following formula.

[0147]

[0148] φ na =φ 2-p ⊙(1-M a -M pa )

[0149] φ pa =φ 2-pas ⊙M pa

[0150] In the second encoding, the distribution prediction parameter φ 2-p In the process, in addition to using the decoded prior information In addition, it can also be obtained from the decoded anchor point Contextual information can be extracted. Alternatively, it can be extracted from the predicted mean of non-anchor points in a simple region.

[0151] After two rounds of encoding, the anchor and non-anchor parts of the latent variables are merged to obtain the complete decoded latent variables.

[0152] During the training process, the mean value of simple regions predicted by prior information can be optionally included in the reconstruction loss function to enhance the accuracy of the prior prediction information.

[0153] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0154] In some embodiments of the present invention:

[0155] For latent variables that need to be entropy encoded The probability distribution parameters required for entropy encoding are obtained using a two-pass fast encoding method, and the bitrate is obtained by arithmetic encoding based on these prediction parameters. During the extraction of contextual information, a content-adaptive approach is used to extract the quantized latent variables. It is divided into three parts: anchor point, pseudo-anchor point and non-anchor point. Convolution is used to extract context information from the anchor point to assist the probability prediction process in the entropy coding of pseudo-anchor point and non-anchor point.

[0156] First, set the anchor point Initialize the latent variables to be distributed according to the chessboard grid intervals. During the first pass of encoding, the latent variables are divided into simple and complex regions based on their complexity. For simple regions, reduce the number of nearby anchor points and set the removed redundant anchor points as pseudo-anchor points. For complex regions, a denser distribution of anchor points with varying intervals is retained, resulting in anchor points adapted to the image content. Only prior information is used as auxiliary information during encoding. In the second encoding pass, adaptive anchor points are obtained from decoding. as well as The predicted mean in the data is used to extract contextual information using convolution, which is then used for the remaining non-anchor points. With pseudo anchors The encoding. Finally, the anchor point. pseudo anchor point Non-anchor point After being combined according to their positional relationships, the latent variables of the quantized decoder input are obtained.

[0157] The latent variables can be divided into complex and simple regions, and a mask M can be used. simple This is used to record the non-anchor points. For the latent variables after encoder and quantization... We use the information obtained through the prior network to determine the magnitude of its entropy, and use this as the basis for judging complexity. Typically, when latent variables are modeled as Gaussian distributions, the prior information includes the parameters (μ, σ) of the initial prediction of the latent variables, i.e., the mean and standard deviation of the Gaussian distribution. The magnitude of the standard deviation reflects the magnitude of the entropy of the Gaussian distribution. We use the standard deviation σ as the basis for judging the complexity of the latent variables. By setting a threshold, regions with a standard deviation σ less than the threshold are defined as simple regions, while those greater than the threshold are defined as complex regions.

[0158] Based on the locations of simple and complex regions, anchor points and non-anchor points can be located, and the top-left anchor point of the non-anchor point in the simple region can be set as a pseudo anchor point. By finding non-anchor points in simple areas Find its position, locate its upper left position, and use another mask M. pa To record pseudo anchor points The location.

[0159] By removing false anchor points from a dense checkerboard pattern of anchor points, the number of redundant anchor points can be reduced. This is achieved by using a checkerboard mask M. ckbd Mask M for recording pseudo-anchor points pa Subtraction yields the content-adaptive anchor point distribution mask M. aThe mask can be calculated using only prior information, and no additional information needs to be transmitted in the bitstream.

[0160] The missing positions of the removed pseudo-anchor points are filled with the predicted mean information of neighboring non-anchor points. In the convolutional input of the context model, besides the already decoded... In addition, the mean information of nearby non-anchor points is also used as input, and the context model g cm The input can be represented by the following formula:

[0161]

[0162] The mean information obtained from the prediction was used when extracting the context, so the predicted mean was also included in the reconstruction loss for backpropagation during the training of the compressed model.

[0163] For an input image x to be encoded, the encoder nonlinearly transforms it into a latent variable y using methods such as convolution. The latent variable is then quantized and entropy-encoded to obtain the bitstream to be transmitted. The decoder then accepts the quantized latent variable and transforms it back into a reconstructed image in the pixel domain.

[0164] The super-prior encoder extracts additional information from the latent variable y in the encoder output and includes it as part of the transmitted bitstream. The super-prior decoder then decodes the super-prior bitstream to obtain super-prior information, which is used to assist in the probability distribution prediction process of the latent variable.

[0165] For the quantized latent variables to be encoded, the context extraction scheme selects a portion of the latent variables as reference information through content adaptation to assist in the probability prediction process of the remaining latent variables. The extracted context information, together with the prior information, is input into the prediction parameter network to obtain the probability distribution prediction parameters for the latent variables.

[0166] Based on the predicted probability distribution parameters, the entropy coding module encodes the quantized latent variables to obtain the required transmission bitstream.

[0167] The results of the content-adaptive context extraction technique provided in the above embodiments of the present invention are evaluated below, using the Kodak and CLIC datasets as test image datasets. Three models, Minnen18, Cheng20-attn, and He2022, are used as baseline models. He2020 is a deep learning image encoder with advanced compression performance. These three baseline models are quantitatively compared using the content-adaptive context extraction technique of the above embodiments and the original checkerboard context model.

[0168] For quantitative evaluation, we used a rate-distortion curve to measure and calculated the corresponding BD-rate savings. In the rate-distortion curve, the upper curve achieves better reconstruction quality and compression performance at the same bitrate. The BD-rate metric quantitatively indicates the percentage of bitrate savings achieved at the same reconstruction quality. Additionally, we compared the impact on encoding / decoding latency after using our method.

[0169] Figure 9 Tables (a) and (b) compare the RD curves of the checkerboard context model and the content-adaptive checkerboard context model of the embodiment on the three baseline models mentioned above, where bpp (bits-per-pixel) represents the number of bits required per pixel, and psnr is the peak signal-to-noise ratio of the reconstructed image to the original image. Table 1 shows the corresponding BD-rate gain. It can be seen that the method of this embodiment can achieve better compression performance than the original checkerboard context model. On the Kodak dataset, the method of this embodiment can achieve a maximum gain of 3.29%, while on the CLIC dataset, the maximum gain reaches 4.48%. On the current best-performing compression model He2022, the method of this embodiment can also achieve bitrate savings of 2.07% and 2.34% on the Kodak and CLIC datasets, respectively.

[0170] Table 2 shows the encoding and decoding latency of the content-adaptive checkerboard context model of this embodiment compared to the original checkerboard context model on three different baseline models. All tests were performed on a single 3090 GPU. The method of this embodiment adds only a small number of decision and masking operations, resulting in minimal additional computation, thus achieving encoding and decoding latency that is essentially the same as the original checkerboard context model.

[0171] Table 1. Quantitative comparison results between the original chessboard context model and the method of this embodiment.

[0172]

[0173]

[0174] Table 2 shows the average encoding / decoding latency test results on the Kodak dataset.

[0175]

[0176] An embodiment of the present invention provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform any of the methods in the above embodiments, or to run any of the systems in the above embodiments of the present invention.

[0177] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0178] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0179] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0180] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0181] According to a sixth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods described above, or to run any of the systems described above.

[0182] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0187] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0188] The deep learning image compression method and its content-adaptive context extraction method provided in the above embodiments of the present invention, for the input image, after nonlinear transformation to latent variables by an encoder, employs a two-pass encoding method in the process of extracting context information for predicting the probability distribution using entropy encoding. The first pass extracts content-adaptive anchor points, and the second pass extracts context information from these anchor points as auxiliary prediction information for encoding the remaining latent variables. When selecting the anchor point distribution, the complexity of different regions of the latent variables is judged using the prior information obtained from the super-prior encoder / decoder or the channel-dimensional context information; redundant pseudo-anchor points are removed from simple regions, making the anchor point distribution in simple regions sparser; the anchor point positions of the content-adaptive distribution are extracted through masking operations, and context information is calculated from them for the second pass encoding; the results of the two passes are merged and input into the decoder to obtain the reconstructed image. The technical solution of the above embodiments of the present invention achieves a more efficient reference anchor point selection scheme, improving overall compression performance with negligible increase in computational complexity.

[0189] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0190] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A content-adaptive context extraction method for deep learning image compression, characterized in that, include: The first encoding process and the second encoding process; where: The first encoding process includes: For the input image, after nonlinear transformation to latent variables, the latent variables are divided into simple and complex regions, and a mask recording the location of the simple regions is obtained accordingly. Then, the latent variables are quantized to obtain the quantized latent variables. ; Using the mask From quantified latent variables Extracting non-anchor points from simple regions The position, and according to the non-anchor point The position index is used to find the left and top adjacent positions of each simple region that are not anchor points, and these positions are set as pseudo anchor points. and use another mask For the pseudo anchor point Record the location; Masks using a checkerboard pattern Remove the mask from it. This yields a mask that adapts to the anchor point positions of the current input image. Using the mask From the quantized latent variables Extracting anchor points The anchor points obtained in this process The context information is saved for extraction during the second pass of encoding. The second encoding process includes: Based on the anchor points obtained during the first encoding process Extract contextual information to complete content adaptive context extraction in deep learning image compression.

2. The content adaptive context extraction method according to claim 1, characterized in that, The hidden variables are divided into simple and complex regions, and a mask is obtained to record the location of the simple regions. ,include: Obtain the distribution prediction parameters of the latent variables, the distribution prediction parameters including: distribution prediction mean and distribution prediction standard deviation; Based on the predicted standard deviation of the distribution, the latent variables are divided into simple and complex regions: When the prediction standard deviation is less than a set threshold, the region is determined to be a simple region, and a mask is used. Recording is performed, including the mask. A value of 1 indicates that the location belongs to a simple region. Otherwise, this area is a complex area.

3. The content adaptive context extraction method according to claim 1, characterized in that, According to the non-anchor point The position index is used to find the left and top adjacent positions of each simple region (excluding anchor points), including: According to the mask Find the non-anchor points in the location of the simple area recorded in the middle. Location distribution, and using a mask Save it, and according to the mask. The non-anchor points recorded in The position index is used to set the left and top adjacent positions of non-anchor points in each simple region as pseudo-anchor points. and the pseudo anchor point The position is set to 1, and a mask is used. Record it; The checkerboard-patterned mask The structure uses 0s and 1s arranged in a checkerboard pattern; the checkerboard pattern mask is used. Subtract the record pseudo anchor point Position mask The resulting anchor point distribution mask, which adapts to the anchor point positions of the current input image, is called the mask. For the removed pseudo-anchor points For the missing positions, extract the values ​​of the non-anchor positions from the distribution prediction mean of the latent variables and fill them into the missing positions; The mask With the quantized latent variables By performing element-wise multiplication, the adaptive anchor points required for the second encoding process can be extracted. .

4. The content adaptive context extraction method according to claim 1, characterized in that, In the second encoding process, the anchor points obtained in the first encoding process are used as a basis. Extract contextual information, including: The context model is constructed using convolutional networks as follows: ； in, Represents the context model, The anchor points obtained in the first pass of encoding. Predict the mean of the distribution of latent variables. For mask Record the location of simple areas. This is an element-wise multiplication operation; From the anchor points already obtained Content adaptive context information is extracted from the distribution prediction mean of latent variables in simple regions, thus completing the content adaptive context extraction in deep learning image compression.

5. A content-adaptive context extraction system for deep learning image compression, characterized in that, include: First-pass encoding module and second-pass encoding module; where: The first encoding module includes: The region segmentation unit is used to nonlinearly transform the input image to latent variables, then segment the latent variables into simple and complex regions, and thereby obtain a mask that records the locations of the simple regions. Then, the latent variables are quantized to obtain the quantized latent variables. ; Non-anchor point and pseudo-anchor point setting unit, which uses the mask From quantified latent variables Extracting non-anchor points from simple regions The position, and according to the non-anchor point The position index is used to find the left and top adjacent positions of each simple region that are not anchor points, and these positions are set as pseudo anchor points. and use another mask For the pseudo anchor point Record the location; Anchor point setting unit, which uses a checkerboard-patterned mask. Remove the mask from it. This yields a mask that adapts to the anchor point positions of the current input image. Using the mask From the quantized latent variables Extracting anchor points The anchor points obtained in this process The context information is saved for extraction during the second pass of encoding. The second encoding module includes: The context extraction unit is based on the anchor points obtained in the first pass of the encoding module. Extract contextual information to complete content adaptive context extraction in deep learning image compression.

6. A deep learning image compression method, characterized in that, include: A deep learning-based encoder reads in the input image. The input image x is nonlinearly transformed into latent variables. ; the hidden variables The input is fed into the super-prior encoder to obtain the super-prior bitstream to be transmitted, and then passed through the super-prior decoder to obtain super-prior information containing latent variable structure information. For the obtained latent variables Quantization operations are performed to obtain the quantized latent variables. ; Extract content adaptive context information, and obtain the quantized latent variables based on the content adaptive context information. Calculate the distribution prediction parameters, and apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the quantized latent variable bitstream; Entropy decoding is performed based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For the quantized latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression; Wherein, the extraction of content adaptive context information adopts the content adaptive context extraction method according to any one of claims 1-5, wherein: The first encoding process for the quantized latent variables The regions are divided into simple and complex regions, and the quantized latent variables are... Divided into anchor points False anchor points Non-anchor point , wherein the anchor point The context information extracted during the second encoding process is used to perform probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

7. The deep learning image compression method according to claim 6, characterized in that, It also includes any one or more of the following: - Employ any deep learning-based end-to-end image encoder to nonlinearly transform the input image x into latent variables. The hidden variable The information is input into a hyper-prior encoder of any implementation to obtain hyper-prior information containing latent variable structure information; wherein, the latent variable structure information includes parameters for preliminary prediction of latent variables. That is, the mean of the Gaussian distribution. and standard deviation ; - Employing scalar or vector quantization methods to extract latent variables. The floating-point number is converted into an integer that can be used for lossless entropy encoding, and the quantization operation is completed to obtain the quantized latent variable. ; - In the first encoding process, spatial context information is not used. Instead, the distribution prediction parameters of the latent variables are obtained through prior information or channel-dimensional context information. - During the second encoding process, based on the anchor point The extracted content adaptive context information is used as the pseudo-anchor point. Non-anchor point Reference information for the probability distribution prediction process; - Based on the distribution prediction parameters, the anchor point is obtained from the entropy decoding of the bitstream. False anchor points and non-anchor points These three parts are added together to obtain the quantified latent variables. This is then input into the decoder to obtain the final reconstructed image. This completes the image compression.

8. A deep learning image compression system, characterized in that, include: The input image processing module reads in the input image using a deep learning-based encoder. The input image x is nonlinearly transformed into latent variables. ; the hidden variables The input is fed into the super-prior encoder to obtain the super-prior bitstream to be transmitted, and then passed through the super-prior decoder to obtain super-prior information containing latent variable structure information. The quantization module is used to process the obtained latent variables. Quantization operations are performed to obtain the quantized latent variables. ; The context extraction module is used to extract content adaptive context information and obtain the quantized latent variables based on the content adaptive context information. Calculate the distribution prediction parameters, and apply the quantized latent variables based on the distribution prediction parameters. Entropy encoding is performed to obtain the quantized latent variable bitstream; Entropy decoding is performed based on the distribution prediction parameters to obtain the quantized latent variables from the bitstream of the quantized latent variables. For the quantized latent variables Reconstruction is performed to obtain the reconstructed image. Complete image compression; The context extraction module employs the content adaptive context extraction method as described in any one of claims 1-5; wherein: The first encoding process for the quantized latent variables The regions are divided into simple and complex regions, and the quantized latent variables are... Divided into anchor points False anchor points Non-anchor point , wherein the anchor point The context information extracted during the second encoding process is used to perform probability distribution prediction, and the obtained probability distribution prediction parameters are used to adjust the pseudo-anchor points. Non-anchor point Perform entropy encoding and entropy decoding.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to execute the content adaptive context extraction method of any one of claims 1-4, or to execute the deep learning image compression method of any one of claims 6-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can be used to perform the content adaptive context extraction method of any one of claims 1-4, or to perform the deep learning image compression method of any one of claims 6-7.

Images