Image compression method, device and readable storage medium

Abstract

This application discloses an image compression method, device, and readable storage medium. In this application, the latent representation corresponding to the image to be compressed is sliced, and each slice is then arithmetically encoded. One of the encoding criteria is conditional information, where multiple methods exist for generating conditional information, resulting in multiple pieces of conditional information. It is understood that arithmetic coding is a type of entropy coding, and since conditional entropy is less than or equal to entropy, this application uses conditional entropy encoding to achieve a higher compression rate.

Description

Technical Field

[0001] The present application relates to the field of image compression, and in particular to an image compression method, device, and readable storage medium. Background Art

[0002] With the rise of social networks, a vast number of images have become available online. For both network service providers and users, finding efficient image compression methods is essential. Traditional image compression methods rely on manual design and lack joint optimization, making it difficult to improve coding efficiency. In recent years, deep learning has rapidly developed. By stacking multi-layer networks, deep learning possesses powerful expressive capabilities and enables end-to-end optimization. However, most current deep learning-based image compression methods perform conditional probability modeling only in the spatial or channel domains. This inaccurate and incomplete conditional probability modeling leads to low compression rates.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide an image compression method, which aims to solve the current image compression methods based on deep learning. Most of them only perform conditional probability modeling in the spatial domain or only in the channel domain. The conditional probability modeling is inaccurate and incomplete. Therefore, the current existing image compression methods still have the technical problem of low compression efficiency.

[0005] To achieve the above object, the present application provides an image compression method, which includes the following steps:

[0006] Extracting auxiliary information from latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a probability distribution of basic features of the latent representation data;

[0007] Slicing the latent representation data to obtain a plurality of slice data, and sequentially obtaining each of the slice data as a current slice to be encoded;

[0008] performing arithmetic coding on the current slice to be coded based on the auxiliary information and the condition information of the current slice to be coded to obtain slice coded data, wherein the condition information is generated based on the current slice to be coded and / or a coded slice before the current slice to be coded, and performing the step of sequentially obtaining each slice data as the current slice to be coded until all the slice data are traversed;

[0009] The auxiliary compressed code stream corresponding to each of the slice encoding data and the auxiliary information is used as the compressed code stream corresponding to the picture to be compressed.

[0010] Furthermore, the condition information includes channel context condition information, local spatial condition information, and global spatial condition information, and the step of encoding the current slice to be encoded based on the auxiliary information and the current slice condition information to obtain slice encoding data includes:

[0011] Dividing the current to-be-encoded slice into a first slice portion and a second slice portion based on a preset chessboard grid division method;

[0012] generating a first feature probability distribution of a first portion of the slice based on the auxiliary information and channel context condition information, wherein the channel context condition information is generated based on the coded slice;

[0013] Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data;

[0014] generating a second feature probability distribution for the second part of the slice based on the auxiliary information, the channel context condition information, the local spatial condition information, and the global condition information, wherein the local spatial condition information is generated based on the first part of the slice, and the global condition information is generated based on a previously encoded slice of the current slice to be encoded and the first part of the slice;

[0015] Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data;

[0016] The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

[0017] Furthermore, the global spatial condition information includes a spatial condition within a slice. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes:

[0018] Convolving the product of the masked attention map of the last encoded slice and the first part of the slice to obtain first fusion information;

[0019] A result of adding the first fusion information to the first part of the slice is input into a preset feedforward network to obtain the spatial domain condition within the slice.

[0020] Furthermore, the global spatial condition information also includes an inter-slice spatial condition. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes:

[0021] Performing masked attention calculation on the first part of the slice and an encoded slice before the current slice to be encoded to obtain a calculation result;

[0022] Convolving the calculation result with the product of the last encoded slice to obtain second fusion information;

[0023] The result of adding the second fusion information and the last coded slice is input into a feedforward network to obtain the inter-slice spatial condition.

[0024] Furthermore, after the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0025] Traversing the first portion of the slice by sliding a preset sliding window;

[0026] Performing mask attention calculation on the corresponding portion of each sliding window in the first part of the slice;

[0027] Convolving the calculation results corresponding to each sliding window to obtain context information within the window;

[0028] The sum of the context information within the window and the first part of the slice is input into a preset feedforward network to generate the local spatial domain condition information.

[0029] Furthermore, after the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0030] The encoded slice is input into a preset channel context model to generate the channel context condition information, wherein the preset channel context model is composed of a convolutional layer and a Gaussian error linear unit.

[0031] Furthermore, after the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0032] If the current slice to be encoded is the first slice of the picture to be compressed, generating a first feature probability distribution of a first part of the slice based on the auxiliary information;

[0033] Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data;

[0034] generating a second feature probability distribution of a second portion of the slice based on the local spatial condition information and the auxiliary information;

[0035] Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data;

[0036] The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

[0037] Furthermore, the step of extracting auxiliary information from the latent representation data corresponding to the image to be compressed includes:

[0038] Performing dimensionality reduction on the latent representation data based on a preset super-prior encoder to obtain dimensionality-reduced data;

[0039] Performing arithmetic encoding and decoding on the dimension-reduced data based on a preset feature distribution probability in a preset full decomposition entropy model to obtain encoded and decoded data;

[0040] The auxiliary information is obtained by performing dimension upgrading on the coded and decoded data based on a preset super-a priori decoder.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides an image compression device, which includes: a memory, a processor, and an image compression program stored on the memory and runnable on the processor, and when the image compression program is executed by the processor, the steps of the image compression method as described above are implemented.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a readable storage medium, on which an image compression program is stored. When the image compression program is executed by a processor, the steps of the above-mentioned image compression method are implemented.

[0043] Embodiments of the present application provide an image compression method, device, and readable storage medium. In this application, when compressing and encoding an image, auxiliary information is extracted from latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a probability distribution of basic features of the latent representation data; the latent representation data is sliced ​​to obtain multiple slice data, and each slice data is sequentially obtained as the current slice to be encoded; the current slice to be encoded is arithmetically encoded based on the auxiliary information and conditional information of the current slice to be encoded to obtain slice coded data, wherein the conditional information is generated based on the current slice to be encoded and / or a previously encoded slice before the current slice to be encoded; the step of sequentially obtaining each slice data as the current slice to be encoded is performed until all slice data are traversed; and the auxiliary compressed code stream corresponding to each slice coded data and the auxiliary information is used as the compressed code stream corresponding to the image to be compressed. It is understood that in this application, the latent representation data corresponding to the image to be compressed is sliced, and then each slice is sequentially arithmetically encoded, and one of the encoding criteria is the conditional information, wherein there are multiple methods for generating the conditional information, and therefore, there are multiple conditional information. It can be understood that arithmetic coding is a type of entropy coding, and conditional entropy is less than or equal to entropy, so this application uses conditional entropy for coding to achieve a higher compression rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0045] Figure 2 This is a flowchart of the first embodiment of the image compression method of the present application;

[0046] Figure 3 This is a flow chart of the second embodiment of the image compression method of the present application;

[0047] Figure 4 This is a flowchart of the third embodiment of the image compression method of the present application;

[0048] Figure 5 This is a flowchart of the fourth embodiment of the image compression method of the present application;

[0049] Figure 6 This is a flowchart of the fifth embodiment of the image compression method of the present application;

[0050] Figure 7 This is a flowchart of a sixth embodiment of the image compression method of the present application;

[0051] Figure 8 Schematic diagram of the global spatial context model in the image compression method of this application;

[0052] Figure 9 This is a schematic diagram of a mask in the image compression method of this application;

[0053] Figure 10 Schematic diagram of the global spatial context model between slices in the image compression method of this application;

[0054] Figure 11 Schematic diagram of the local spatial context model in the image compression method of this application;

[0055] Figure 12 This is another mask schematic diagram in the image compression method of this application;

[0056] Figure 13 Schematic diagram of the channel context model in the image compression method of this application;

[0057] Figure 14 This is a schematic diagram of the image encoding and decoding process in the image compression method of this application;

[0058] Figure 15 This is a schematic diagram of the preset chessboard grid division result in the image compression method of this application;

[0059] Figure 16 This is a schematic diagram comparing the rate-distortion performance of the image compression method of this application at various bitrate points.

[0060] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0062] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0063] The device in the embodiment of the present application may be a PC, or an electronic terminal device with data processing capabilities, such as a smart phone, a tablet computer, or a portable computer.

[0064] like Figure 1As shown, the device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0065] Optionally, the device may also include a camera, RF (Radio Frequency) circuit, sensor, audio circuit, WiFi module, etc. Among them, sensors include light sensors, motion sensors and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor can turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile device can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0066] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0067] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image compression program.

[0068] exist Figure 1 In the device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the image compression program stored in the memory 1005 and perform the following operations:

[0069] Extracting auxiliary information from latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a probability distribution of basic features of the latent representation data;

[0070] Slicing the latent representation data to obtain a plurality of slice data, and sequentially obtaining each of the slice data as a current slice to be encoded;

[0071] performing arithmetic coding on the current slice to be coded based on the auxiliary information and the condition information of the current slice to be coded to obtain slice coded data, wherein the condition information is generated based on the current slice to be coded and / or a coded slice before the current slice to be coded, and performing the step of sequentially obtaining each slice data as the current slice to be coded until all the slice data are traversed;

[0072] The auxiliary compressed code stream corresponding to each of the slice encoding data and the auxiliary information is used as the compressed code stream corresponding to the picture to be compressed.

[0073] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0074] The condition information includes channel context condition information, local spatial domain condition information, and global spatial domain condition information. The step of encoding the current to-be-encoded slice based on the auxiliary information and the current to-be-encoded slice condition information to obtain slice encoding data includes:

[0075] Dividing the current to-be-encoded slice into a first slice portion and a second slice portion based on a preset chessboard grid division method;

[0076] generating a first feature probability distribution of a first portion of the slice based on the auxiliary information and channel context condition information, wherein the channel context condition information is generated based on the coded slice;

[0077] Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data;

[0078] generating a second feature probability distribution for the second part of the slice based on the auxiliary information, the channel context condition information, the local spatial condition information, and the global condition information, wherein the local spatial condition information is generated based on the first part of the slice, and the global condition information is generated based on a previously encoded slice of the current slice to be encoded and the first part of the slice;

[0079] Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data;

[0080] The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

[0081] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0082] The global spatial condition information includes a spatial condition within a slice. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes:

[0083] Convolving the product of the masked attention map of the last encoded slice and the first part of the slice to obtain first fusion information;

[0084] A result of adding the first fusion information to the first part of the slice is input into a preset feedforward network to obtain the spatial domain condition within the slice.

[0085] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0086] The global spatial condition information further includes an inter-slice spatial condition. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes:

[0087] Performing masked attention calculation on the first part of the slice and an encoded slice before the current slice to be encoded to obtain a calculation result;

[0088] Convolving the calculation result with the product of the last encoded slice to obtain second fusion information;

[0089] The result of adding the second fusion information and the last coded slice is input into a feedforward network to obtain the inter-slice spatial condition.

[0090] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0091] After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0092] Traversing the first portion of the slice by sliding a preset sliding window;

[0093] Performing mask attention calculation on the corresponding portion of each sliding window in the first part of the slice;

[0094] Convolving the calculation results corresponding to each sliding window to obtain context information within the window;

[0095] The sum of the context information within the window and the first part of the slice is input into a preset feedforward network to generate the local spatial domain condition information.

[0096] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0097] After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0098] The encoded slice is input into a preset channel context model to generate the channel context condition information, wherein the preset channel context model is composed of a convolutional layer and a Gaussian error linear unit.

[0099] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0100] After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes:

[0101] If the current slice to be encoded is the first slice of the picture to be compressed, generating a first feature probability distribution of a first part of the slice based on the auxiliary information;

[0102] Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data;

[0103] generating a second feature probability distribution of a second portion of the slice based on the local spatial condition information and the auxiliary information;

[0104] Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data;

[0105] The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

[0106] Furthermore, the processor 1001 may call the image compression program stored in the memory 1005 and perform the following operations:

[0107] The step of extracting auxiliary information from the latent representation data corresponding to the image to be compressed includes:

[0108] Performing dimensionality reduction on the latent representation data based on a preset super-prior encoder to obtain dimensionality-reduced data;

[0109] Performing arithmetic encoding and decoding on the dimension-reduced data based on a preset feature distribution probability in a preset full decomposition entropy model to obtain encoded and decoded data;

[0110] The auxiliary information is obtained by performing dimension upgrading on the coded and decoded data based on a preset super-a priori decoder.

[0111] Reference Figure 2 , the first embodiment of the image compression method of the present application, the image compression method comprises:

[0112] Step S10, extracting auxiliary information from the latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a basic feature probability distribution of the latent representation data;

[0113] It should be noted that, in this embodiment, the image compression process and the image decompression process are symmetrical, that is, the image decompression process is the inverse process of the image compression process. Therefore, this embodiment describes the image compression process, and the image decompression process can refer to the image compression process, which will not be repeated here.

[0114] Specifically, before compressing the image to be compressed, the image is preprocessed to reduce redundancy. This can be processed by a preset primary encoder to produce more compact latent representation data. The primary encoder convolution kernel can be configured or designed by a technician based on actual conditions. In this embodiment, the primary encoder downsamples the image to be compressed four times to extract latent representation data. This latent representation data is used to extract auxiliary information, which is the probability distribution of the underlying features of the latent representation data. It will be appreciated that in this embodiment, the latent representation data can be encoded and compressed using an arithmetic encoder, and the auxiliary information can serve as one of the bases for the arithmetic encoding.

[0115] Furthermore, the step of extracting auxiliary information from the latent representation data corresponding to the image to be compressed includes: reducing the dimension of the latent representation data based on a preset super-prior encoder to obtain reduced-dimensional data; arithmetically encoding and decoding the reduced-dimensional data based on a preset feature distribution probability in a preset full decomposition entropy model to obtain encoded and decoded data; and increasing the dimension of the encoded and decoded data based on a preset super-prior decoder to obtain the auxiliary information.

[0116] Specifically, the preset super a priori encoder can be composed of a convolution layer and a downsampling convolution layer. The specific number of layers can be set by technical personnel according to needs. The preset super a priori encoder performs multiple downsampling on the latent representation data to reduce the dimension and obtain the reduced dimension data. The reduced dimension data is quantized so that it can be arithmetically encoded, and then the quantized reduced dimension data is arithmetically encoded and arithmetically decoded based on the preset feature distribution probability in the preset full decomposition entropy model, wherein the preset feature distribution probability of the preset full decomposition entropy model can be obtained by statistics on the training data. The coded and decoded data obtained by arithmetic encoding and decoding is then decoded by the super a priori decoder to extract the basic feature probability distribution of the image to be compressed (i.e., the above-mentioned auxiliary information).

[0117] Step S20, slicing the latent representation data to obtain a plurality of slice data, and sequentially obtaining each of the slice data as a current slice to be encoded;

[0118] Specifically, the latent representation data is sliced ​​to obtain a plurality of slice data. For example, if the number of channels of the latent representation data is 320, and the number of channels of each slice is 32, then the number of corresponding slices is 10. This application does not limit the number of slices, and it can be set based on demand. Slice data is obtained in sequence from multiple continuous slices as the current slice to be encoded. Therefore, in this embodiment, encoding the compressed image is actually encoding each slice corresponding to the compressed image in sequence. For the latent representation, its dimension is (CxHxW), C is the channel dimension, and there is a certain correlation or redundancy between channels. Therefore, the latent representation can be divided into multiple slices. For the current slice, the previously encoded and decoded slice can be used as a condition. That is, when the features are divided into slices, the correlation of each slice in the spatial domain is always similar. Therefore, when capturing the global spatial context of the current slice, the attention map of the previous slice can be used to approximate the global correlation of the current slice to establish a global spatial context model between slices and a global spatial context model within a slice.

[0119] Step S30: performing arithmetic coding on the current slice to be coded based on the auxiliary information and the condition information of the current slice to be coded to obtain slice coding data, wherein the condition information is generated based on the current slice to be coded and / or a coded slice before the current slice to be coded, and performing the step of sequentially obtaining each slice data as the current slice to be coded until all the slice data are traversed;

[0120] Specifically, the auxiliary information includes a basic feature probability distribution generated based on the entire latent representation data. Therefore, when performing arithmetic coding on the current slice to be coded, the auxiliary information can be used as one of the coding bases. The conditional information corresponding to the current slice to be coded can be based on the current slice to be coded itself, or generated based on the previously coded slice before the current slice to be coded, or generated based on both the current slice to be coded and the coded slice. Gaussian probability estimation is performed using the conditional information and the auxiliary information to obtain an estimated feature probability distribution corresponding to the current slice to be coded. Based on this estimated feature probability distribution, the current slice to be coded can be encoded to obtain slice coded data corresponding to the current slice to be coded. Through the above process, each slice data is arithmetically coded in sequence to obtain each slice coded data.

[0121] Furthermore, the condition information includes channel context condition information, local spatial domain condition information and global spatial domain condition information, and the step of encoding the current slice to be encoded based on the auxiliary information and the current slice condition information to obtain slice encoding data includes: dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method; generating a first feature probability distribution of the first slice part based on the auxiliary information and the channel context condition information, wherein the channel context condition information is generated based on the encoded slice; arithmetically encoding the first slice part based on the first feature probability distribution to obtain first encoded data; generating a second feature probability distribution of the second slice part based on the auxiliary information, the channel context condition information, the local spatial domain condition information and the global condition information, wherein the local spatial domain condition information is generated based on the first slice part, and the global condition information is generated based on the previous encoded slice of the current slice to be encoded and the first slice part; arithmetically encoding the second slice part based on the second feature probability distribution to obtain second encoded data; and using the first encoded data and the second encoded data as the slice encoding data corresponding to the current slice to be encoded.

[0122] Specifically, before encoding a current slice to be encoded, the current slice to be encoded may be divided into two parts. The division method may be a preset chessboard grid division method. For example, the current slice to be encoded is divided into a plurality of grids. Based on a preset certain division rule, each grid is divided into two types of grids. Each type of grid is a part. The division rule is to treat any adjacent grids whose side lengths do not connect as one type, that is, the first part of the slice or the second part of the slice. It should be noted that the first part of the slice or the second part of the slice may be any one of the two types of grids. For example, referring to Figure 15 , Schematic diagram of the preset chessboard grid division result, a slice can be divided into two parts.

[0123] Gaussian probability estimation is performed through auxiliary information and channel context condition information to obtain the first characteristic probability distribution of the first part of the slice. Based on the first characteristic probability distribution, the first part of the slice can be arithmetically encoded based on the first characteristic probability distribution to obtain the first coded data. Wherein, the channel context condition information is generated based on the coded slice, and the coded slice generated can be the slice data that has been coded before the current slice to be coded, or it can be partially coded slice data. Gaussian probability estimation is performed through auxiliary information, channel context condition information, local spatial condition information and global condition information to obtain the second characteristic probability distribution of the second part of the slice, and then the second part of the slice is arithmetically encoded based on the second characteristic probability distribution to obtain the second coded data, wherein the local spatial condition information is generated based on the first part of the slice, and the global condition information is generated based on the previous coded slice and the first part of the slice of the current slice to be coded. The first coded data and the second coded data are the slice coding data corresponding to the current slice to be coded.

[0124] Step S40: Using the slice encoding data and the auxiliary compressed code stream corresponding to the auxiliary information as the compressed code stream corresponding to the picture to be compressed.

[0125] Specifically, the auxiliary compressed code stream corresponding to the coded data of each slice and the auxiliary information is obtained as the compressed code stream of the image to be compressed, wherein the auxiliary compressed code stream can be obtained by arithmetic coding the output data of a preset super-prior encoder.

[0126] In this embodiment, when compressing and encoding an image, auxiliary information is extracted from the latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a probability distribution of basic features of the latent representation data; the latent representation data is sliced ​​to obtain multiple slice data, and each slice data is sequentially obtained as the current slice to be encoded; the current slice to be encoded is arithmetically encoded based on the auxiliary information and conditional information of the current slice to be encoded to obtain slice coded data, wherein the conditional information is generated based on the current slice to be encoded and / or a previously encoded slice before the current slice to be encoded; the step of sequentially obtaining each slice data as the current slice to be encoded is performed until all slice data are traversed; and the auxiliary compressed codestream corresponding to each slice coded data and the auxiliary information is used as the compressed codestream corresponding to the image to be compressed. It will be understood that in this application, the latent representation data corresponding to the image to be compressed is sliced, and then each slice is sequentially arithmetically encoded, and one of the encoding criteria is the conditional information, wherein there are multiple methods for generating the conditional information, and therefore, there are multiple conditional information. It can be understood that arithmetic coding is a type of entropy coding, and conditional entropy is less than or equal to entropy, so this application uses conditional entropy for coding to achieve a higher compression rate.

[0127] Reference Figure 3Based on the first embodiment of the image compression method of the present application, a second embodiment of the image compression method is proposed. The parts of this embodiment that are identical to the above embodiments may be referred to above and will not be repeated here. The global spatial condition information includes the spatial condition within the slice. After the step of dividing the current to-be-encoded slice into a first slice portion and a second slice portion based on a preset chessboard grid division method, the method includes:

[0128] Step A10, convolving the product of the masked attention map of the last encoded slice and the first part of the slice to obtain first fusion information;

[0129] Step A20: Input the result of adding the first fusion information and the first part of the slice into a preset feedforward network to obtain the spatial domain condition within the slice.

[0130] Specifically, when capturing the global information within a slice (i.e., the spatial conditions within the slice), the global spatial context model within the slice is used. The model structure is as follows: Figure 8 As shown. The steps in this model are as follows: Since the previous slice has been encoded, the attention map of the first part of the slice of the previous encoded slice and the second part of the slice of the previous encoded slice can be used to predict the global spatial dependency within the current slice to be encoded. Specifically, the masked attention map of the previous encoded slice is multiplied by the first part of the slice of the current slice to be encoded, and then a layer of convolution is used to further fuse the results to obtain the first fusion information, and then the first fusion information is added to the first part of the slice of the current slice to be encoded, and the result of the addition is sent to the feedforward network to obtain the global context information within the slice, that is, the spatial condition within the slice. It should be noted that the mask limits the interaction between the first part of the slice of the previous encoded slice and the second part of the slice of the previous encoded slice and the interaction in the local spatial domain. The mask is as follows Figure 9 shown.

[0131] Reference Figure 4 Based on the second embodiment of the image compression method of the present application, a third embodiment of the image compression method is proposed. The parts of this embodiment that are identical to the above embodiments may be referred to above and will not be repeated here. The global spatial condition information also includes inter-slice spatial conditions. After the step of dividing the current to-be-encoded slice into a first slice portion and a second slice portion based on a preset chessboard grid division method, the method includes:

[0132] Step B10, performing mask attention calculation on the first part of the slice and the previous encoded slice of the current slice to be encoded to obtain a calculation result;

[0133] Step B20: Convolve the calculation result and the product of the last encoded slice to obtain second fusion information;

[0134] Step B30: Input the result of adding the second fusion information and the last coded slice into a feedforward network to obtain the inter-slice spatial condition.

[0135] Specifically, in order to obtain the global spatial context information between slices (i.e., the spatial conditions between slices), the global spatial context model between slices is used, the first part of the slice is used as an approximation of the second part of the slice, the first part of the slice of the current slice to be encoded and the previous encoded slice are subjected to mask attention calculation to obtain the calculation result, and then a layer of convolution is used to further convolve the calculation result to obtain the second fusion information, and then the result of the addition of the second fusion information and the previous encoded slice is sent to the feedforward network to obtain the global context information between slices. The global spatial context model between slices is as follows: Figure 10 shown.

[0136] Reference Figure 5 Based on the third embodiment of the image compression method of the present application, a fourth embodiment of the image compression method is proposed. The parts of this embodiment that are identical to the above embodiments may be referred to above and will not be repeated here. After the step of dividing the current slice to be encoded into a first slice portion and a second slice portion based on a preset chessboard grid division method, the method includes:

[0137] Step C10, traversing the first part of the slice by sliding a preset sliding window;

[0138] Step C20, performing mask attention calculation on the corresponding portion of each sliding window in the first part of the slice;

[0139] Step C30, performing convolution on the calculation results corresponding to each sliding window to obtain context information within the window;

[0140] Step C40: Inputting the sum of the context information within the window and the first part of the slice into a preset feedforward network to generate the local spatial domain condition information.

[0141] Specifically, when constructing the local spatial context information (local spatial condition information), the checkerboard attention local spatial context model is used to extract the first part of the slice to be encoded. The local spatial context model based on the checkerboard attention is as follows: Figure 11 As shown in . In this model, if K×K is the preset sliding window size and 1 is the sliding stride, a local context window can be constructed, and the first part of the slice can be traversed based on the window. A mask attention calculation is performed on the content in the first part of the slice corresponding to each window, and the mask becomes a checkerboard mask, so that there is no interaction between the first part of the slice and the second part of the slice. The mask part is as follows Figure 12As shown. Then, a K×K convolution fusion is used to obtain the calculation result based on the above window to obtain the context information within the window. The sum of the context information within the window and the first part of the slice is sent to the feedforward network layer. The final output of the feedforward network is the final extracted context information in the local spatial domain, that is, the local spatial domain condition information. In addition, for a picture, there can be similarities and correlations between adjacent and distant pixels in the picture. This part of the correlation can be used to establish context. For local or adjacent pixels, a window-based segmentation model and convolutional network are used. The receptive field of the window-based segmentation model and convolutional network is limited and cannot be used to capture the context of the global spatial domain or distant pixels.

[0142] Reference Figure 6 Based on the third embodiment of the image compression method of the present application, a fourth embodiment of the image compression method is proposed. The parts of this embodiment that are identical to the above embodiments may be referred to above and will not be repeated here. After the step of dividing the current slice to be encoded into a first slice portion and a second slice portion based on a preset chessboard grid division method, the method includes:

[0143] Step D10: input the encoded slice into a preset channel context model to generate the channel context condition information, wherein the preset channel context model is composed of a convolutional layer and a Gaussian error linear unit.

[0144] Specifically, this application uses the previously encoded slices as the context information on the channel. The context information existing in the previously encoded slices is extracted through the channel context model, that is, the channel context condition information. The channel context model used in this application is as follows: Figure 13 As shown in Figure 1, the channel context model consists of a convolutional layer and a Gaussian error linear unit, with the first and last layers being convolutional layers respectively.

[0145] Reference Figure 7 Based on the third embodiment of the image compression method of the present application, a fourth embodiment of the image compression method is proposed. The parts of this embodiment that are identical to the above embodiments may be referred to above and will not be repeated here. After the step of dividing the current slice to be encoded into a first slice portion and a second slice portion based on a preset chessboard grid division method, the method includes:

[0146] Step E10: If the current slice to be encoded is the first slice of the picture to be compressed, generating a first feature probability distribution of the first part of the slice based on the auxiliary information;

[0147] Step E20, performing arithmetic coding on the first portion of the slice based on the first feature probability distribution to obtain first coded data;

[0148] Step E30, generating a second feature probability distribution of the second part of the slice based on the local spatial condition information and the auxiliary information;

[0149] Step E40, performing arithmetic coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data;

[0150] Step E50: Use the first coded data and the second coded data as slice coded data corresponding to the current slice to be encoded.

[0151] It can be understood that when encoding the first slice, since other slices are encoded again before, only the context modeling in the local spatial domain is involved. Therefore, if the current slice to be encoded is the first slice of the picture to be compressed, Gaussian estimation can be directly performed based on the auxiliary information to obtain the mean and variance of the Gaussian distribution of the first part of the slice, that is, the first characteristic probability distribution, and the first part of the slice is encoded based on the first characteristic probability distribution. Further, when encoding the second part of the slice, Gaussian probability estimation is performed based on the local spatial domain condition information generated for the first part of the slice (the generation process can refer to the above embodiment and will not be repeated here) and the auxiliary information. The first encoded data and the second encoded data can be used as the slice encoding data corresponding to the current slice to be encoded.

[0152] Furthermore, the decoding process is the inverse of the encoding process. First, the auxiliary information is decoded, and the feature probability distribution of the first slice portion of the first slice is calculated using the auxiliary information. The first slice portion of the first slice is then decoded. Local spatial context information is calculated based on the first slice portion of the first slice. This local spatial context information is then fused with the auxiliary information to calculate the feature probability distribution of the second slice portion of the first slice, thereby decoding the second slice portion of the first slice.

[0153] When decoding the second and subsequent slices, channel context information is first calculated based on the previously decoded slices. This channel context information is then fused with auxiliary information to determine the distribution of the first slice portion of the current slice to be decoded, and the first slice portion of the current slice to be decoded is decoded. When decoding the second slice portion of the current slice, local spatial context information is calculated based on the first slice portion. An attention map is calculated for the previously decoded slice, and global spatial information within the slice is calculated based on the attention map and the first slice portion of the current slice to be decoded. Global spatial information between slices is calculated based on the first slice portion of the current slice to be decoded and the previously decoded slice. Based on the fusion of auxiliary information, local spatial context information, global spatial context information between slices, and global spatial context information within a slice, the feature probability distribution of the second slice portion of the current slice to be decoded is calculated, and the second slice portion is decoded. Entropy coding is performed on the latent representation data, and the decoded latent representation data is then fed into the primary decoder to recover the uncompressed image.

[0154] As reference Figure 14, a schematic diagram of the picture encoding and decoding process of the present application, the picture is input to the main encoder, and the main encoder is composed of the downsampling residual network, residual network, downsampling residual network, residual network, downsampling residual network, residual network, and downsampling residual network in sequence. The main encoder outputs latent representation data, and the latent representation data can be used to generate auxiliary information, that is, the latent representation data will be input to the super-prior coding model, which is composed of the convolution layer, convolution layer, downsampling convolution layer, convolution layer, and downsampling convolution layer in sequence in the figure. The output data of the super-prior coding model can be arithmetically encoded after quantization, and the feature probability distribution of arithmetic encoding comes from the full decomposition entropy model. Similarly, the feature probability distribution of arithmetic decoding also comes from the full decomposition entropy model, and the output of arithmetic decoding will be input to the super-prior decoding model, wherein the super-prior decoding model is composed of the upsampling convolution layer, convolution layer, upsampling convolution layer, convolution layer, and convolution layer in sequence in the figure. The super-prior decoding model extracts the basic feature probability distribution, i.e., auxiliary information, from the entire latent representation data. At the same time, the latent representation data will be sliced ​​to obtain multiple slice data, and each slice will be arithmetically encoded. The characteristic probability distribution of the arithmetically encoded data comes from the result of Gaussian probability estimation, and the basis of Gaussian probability estimation is the above-mentioned auxiliary information and the output of the local spatial domain / channel / global spatial domain within a slice / global spatial domain context model between slices. The input of the local spatial domain / channel / global spatial domain within a slice / global spatial domain context model between slices is the current slice and / or the encoded slice. The corresponding black rectangular bar in the figure is the compressed code stream corresponding to the image. When decoding the compressed code stream, the output of the arithmetic decoder will be input to the main decoder for picture reconstruction, wherein the main decoder is composed of the residual network, upsampling residual network, residual network, upsampling residual network, residual network, upsampling residual network, residual network, upsampling residual network, and upsampling residual network in sequence as shown in the figure. As shown in reference Figure 16 , the rate-distortion performance comparison effect of this application at each bit rate point compared with the traditional image coding method VVC (versatile video coding), and the present application achieves a 13% bit rate saving compared with the traditional image coding method VVC intra-frame mode, with better compression effect.

[0155] In addition, an embodiment of the present application also proposes an image compression device, which includes: a memory, a processor, and an image compression program stored in the memory and runnable on the processor. When the image compression program is executed by the processor, the steps of the image compression method described above are implemented.

[0156] In addition, an embodiment of the present application further provides a readable storage medium, on which an image compression program is stored. When the image compression program is executed by a processor, the steps of the above-mentioned image compression method are implemented.

[0157] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0158] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0159] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0160] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image compression method, characterized in that: The image compression method comprises the following steps: Extracting auxiliary information from latent representation data corresponding to the image to be compressed, wherein the auxiliary information is a probability distribution of basic features of the latent representation data; Slicing the latent representation data to obtain a plurality of slice data, and sequentially obtaining each of the slice data as a current slice to be encoded; performing arithmetic coding on the current slice to be coded based on the auxiliary information and the condition information of the current slice to be coded to obtain slice coded data, wherein the condition information includes channel context condition information, local spatial domain condition information, and global spatial domain condition information, and the condition information is generated based on the current slice to be coded and / or a coded slice before the current slice to be coded, and performing the step of sequentially obtaining each slice data as the current slice to be coded until all the slice data are traversed; The auxiliary compressed code stream corresponding to each of the slice encoding data and the auxiliary information is used as the compressed code stream corresponding to the picture to be compressed.

2. The image compression method according to claim 1, wherein: The step of encoding the current slice to be encoded based on the auxiliary information and the current slice condition information to obtain slice encoding data includes: Dividing the current to-be-encoded slice into a first slice portion and a second slice portion based on a preset chessboard grid division method; generating a first feature probability distribution of a first portion of the slice based on the auxiliary information and channel context condition information, wherein the channel context condition information is generated based on the coded slice; Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data; generating a second feature probability distribution for the second part of the slice based on the auxiliary information, the channel context condition information, the local spatial condition information, and the global condition information, wherein the local spatial condition information is generated based on the first part of the slice, and the global condition information is generated based on a previously encoded slice of the current slice to be encoded and the first part of the slice; Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data; The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

3. The image compression method according to claim 2, wherein: The global spatial condition information includes a spatial condition within a slice. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes: Convolving the product of the masked attention map of the last encoded slice and the first part of the slice to obtain first fusion information; A result of adding the first fusion information to the first part of the slice is input into a preset feedforward network to obtain the spatial domain condition within the slice.

4. The image compression method according to claim 3, wherein: The global spatial condition information further includes an inter-slice spatial condition. After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset checkerboard grid division method, the method includes: Performing masked attention calculation on the first part of the slice and an encoded slice before the current slice to be encoded to obtain a calculation result; Convolving the calculation result with the product of the last encoded slice to obtain second fusion information; The result of adding the second fusion information and the last coded slice is input into a feedforward network to obtain the inter-slice spatial condition.

5. The image compression method according to claim 4, wherein: After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes: Traversing the first portion of the slice by sliding a preset sliding window; Performing mask attention calculation on the corresponding portion of each sliding window in the first part of the slice; Convolving the calculation results corresponding to each sliding window to obtain context information within the window; The sum of the context information within the window and the first part of the slice is input into a preset feedforward network to generate the local spatial domain condition information.

6. The image compression method according to claim 5, wherein: After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes: The encoded slice is input into a preset channel context model to generate the channel context condition information, wherein the preset channel context model is composed of a convolutional layer and a Gaussian error linear unit.

7. The image compression method according to claim 6, wherein: After the step of dividing the current slice to be encoded into a first slice part and a second slice part based on a preset chessboard grid division method, the method includes: If the current slice to be encoded is the first slice of the picture to be compressed, generating a first feature probability distribution of a first part of the slice based on the auxiliary information; Performing arithmetic encoding on the first portion of the slice based on the first feature probability distribution to obtain first encoded data; generating a second feature probability distribution of a second portion of the slice based on the local spatial condition information and the auxiliary information; Performing arithmetical coding on the second portion of the slice based on the second feature probability distribution to obtain second coded data; The first encoded data and the second encoded data are used as slice encoded data corresponding to the current slice to be encoded.

8. The image compression method according to claim 7, wherein: The step of extracting auxiliary information from the latent representation data corresponding to the image to be compressed includes: Performing dimensionality reduction on the latent representation data based on a preset super-prior encoder to obtain dimensionality-reduced data; Performing arithmetic encoding and decoding on the dimension-reduced data based on a preset feature distribution probability in a preset full decomposition entropy model to obtain encoded and decoded data; The auxiliary information is obtained by performing dimension upgrading on the coded and decoded data based on a preset super-a priori decoder.

9. An image compression device, characterized in that: The image compression device includes: a memory, a processor, and an image compression program stored in the memory and executable on the processor. When the image compression program is executed by the processor, the steps of the image compression method according to any one of claims 1 to 8 are implemented.

10. A readable storage medium, characterized in that: An image compression program is stored on the readable storage medium, and when the image compression program is executed by the processor, the steps of the image compression method according to any one of claims 1 to 8 are implemented.

Images