Potential space compression method and device, potential space decompression method and device, equipment and program product

Abstract

The invention relates to the technical field of computers, and provides a potential space compression method, a potential space decompression method, a potential space compression device, a potential space decompression device, potential space compression equipment and a potential space decompression program product. Compressing the first potential variable through a neural network compression model to obtain a compressed second potential variable, and compressing the first potential variable through a neural network down-sampling model to obtain a down-sampled first potential variable; the arithmetic coding model is used for carrying out lossless Gaussian arithmetic coding on the down-sampled first potential variable to obtain a coded second potential variable; and the compressed second potential variable is transmitted to a decoding end, so that the decoding end decodes the second potential variable to obtain a decoded target image, the structural information of the data can be effectively utilized, efficient lossless compression can be realized, the first potential variable can be compressed more efficiently, the compression ratio of the image data is further improved, and the compression efficiency of the image data is improved. And image data compression cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of data compression technology, and in particular to a latent space compression method, decompression method, apparatus, device and program product. Background Art

[0002] Since OpenAI launched ChatGPT, interest in artificial intelligence (AI) has increased significantly across many industries. Large language models (LLMs) have provided impressive capabilities across many different fields, and the technology for generating images and videos has also advanced rapidly. With simple text, image, or video input, realistic images or video sequences can be generated. The rapid development and application of GPU computing has made it easier to generate videos and images using generative AI, such as using the stable diffusion method to generate images and SORA to generate videos. In the interactive generation of images or videos, some form of compressed communication is required between the data center performing the generation and the end user providing input or viewing the generated video. Existing image data compression methods suffer from low compression ratios and high costs. Summary of the Invention

[0003] In order to solve or at least partially solve the above technical problems, the present application provides a latent space compression method, decompression method, device, equipment and program product, which can improve the compression ratio of image data and reduce the cost of image data compression.

[0004] In a first aspect, the present application provides a latent space compression method, applied to an encoding end device, the method comprising:

[0005] Obtain the first latent variable generated by generative AI;

[0006] Compressing the first latent variable using a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain a downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain an encoded second latent variable;

[0007] The compressed second latent variable is transmitted to a decoding end, so that the decoding end decodes the second latent variable to obtain a decoded target image.

[0008] In a second aspect, the present application provides a latent space decompression method, applied to a decoding end device, the method comprising:

[0009] receiving the compressed second latent variable sent by the encoding end;

[0010] decoding the second latent variable through an arithmetic coding model to obtain a decoded second latent variable;

[0011] Decompressing the decoded second latent variable through a neural network upsampling model to obtain an upsampled first latent variable;

[0012] The upsampled first latent variable is decoded by a variational autoencoder to obtain a target image.

[0013] In a third aspect, the present application provides a potential space compression device, comprising:

[0014] An acquisition module, for acquiring a first latent variable generated by the generative AI;

[0015] a compression module, configured to compress the first latent variable using a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is configured to compress the first latent variable to obtain a downsampled first latent variable; and the arithmetic coding model is configured to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain an encoded second latent variable;

[0016] The sending module is used to transmit the compressed second latent variable to the decoding end, so that the decoding end decodes the second latent variable to obtain a decoded target image. In a third aspect, the present application provides an image compression device, comprising:

[0017] In a fourth aspect, the present application provides a latent space decompression device, comprising:

[0018] a receiving module, configured to receive the compressed second latent variable sent by the encoding end;

[0019] A decoding module is configured to decode the second latent variable through an arithmetic coding model to obtain a decoded second latent variable; decompress the decoded second latent variable through a neural network upsampling model to obtain an upsampled first latent variable; and decode the upsampled first latent variable through a variational autoencoder to obtain a target image.

[0020] In a fifth aspect, an embodiment of the present application also provides an encoding end device, comprising a processor and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include steps for the latent space compression method as described in the first aspect.

[0021] In the sixth aspect, an embodiment of the present application also provides a decoding end device, comprising a processor and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include steps for the latent space decompression method as described in the second aspect.

[0022] In a seventh aspect, the embodiments of the present application further provide a computer-readable program product storing program instructions, which, when executed by a computer, enables the computer to execute the latent space compression method described in the first aspect or the latent space decompression method described in the second aspect.

[0023] The implementation method of the present application obtains a first latent variable generated by generative AI; compresses the first latent variable through a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain the downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain the encoded second latent variable; the compressed second latent variable is transmitted to the decoding end so that the decoding end decodes the second latent variable to obtain a decoded target image, and lossless Gaussian arithmetic coding is performed on the downsampled first latent variable through the arithmetic coding model to achieve lossless compression, which can effectively utilize the structural information of the data to achieve efficient lossless compression, and can more efficiently compress the first latent variable, thereby improving the compression ratio of the image data and reducing the image data compression cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the embodiments of the present application, the following briefly introduces the relevant drawings. It should be understood that the drawings described below are only used to illustrate some embodiments of the present application, and those skilled in the art can also obtain many other technical features and connection relationships not mentioned herein based on these drawings.

[0025] Figure 1 A flowchart of a potential space compression method provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram illustrating image compression using a neural network compression model provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of another potential space compression method provided in an embodiment of the present application;

[0028] Figure 4A flowchart of a latent space decompression method provided in an embodiment of the present application;

[0029] Figure 5 A schematic structural diagram of a potential space compression device provided in an embodiment of the present application;

[0030] Figure 6 A schematic structural diagram of a latent space decompression device provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of an encoding terminal device provided in an embodiment of the present application;

[0032] Figure 8 A schematic structural diagram of a decoding terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0035] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] Currently, generative algorithms and technologies are being applied in the gaming industry. Technologies such as procedural generation, texture creation, dialogue generation, speech synthesis, voice modification, and generative music are all helping to accelerate and reduce the cost of game development. Some image generation companies are attempting to develop a "holodeck" to create a generative interactive world. This new form of generative gaming is an interactive process. Games are not created entirely from predefined texture maps and structured worlds, but are generated in real time through user input, creating a nearly endless world of imagination. These interactively generated worlds require some form of compressed communication between the data center where the generation occurs and the end user who provides input or views the generated video. In interactive generative creation scenarios, in order to transmit these generated images and videos, they need to be transmitted in a compressed format.

[0037] In embodiments of the present application, Neural Network Compression (NNC) technology can be used to compress images and videos. By utilizing the diffusion-based structure of videos and images, we can compress hidden information (information in the latent space), thereby reducing overall costs and speeding up processing. By generating a latent space for images and videos, and then compressing the latent space, we can reduce overall costs and speed up processing.

[0038] In order to better understand this solution, a neural network compression model used in this solution is introduced here, such as Figure 2 As shown, the neural network compression model includes a variational auto-encoder (VAE), which is a deep generation model. The main goal of VAE is to learn the potential representation of data and to generate new data samples from it.

[0039] VAE consists of two main parts: encoder and decoder, Figure 2 In the example, (E) and (D) represent the encoder and decoder, respectively. The encoder maps the input data into a distribution in a latent space, typically represented by a Gaussian distribution, outputting a mean and a variance. The decoder samples from this distribution in the latent space and maps the sampled points back to the original data space, generating new data.

[0040] VAE encoding can express the content of the original image with less information (information in the latent space), and the VAE decoder can be used to restore the original image. In generative AI, new images can be generated by modifying the latent space after VAE encoding. Therefore, this patent focuses on compressing the latent space to further improve the compression rate and is also applicable to generative AI.

[0041] The neural network compression model also includes the conversion model g aAnd the convolutional network model g s , is an extension of the variational autoencoder, which aims to serve as an interface / adapter for the general VAE model and the Gaussian conditional arithmetic coding model, while downsampling and expanding the number of channels in the latent space, with the goal of more flexible entropy reduction and bit rate minimization in arithmetic compression coding.

[0042] The neural network compression model also includes an arithmetic coding model, which is an entropy coding technology that compresses the input data into a real number interval based on its probability distribution, thereby achieving efficient lossless compression. Figure 2 As shown, AE represents arithmetic encoding and AD represents arithmetic decoding.

[0043] An example of an arithmetic coding model is the Gaussian Conditional Arithmetic Coding (GCAC) model. GCAC is a data compression method that combines probabilistic models with arithmetic coding techniques. It is commonly used in the fields of deep learning and data compression, particularly in image and video compression. GCAC utilizes models such as deep neural networks to perform probabilistic data modeling. For a given data point, the model predicts the probability distribution it follows. Under Gaussian conditions, this is typically a Gaussian distribution, and the model outputs the mean and variance of the data. During compression, GCAC predicts the probability distribution of the current data point based on the previously encoded data. This conditional probability calculation enables the encoding process to more effectively utilize the data's structural information. After obtaining the probability distribution, GCAC compresses the data using arithmetic coding. During decoding, GCAC reconstructs the original data based on the received encoded information and the predicted probability distribution. The decoding process is the reverse of the encoding process, progressively decoding each data point to ultimately restore the complete data. By combining probabilistic models with arithmetic coding, GCAC achieves efficient compression while maintaining data quality.

[0044] VAE can map the original image to a smaller latent space (lossy compression), while arithmetic coding is pure data compression (lossless compression).

[0045] The neural network compression model also includes a first hyper-prior model h a and the second hyper-prior model h s In order to use arithmetic coding to decode the received binary compressed data, it is necessary to obtain the Gaussian variance σ; this can be obtained by the first hyper-prior model h a Extract useful information from image data, the first hyper-prior model h a is a data compression model that can be obtained through the second hyper-prior model h s to obtain the Gaussian variance σ.

[0046] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.

[0047] Example 1

[0048] like Figure 1 As shown, an embodiment of the present application proposes a potential space compression method, which is applied to an encoding end device. Figure 1 A schematic flow chart of a latent space compression method provided in an embodiment of the present application, the method comprising:

[0049] 101. Obtain the first latent variable generated by generative AI.

[0050] In the interactive generation of images or videos, the encoding device can be a data center, such as a server, that generates the video. SORA is a large-scale text-to-video model developed by OpenAI that converts short text descriptions into high-quality videos. The encoding device can generate images or videos using the stable diffusion method. The stable diffusion method is a diffusion model-based technique primarily used for image generation. This method gradually adds and removes noise, resulting in more realistic and natural images. Stable diffusion models have made significant progress in image generation, generating high-quality images that meet user requirements for realism and detail. SORA is a text-to-video model that uses text prompts to create short videos. Based on the Transformer architecture, SORA employs a diffusion model and training stability techniques to generate high-quality video content. Unlike the stable diffusion method, SORA aims to generate videos rather than images. It can generate videos from simple user-provided text descriptions while maintaining visual quality and adhering to user instructions.

[0051] Specifically, a user may input simple text, image or video into a user terminal, and a realistic image or video may be generated.

[0052] Among them, the initial image is an image in the RGB space, and the initial image in the RGB space can be expressed as x∈R H×W×Z The initial image x can be obtained first, mapped to the first latent space through the variational autoencoder, and the first latent space is sampled to obtain the sampled first latent variable z, which represents the potential representation of the image data in the first latent space.

[0053] A variational auto-encoder (VAE) is a deep generative model based on the structure of an autoencoder. It maps samples to latent variables in a low-dimensional space through encoding, and then recovers these latent variables into reconstructed samples through decoding. VAEs can introduce latent variables to represent the distribution of image data. They learn the probability distribution of image data and sample latent variables from this probability distribution. The latent variables represent the potential representation of the image data in the first latent space.

[0054] Latent space refers to an abstract multidimensional space used to learn the latent features of data. This scheme can learn the latent features of the initial image in a low-dimensional latent space and simplify the latent feature data to compress the image data so that the compressed image data is easy to transmit and used for subsequent image processing.

[0055] Optionally, obtaining a first latent variable generated by generative AI includes: obtaining an initial image x; downsampling the initial image x according to a first scaling factor f1 through a variational autoencoder to obtain a first latent variable z in a first latent space.

[0056] The initial image is input into a variational autoencoder; the encoder of the variational autoencoder downsamples the input initial image x according to a first scaling factor f1 to obtain a first latent variable z of a first latent space.

[0057] Among them, through the encoder of the variational self-encoder ( Figure 2 (denoted by E in the figure) can transform the initial image into the first latent space and obtain the first latent variable z of the first latent space.

[0058] In a specific implementation, the initial input image is downsampled by the variational autoencoder according to the first scaling factor f1 to obtain the first latent variable z of the first latent space, where: f1 can be 8, for example. Figure 2 As shown, H, h is used to represent the kernel height of the convolution kernel, W, w is used to represent the kernel width of the convolution kernel, and the initial image x∈R is transformed by the variational self-encoder (E) H×W×Z Transform to the first latent space (z), where z∈R h×w×c , z = ε(x). The present invention can support any standard model that generates a latent space, without the need for further training of these models, which usually requires a large amount of time, data, and computing resources. Therefore, this solution can quickly compress image data and save computing resources, thereby improving data compression efficiency and saving computing costs. VAE can map the original image to a smaller first latent space and perform lossy compression on the initial image to reduce the bit rate of transmission over bandwidth-limited channels.

[0059] 102. Compress the first latent variable using a neural network compression model to obtain a compressed second latent variable.

[0060] Among them, the neural network compression model includes the neural network downsampling model g a and arithmetic coding model, the neural network downsampling model g a The arithmetic coding model is used to compress the first latent variable z to obtain a downsampled second latent variable y; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled second latent variable y to obtain an encoded second latent variable

[0061] Neural Network Compression (NNC) refers to reducing network parameters or storage space by changing the network structure or using quantization and approximation methods, thereby reducing network computational cost and storage space without affecting the performance of the neural network.

[0062] Downsampling model g through neural network a Compress the first latent variable z to obtain the downsampled second latent variable y, the neural network downsampling model g a A single-layer 2D convolution with a convolution kernel size of 4 can be used, and the neural network downsampling model g a Reduce the first latent space to the second latent space y∈R h’×w’×M , y = g a The second latent space (y) after dimensionality reduction of (x) is further downsampled according to the second scaling factor f2 to obtain the downsampled second latent variable y, where, f2 can be 2, for example. Since the VAE has already downsampled by a scaling factor f1=8, only a small downsampling factor is needed here. Since the VAE used in the generative model usually has good accuracy and robustness, using only a single convolution layer and no activation function is sufficient to achieve good compression performance. The number of channels M of the second latent space (y) after the transformation is larger than the first latent space (z) before the transformation, so as to more flexibly reduce entropy and minimize bit rate in arithmetic compression coding, where M is the number of channels used in the latent convolution.

[0063] Arithmetic coding is an entropy coding technology that compresses the input data into a real number interval according to its probability distribution, thereby achieving efficient lossless compression. This scheme adopts an arithmetic coding model, which performs lossless Gaussian arithmetic coding on the downsampled second latent variable y to obtain the encoded second latent variable The first latent variable z of the first latent space can be compressed. Compared with directly compressing the initial image through a neural network compression model, this scheme compresses the first latent variable z of the first latent space through an arithmetic coding model, which can improve the compression ratio of image data and reduce the cost of image data compression.

[0064] 103. Transmit the compressed second latent variable to a decoding end device, so that the decoding end device decodes the second latent variable to obtain a decoded target image.

[0065] The decoding end device can be a user terminal. In the scenario of interactively generating images or videos, the user terminal can provide input, and the encoding end device generates the initial image and compresses it to obtain the compressed second latent variable The encoding end device converts the compressed second latent variable The second latent variable is transmitted to the decoding end device, and the decoding end device Decode the target image and watch the target image or the generated video composed of the target image.

[0066] This solution obtains a first latent variable generated by generative AI; compresses the first latent variable through a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain the downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain the encoded second latent variable; the compressed second latent variable is transmitted to the decoding end so that the decoding end decodes the second latent variable to obtain a decoded target image, and lossless Gaussian arithmetic coding is performed on the downsampled first latent variable through the arithmetic coding model to achieve lossless compression, which can effectively utilize the structural information of the data to achieve efficient lossless compression, and can more efficiently compress the first latent variable, thereby improving the compression ratio of the image data and reducing the image data compression cost.

[0067] Optionally, the first latent variable is compressed by a neural network compression model to obtain a compressed second latent variable include:

[0068] The neural network downsamples the model g a Compressing the first latent variable z to obtain a downsampled second latent variable y;

[0069] The downsampled second latent variable y is quantized to obtain the quantized second latent variable

[0070] The quantized second latent variable is coded by the arithmetic coding model. Perform lossless Gaussian arithmetic coding to obtain the encoded second latent variable

[0071] Among them, the downsampled second latent variable y is quantized to obtain the quantized second latent variable The second latent space (y) after dimensionality reduction is quantized into a low entropy latent space Low-entropy transformation latent space Ability to perform arithmetic coding through Gaussian conditional arithmetic encoder; through arithmetic coding model according to the quantized second latent variable Perform lossless Gaussian arithmetic coding to obtain the encoded second latent variable

[0072] Optionally, the arithmetic coding model is used to calculate the quantized first latent variable Perform lossless Gaussian arithmetic coding to obtain the encoded second latent variable include:

[0073] Estimate the Gaussian variance of the downsampled second latent variable y through the arithmetic coding model to obtain the Gaussian variance;

[0074] The arithmetic coding model is used according to the compressed first latent variable Gaussian arithmetic coding is performed on the Gaussian variance to obtain the encoded second latent variable

[0075] By calculating the Gaussian variance σ, the Gaussian variance can be used to calculate the first latent variable after compression. Perform Gaussian arithmetic coding to obtain the compressed data, that is, the encoded second latent variable

[0076] Optionally, the neural network compression model includes a first hyper-prior model h a and the second hyper-prior model h s , the Gaussian variance of the first latent variable y after downsampling is estimated by the arithmetic coding model, and the Gaussian variance is obtained, including:

[0077] Through the first hyper-prior model h a Predict the first hyper-prior model h based on the downsampled second latent variable y a The corresponding third latent variable w of the third latent space;

[0078] The third latent variable w of the third latent space is quantized to obtain the quantized third latent variable

[0079] The quantized third latent variable is coded by an arithmetic coding model. Perform arithmetic coding to obtain the encoded third latent variable

[0080] Through the second hyper-prior model h s According to the coded third latent variable Estimate the Gaussian variance and get the Gaussian variance.

[0081] Among them, Figure 2 As shown, the first hyper-prior model h a It may include three convolutional layers and two activation functions, where the activation function may be, for example, a ReLU function; the second hyper-prior model h s It can also include three convolutional layers and two activation functions, the activation function can be, for example, a ReLU function; the first super prior model h a and the second hyper-prior model h s is the hyper-prior model for predicting Gaussian variance σ, the first hyper-prior model h a It is a data compression model, through the first hyper-prior model h a Extract the useful information from the first latent variable y after downsampling, that is, the third latent variable w of the third latent space, the second hyper-prior model h s It is a data decompression model that can be obtained through the second hyper-prior model h s From the coded third latent variable To obtain the Gaussian variance σ, the first hyper-prior model h a and the second hyper-prior model h s It can predict the probability distribution of the entropy-reducing output, thereby improving the efficiency of arithmetic coding.

[0082] Among them, through the first hyper-prior model h a The downsampled second latent variable y=g is scaled according to the third scaling factor f3. a (x) is downsampled to obtain the third latent variable w=h in the third latent space a (y), where f3 can be 4, for example,

[0083] Among them, the quantized third latent variable is quantized by the arithmetic coding model Arithmetic coding may be performed using standard arithmetic coding, that is, there is no need to perform Gaussian arithmetic coding based on Gaussian variance.

[0084] According to the second latent variable y=g after downsampling a (x) Predict the first hyper-prior model h a The third latent variable w of the corresponding third latent space, the third latent space w∈Rh”×w'’×M , where w = h a (y), used to represent the third latent variable of the third latent space, the third latent variable w=h a (y) is quantified to obtain the quantified third latent variable The quantified third latent variable The third potential space The data in the arithmetic coding model is then used to quantize the third latent variable Estimating Gaussian variance

[0085] Optionally, the method further includes:

[0086] The coded third latent variable Transmitted to the decoding end device so that the decoding end device can transmit the encoded third latent variable Estimate Gaussian variance; and estimate the second latent variable according to the Gaussian variance Decode to obtain the decoded target image ^X.

[0087] Among them, the amount of data of Gaussian variance σ is much larger than w. This scheme uses the first super prior model h a and the second hyper-prior model h s To transmit w, thereby reducing the number of bits required to transmit the Gaussian variance σ. Specifically, the encoded third latent variable Transmitted to the decoding end device, the decoding end device can be based on the encoded third latent variable Estimate the Gaussian variance, so that when the decoding end device decompresses the data, the second latent variable can be calculated by the Gaussian variance σ Decode to obtain the decoded target image ^X.

[0088] Optionally, the method further includes:

[0089] After obtaining a first latent variable z of the first latent space through the variational autoencoder, transmitting the first latent variable z of the first latent space to a decoding end device;

[0090] On a decoding end device, decoding the first latent variable z of the first latent space by a variational autoencoder to obtain uncompressed image data of the first latent space;

[0091] The compressed second latent variable is obtained by using the uncompressed image data of the first latent space Perform compression quality analysis and obtain analysis results.

[0092] Among them, Figure 2As shown, in the decoding end device, the decoder (D) of the variational autoencoder can be used to decode the first latent variable z of the first latent space to obtain the uncompressed image data x of the first latent space. ’ , through the uncompressed image data x of the first latent space ’ The compressed image data Perform compression quality analysis and obtain analysis results to analyze compression quality.

[0093] Example 2

[0094] Consistent with the above embodiment, Figure 3 As shown, an embodiment of the present application proposes a potential space compression method, which is applied to an encoding end device. Figure 3 A flowchart of a potential space compression method provided in an embodiment of the present application, the method comprising:

[0095] 201. Obtain a first latent variable generated by the generative AI.

[0096] 202. Compress the first latent variable z using the neural network downsampling model to obtain a downsampled second latent variable.

[0097] 203. Perform quantization processing on the downsampled second latent variable to obtain a quantized second latent variable.

[0098] 204. Predict a third latent variable in a third latent space corresponding to the first super a priori model based on the downsampled second latent variable using the first super a priori model.

[0099] 205. Quantize the third latent variable in the third latent space to obtain a quantized third latent variable.

[0100] 206. Perform arithmetic coding on the quantized third latent variable through an arithmetic coding model to obtain a coded third latent variable.

[0101] 207. Estimate the Gaussian variance according to the encoded third latent variable through the second super prior model to obtain the Gaussian variance.

[0102] 208. Perform Gaussian arithmetic coding according to the compressed first latent variable and Gaussian variance using the arithmetic coding model to obtain a coded second latent variable.

[0103] 209. Transmit the encoded second latent variable to the decoding end device so that the decoding end device can decode the second latent variable. Decode and obtain the decoded target image.

[0104] By obtaining the first latent variable generated by generative AI; downsampling model g through neural network a Compress the first latent variable z to obtain the downsampled second latent variable y; quantize the downsampled second latent variable y to obtain the quantized second latent variable Through the first hyper-prior model h a Predict the third latent variable w of the third latent space corresponding to the first super prior model according to the downsampled second latent variable y; quantize the third latent variable w of the third latent space to obtain the quantized third latent variable The quantized third latent variable is coded by an arithmetic coding model. Perform arithmetic coding to obtain the encoded third latent variable Through the second hyper-prior model h s According to the coded third latent variable Estimate the Gaussian variance to obtain the Gaussian variance; calculate the Gaussian variance according to the compressed first latent variable through the arithmetic coding model Gaussian arithmetic coding is performed with Gaussian variance to obtain the encoded second latent variable The coded second latent variable is transmitted to the decoding end device so that the decoding end device can detect the second potential variable Decoding is performed to obtain the decoded target image. By adopting different scaling factors to generate different models with different bit rate distortion trade-offs, the combination of bit rate and distortion of the latent space (z) can be minimized, and the two losses of entropy of the latent space and the super priori latent space can be reduced. Entropy optimization is performed through Gaussian conditional arithmetic coding, and the latent space compression has very low perceptual distortion, which significantly improves the performance, can quickly compress image data, save computing resources, thereby improving data compression efficiency and saving computing costs.

[0105] Example 3

[0106] like Figure 4 As shown, an embodiment of the present application proposes a latent space decompression method, which is applied to a decoding end device. Figure 4 A flowchart of a latent space decompression method provided in an embodiment of the present application is provided, the method comprising:

[0107] 301. Receive a compressed second latent variable sent by an encoding end device.

[0108] 302. Decode the second latent variable using an arithmetic coding model to obtain a decoded second latent variable;

[0109] 303. The decoded second latent variable is processed by a neural network upsampling model. Decompress and obtain the first latent variable after upsampling;

[0110] 304. Decode the upsampled first latent variable using a variational autoencoder to obtain a target image.

[0111] Among them, Figure 2 As shown, at the decoding end device, the second latent variable can be coded by an arithmetic coding model. Decode and obtain the decoded second latent variable Then, the neural network upsampling model g s The decoded second latent variable Decompress and get the first latent variable after upsampling The first latent variable after upsampling is transformed into Decode and obtain the target image, thereby completing the decompression of the compressed data.

[0112] Optionally, the method further includes:

[0113] Receive the encoded third latent variable sent by the encoding end device

[0114] According to the coded third latent variable Estimate Gaussian variance;

[0115] The second latent variable is coded by an arithmetic coding model. Decode and obtain the decoded second latent variable include:

[0116] The second latent variable is coded according to the Gaussian variance by an arithmetic coding model. Decode and obtain the decoded second latent variable

[0117] By calculating the Gaussian variance σ, the Gaussian variance can be used to estimate the second latent variable Decode and obtain the decoded second latent variable

[0118] Optionally, the third latent variable after encoding Estimate Gaussian variance, including:

[0119] The encoded third latent variable is converted into Decode and obtain the decoded third latent variable

[0120] Through the second hyper-prior model h sAccording to the coded third latent variable Estimate the Gaussian variance and get the Gaussian variance.

[0121] Among them, the second hyper-prior model h s It is a data decompression model that can be obtained through the second hyper-prior model h s From the coded third latent variable To obtain the Gaussian variance σ, we use the predicted Gaussian variance in

[0122] Example 4

[0123] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a latent space compression device provided in an embodiment of the present application. An embodiment of the present application proposes a latent space compression device 1000, including an acquisition module 100, a compression module 200 and a sending module 300.

[0124] An acquisition module 100 is configured to acquire a first latent variable generated by generative AI;

[0125] A compression module 200 is configured to compress the first latent variable using a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is configured to compress the first latent variable to obtain a downsampled first latent variable, and the arithmetic coding model is configured to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain an encoded second latent variable;

[0126] The sending module 300 is configured to transmit the compressed second latent variable to a decoding end, so that the decoding end decodes the second latent variable to obtain a decoded target image.

[0127] The specific implementation steps of the latent space compression device 1000 may refer to the specific steps of the aforementioned latent space compression method, which will not be repeated here.

[0128] This solution obtains a first latent variable generated by generative AI; compresses the first latent variable through a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain the downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain the encoded second latent variable; the compressed second latent variable is transmitted to the decoding end so that the decoding end decodes the second latent variable to obtain a decoded target image, and lossless Gaussian arithmetic coding is performed on the downsampled first latent variable through the arithmetic coding model to achieve lossless compression, which can effectively utilize the structural information of the data to achieve efficient lossless compression, and can more efficiently compress the first latent variable, thereby improving the compression ratio of the image data and reducing the image data compression cost.

[0129] Example 5

[0130] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a spatial decompression device provided by an embodiment of the present application. An embodiment of the present application proposes a latent space decompression device 2000, including a receiving module 400 and a decoding module 500.

[0131] The receiving module 400 is configured to receive the compressed second latent variable sent by the encoding end;

[0132] The decoding module 500 is used to decode the second latent variable through an arithmetic coding model to obtain a decoded second latent variable; decompress the decoded second latent variable through a neural network upsampling model to obtain an upsampled first latent variable; and decode the upsampled first latent variable through a variational autoencoder to obtain a target image.

[0133] The specific implementation steps of the latent space decompression device 2000 may refer to the specific steps of the aforementioned latent space decompression method, which will not be repeated here.

[0134] Example 6

[0135] like Figure 7 As shown, Figure 7The encoding terminal device provided in the embodiments of the present application is characterized by including a processor 410 and a memory 420; and one or more programs, one or more programs being stored in the memory. The memory 420 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as a disk storage. The memory 4000 is used to store a set of program codes, and the processor 410 is used to call the program code stored in the memory 420 to perform the following operations:

[0136] Obtain the first latent variable generated by generative AI;

[0137] Compressing the first latent variable using a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain a downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain an encoded second latent variable;

[0138] The compressed second latent variable is transmitted to a decoding end, so that the decoding end decodes the second latent variable to obtain a decoded target image.

[0139] The specific implementation steps of the above-mentioned encoding end device can refer to the specific steps of the aforementioned latent space compression method, which will not be repeated here.

[0140] The implementation method of the present application obtains a first latent variable generated by generative AI; compresses the first latent variable through a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain the downsampled first latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain the encoded second latent variable; the compressed second latent variable is transmitted to the decoding end so that the decoding end decodes the second latent variable to obtain a decoded target image, and lossless Gaussian arithmetic coding is performed on the downsampled first latent variable through the arithmetic coding model to achieve lossless compression, which can effectively utilize the structural information of the data to achieve efficient lossless compression, and can more efficiently compress the first latent variable, thereby improving the compression ratio of the image data and reducing the image data compression cost.

[0141] Example 7

[0142] like Figure 8 As shown, Figure 8A decoding device provided in an embodiment of the present application is characterized by including a processor 510 and a memory 520; and one or more programs, one or more programs being stored in the memory. The memory 520 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as a disk storage. The memory 5000 is used to store a set of program codes, and the processor 510 is used to call the program code stored in the memory 420 to perform the following operations:

[0143] receiving the compressed second latent variable sent by the encoding end;

[0144] decoding the second latent variable through an arithmetic coding model to obtain a decoded second latent variable;

[0145] The decoded second latent variable is processed by a neural network upsampling model Decompress and obtain the first latent variable after upsampling;

[0146] The first latent variable after upsampling is transformed into Decode and get the target image.

[0147] The specific implementation steps of the above-mentioned decoding end device can refer to the specific steps of the aforementioned latent space decompression method, which will not be repeated here.

[0148] The embodiments of the present application further provide a computer-readable program product storing program instructions, which, when executed by a computer, enables the computer to perform the aforementioned latent space compression method or latent space decompression method.

[0149] The present application also provides a computer program product, typically stored on a computer-readable storage medium, that is operable to cause a computer to execute some or all of the steps described in any of the latent space compression methods or latent space decompression methods described in the present application. The computer program product may be a software installation package.

[0150] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other changes to the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0151] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices (equipment), or computer program products. Therefore, the application can adopt the form of complete hardware embodiment, complete software embodiment, or the embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as a part for hardware, or other distribution forms can be adopted, such as by the Internet or other wired or wireless telecommunication systems.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (devices) and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable human-vehicle trajectory analysis device to produce a machine, so that the instructions executed by the processor of the computer or other programmable human-vehicle trajectory analysis device generate instructions for implementing the process Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable human-vehicle trajectory analysis device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable human-vehicle trajectory analysis device, so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0155] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

Claims

1. A latent space compression method, characterized in that: Applied to an encoding end device, the method comprises: Obtain the first latent variable generated by the generative AI; The first latent variable is compressed by a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used to compress the first latent variable to obtain a downsampled second latent variable; the arithmetic coding model is used to perform lossless Gaussian arithmetic coding on the downsampled second latent variable to obtain an encoded second latent variable; The compressed second latent variable is transmitted to a decoding end device, so that the decoding end decodes the second latent variable to obtain a decoded target image.

2. The latent space compression method according to claim 1, characterized in that: The first potential variable generated by the generative AI is obtained, comprising Get the initial image; The initial image is downsampled according to a first scaling factor by a variational autoencoder to obtain the first latent variable of a first latent space.

3. The latent space compression method according to claim 1, characterized in that: The step of compressing the first latent variable by using a neural network compression model to obtain a compressed second latent variable includes: Compressing the first latent variable through the neural network downsampling model to obtain a downsampled second latent variable; Quantizing the downsampled second latent variable to obtain a quantized second latent variable; The quantized second latent variable is subjected to lossless Gaussian arithmetic coding through the arithmetic coding model to obtain an encoded second latent variable.

4. The latent space compression method according to claim 3, characterized in that: The step of performing lossless Gaussian arithmetic coding on the quantized first latent variable through the arithmetic coding model to obtain the encoded second latent variable comprises: Estimate the Gaussian variance of the downsampled second latent variable through an arithmetic coding model to obtain a Gaussian variance; The arithmetic coding model is used to perform Gaussian arithmetic coding according to the compressed first latent variable and the Gaussian variance to obtain an encoded second latent variable.

5. The latent space compression method according to claim 4, characterized in that: The neural network compression model includes a first super prior model and a second super prior model, and estimates the Gaussian variance of the downsampled first latent variable through an arithmetic coding model to obtain the Gaussian variance, including: Predicting a third latent variable of a third latent space corresponding to the first super prior model according to the downsampled second latent variable by using the first super prior model; quantizing the third latent variable of the third latent space to obtain a quantized third latent variable; Performing arithmetic coding on the quantized third latent variable through an arithmetic coding model to obtain a coded third latent variable; The Gaussian variance is estimated according to the encoded third latent variable through the second hyper-prior model to obtain the Gaussian variance.

6. The latent space compression method according to claim 5, characterized in that: The method further comprises: The encoded third latent variable is transmitted to a decoding end device so that the decoding end device estimates the Gaussian variance according to the encoded third latent variable; and the second latent variable is decoded according to the Gaussian variance to obtain a decoded target image.

7. A latent space decompression method, characterized in that: Applied to a decoding end device, the method comprises: receiving a compressed second latent variable sent by an encoding end device; Decoding the second latent variable through an arithmetic coding model to obtain a decoded second latent variable; Decompressing the decoded second latent variable through a neural network upsampling model to obtain an upsampled first latent variable; The upsampled first latent variable is decoded by a variational autoencoder to obtain a target image.

8. The latent space decompression method according to claim 7, characterized in that: The method further comprises: receiving the encoded third latent variable sent by the encoding end device; estimating a Gaussian variance based on the encoded third latent variable; The second latent variable is decoded by an arithmetic coding model to obtain a decoded second latent variable. include: The second latent variable is decoded according to the Gaussian variance through an arithmetic coding model to obtain a decoded second latent variable.

9. The latent space decompression method according to claim 8, characterized in that: The estimating the Gaussian variance according to the encoded third latent variable comprises: Decoding the encoded third latent variable through an arithmetic coding model to obtain a decoded third latent variable; The Gaussian variance is estimated according to the encoded third latent variable through the second hyper-prior model to obtain the Gaussian variance.

10. A potential space compression device, characterized in that: include: An acquisition module, used to acquire a first latent variable generated by the generative AI; A compression module, used for compressing the first latent variable through a neural network compression model to obtain a compressed second latent variable, wherein the neural network compression model includes a neural network downsampling model and an arithmetic coding model, the neural network downsampling model is used for compressing the first latent variable to obtain a downsampled first latent variable; the arithmetic coding model is used for performing lossless Gaussian arithmetic coding on the downsampled first latent variable to obtain an encoded second latent variable; The sending module is used to transmit the compressed second latent variable to the decoding end, so that the decoding end decodes the second latent variable to obtain a decoded target image.

11. A latent space decompression device, characterized in that: include: A receiving module, used for receiving the compressed second latent variable sent by the encoding end; A decoding module is used to decode the second latent variable through an arithmetic coding model to obtain a decoded second latent variable; decompress the decoded second latent variable through a neural network upsampling model to obtain an upsampled first latent variable; and decode the upsampled first latent variable through a variational autoencoder to obtain a target image.

12. A coding end device, characterized in that: It comprises a processor and a memory; and one or more programs, which are stored in the memory and configured to be executed by the processor, and the programs include steps for the potential space compression method as described in any one of claims 1 to 6.

13. A decoding end device, characterized in that: Comprising a processor and a memory; and one or more programs, the one or more programs being stored in the memory and configured to be executed by the processor, the programs comprising steps for a latent space decompression method as described in any one of claims 7 to 9.

14. A computer program product, characterized in that When the computer program in the computer program product is executed by a processor, it can implement the steps of the latent space compression method described in any one of claims 1 to 6 or the steps of the latent space decompression method described in any one of claims 7 to 9.

Images