A data compression method and related device

Abstract

The present application relates to the field of artificial intelligence and discloses a data compression method, comprising: obtaining first target data, the first target data including first sub-data and second sub-data; obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, the first probability distribution being used as a conditional probability distribution for the second sub-data; compressing the second sub-data based on the first probability distribution through an entropy encoder to obtain a first bit stream; and compressing the first sub-data into the first bit stream to obtain a second bit stream. Compared to the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiments of the present application do not require additional initial bits, can achieve compression of a single data point, and reduce the compression ratio during parallel compression.

Description

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 14, 2022, with application number 202210249906.1 and invention name “A data compression method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a data compression method and related equipment. Background Art

[0003] Multimedia data now accounts for the vast majority of Internet traffic. Image data compression plays a crucial role in its storage and efficient transmission. Therefore, image coding is a technology with significant practical value.

[0004] Image coding research has a long history. Researchers have proposed numerous methods and established numerous international standards, such as JPEG, JPEG2000, WebP, and BPG. While these methods are widely used, they are currently experiencing limitations in response to the ever-increasing amount of image data and the emergence of new media types.

[0005] AI-based lossless compression schemes leverage the ability of deep generative models to more accurately estimate the probability distribution of data than traditional approaches, achieving compression ratios far superior to those of traditional lossless compression schemes. Among AI-based lossless compression schemes, widely used deep generative models include autoregressive models, variational auto-encoders (VAEs), and normalizing flows. Generally speaking, autoregressive models are well compatible with arithmetic encoders and Huffman codes; VAEs, combined with a bits-back mechanism, are well compatible with asymmetric digital systems; and flow models are compatible with the three aforementioned entropy encoders. In addition to compression ratio, throughput is also a metric used to evaluate lossless compression solutions. AI-based lossless compression solutions, due to their significantly larger model size, exhibit lower overall throughput than traditional solutions. Furthermore, considering both compression ratio and throughput, there is currently no absolute superiority among lossless compression solutions based on different generative models. Current research is still at the stage of exploring the Pareto frontier of compression schemes based on different generative models.

[0006] Among them, unlike full-observation models (such as autoregressive models), the variational autoencoder model is a latent variable model. This type of model does not directly model the data itself, but instead introduces one (or more) latent variables, and then models the prior distribution, likelihood function, and approximate posterior distribution. Since the marginal distribution of the data cannot be directly obtained from the variational autoencoder, the traditional entropy coding method cannot be directly used. In order to use the variational autoencoder for lossless data compression, a variational autoencoder lossless compression scheme based on the inverse coding mechanism was proposed. Bits-back ANS is the original form of this scheme, which is suitable for variational autoencoder models containing only one latent variable and can be generalized to variational autoencoder models containing multiple latent variables.

[0007] Existing lossless compression schemes for variational autoencoders based on inverse encoding mechanisms all require additional initial bits to decompress the latent variable samples. These additional initial bits are randomly generated data, and their size must be factored into the compression cost. The additional average cost is high when the number of data points to be compressed serially is small. Furthermore, because the required additional initial bits are proportional to the number of data points to be compressed, efficient parallel compression is not possible. Summary of the Invention

[0008] The present application provides a data compression method. Compared with the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiments of the present application do not require additional initial bits, can achieve compression of a single data point, and greatly reduce the compression ratio during parallel compression.

[0009] In a first aspect, the present application provides a data compression method, comprising: acquiring first target data, where the first target data includes first sub-data and second sub-data;

[0010] In a possible implementation, the first target data may be image data for compression or other data (such as text, video, binary stream, etc.).

[0011] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or,

[0012] The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or

[0013] The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or

[0014] The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

[0015] In one possible implementation, the first sub-data and the second sub-data are obtained by segmenting the image block in a spatial dimension or a channel dimension, wherein the image data includes one channel dimension (C) and two spatial dimensions (width W and height H).

[0016] According to the first sub-data, a first probability distribution is obtained through a first decoder of a variational autoencoder, and the first probability distribution is used as a conditional probability distribution of the second sub-data; according to the first probability distribution, the second sub-data is compressed through an entropy encoder to obtain a first bit stream; using the first bit stream as an initial bit stream, the first sub-data is compressed (that is, the first sub-data is compressed to the first bit stream) to obtain a second bit stream.

[0017] Compared with the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiments of the present application do not require additional initial bits, can achieve compression of a single data point, and greatly reduce the compression ratio during parallel compression.

[0018] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block.

[0019] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0020] In one possible implementation, the variational autoencoder may include a variational encoder, a decoder (such as the first decoder and the second decoder in the embodiment of the present application), and a prior distribution of latent variables.

[0021] In one possible implementation, the decoder can be composed of decoder layers (e.g., the first convolutional neural network and the second convolutional neural network in the embodiment of the present application), and the number of decoder layers is the same as the number of latent variables in the variational autoencoder. The decoder layer is used to input the latent variables of the deeper layer and output the conditional probability distribution of the current layer data (the current layer data can be the latent variables of the shallower layer or the data data).

[0022] In the existing variational autoencoder model, the variational encoder needs to input the entire data to predict the approximate posterior distribution of the latent variable, and the latent variable is input into the decoder to directly predict the conditional probability distribution of the entire data. In an embodiment of the present application, the data to be compressed is divided into at least two parts, namely: the first sub-data and the second sub-data. Unlike the existing method of inputting all the data into the variational encoder, in the embodiment of the present application, only a part of the data (the first sub-data) is input into the variational encoder to predict the approximate posterior distribution of the latent variable, and the conditional probability distribution of the first sub-data is predicted after the latent variable is input into the first decoder; the conditional probability distribution of the second sub-data depends on the first sub-data, which can be specifically determined by inputting the first sub-data into the first decoder.

[0023] In one possible implementation, the decoder may implement a pixel reset operation.

[0024] In one possible implementation, the first decoder may include a first convolutional neural network and a second convolutional neural network. Obtaining the first probability distribution based on the first sub-data through the first decoder of the variational autoencoder may specifically include: performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, where the second target data and the first target data have the same size, and the third sub-data and the first sub-data have the same size in the spatial dimension.

[0025] In which, the second target data including the second sub-data can be data of the same size as the first target data, wherein, in the first target data, elements other than the second sub-data can be set to zero (or other preset values) to obtain the second target data, and after the second target data is subjected to a pixel reset operation, it can be converted into a third sub-data having the same size as the first sub-data in the spatial dimension.

[0026] The embodiment of the present application fully utilizes the correlation between image pixels by using an encoder layer with an autoregressive structure defined based on channel-priority pixel reset, thereby significantly reducing the number of parameters required for the model while obtaining a lower coding length, thereby improving the compression throughput and reducing the space cost of model storage.

[0027] In one possible implementation, fourth sub-data can be obtained based on the first sub-data through the first convolutional neural network, and the fourth sub-data has the same size as the third sub-data in the channel dimension. In other words, the first sub-data can be subjected to feature extraction and size transformation through the first convolutional neural network to obtain fourth sub-data having the same size as the third Hull sub-data in the channel dimension.

[0028] In a possible implementation, the third sub-data and the fourth sub-data may be fused to obtain fused sub-data. Optionally, the fusion method may be data replacement of corresponding channels.

[0029] In a possible implementation, the fusing of the third sub-data and the fourth sub-data may specifically include: replacing data of some channels in the fourth sub-data with data of corresponding channels in the third sub-data to obtain fused sub-data.

[0030] In a possible implementation, the first probability distribution can be obtained by passing the fused sub-data through the second convolutional neural network.

[0031] In a possible implementation, the fused sub-data and the first sub-data may be concatenated along the channel dimension to obtain concatenated sub-data; and further, obtaining the first probability distribution through the second convolutional neural network based on the fused sub-data may specifically include obtaining the first probability distribution through the second convolutional neural network based on the concatenated sub-data.

[0032] In a second aspect, the present application provides a data decompression method, comprising:

[0033] Obtaining a second bitstream;

[0034] decoding the first sub-data from the second bit stream to obtain a first bit stream;

[0035] Obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data;

[0036] According to the first probability distribution, second sub-data are decompressed from the first bit stream through an entropy encoder; the first sub-data and the second sub-data are used to restore first target data.

[0037] In a possible implementation, decoding the first sub-data from the second bit stream to obtain the first bit stream includes:

[0038] Obtain the prior distribution of latent variables;

[0039] decompressing the latent variable from the second bitstream using an entropy encoder according to the prior distribution to obtain a fourth bitstream;

[0040] According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0041] decompressing the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream;

[0042] Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational self-decoder according to the first sub-data;

[0043] According to the approximate posterior distribution, the latent variable is compressed into the third bit stream by the entropy encoder to obtain a first bit stream.

[0044] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or,

[0045] The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or

[0046] The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or

[0047] The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

[0048] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0049] In one possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and obtaining the first probability distribution through the first decoder of the variational autoencoder according to the first sub-data includes:

[0050] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0051] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0052] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0053] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0054] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0055] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0056] In one possible implementation, the method further includes:

[0057] Performing a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0058] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0059] In a third aspect, the present application provides a data compression device, comprising:

[0060] An acquisition module, configured to acquire first target data, where the first target data includes first sub-data and second sub-data;

[0061] a compression module, configured to obtain, based on the first sub-data, a first probability distribution through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution of the second sub-data;

[0062] compressing the second sub-data using an entropy encoder according to the first probability distribution to obtain a first bitstream;

[0063] The first sub-data is compressed into the first bit stream to obtain a second bit stream.

[0064] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or,

[0065] The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or

[0066] The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or

[0067] The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

[0068] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0069] In a possible implementation, the compression module is specifically configured to:

[0070] Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational self-decoder according to the first sub-data;

[0071] Decoding the latent variable from the first bit stream using the entropy encoder according to the approximate posterior distribution to obtain a third bit stream;

[0072] According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0073] compressing the first sub-data into the third bit stream by the entropy encoder according to the second probability distribution to obtain a fourth bit stream;

[0074] According to the prior distribution of the latent variable, the latent variable is compressed into the fourth bit stream by the entropy encoder to obtain a second bit stream.

[0075] In one possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and the compression module is specifically configured to:

[0076] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0077] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0078] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0079] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0080] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0081] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0082] In a possible implementation, the apparatus further includes:

[0083] a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0084] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0085] In a fourth aspect, the present application provides a data decompression device, comprising:

[0086] An acquisition module, configured to acquire a second bit stream;

[0087] a decompression module, configured to decode the first sub-data from the second bit stream to obtain a first bit stream;

[0088] Obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data;

[0089] According to the first probability distribution, second sub-data are decompressed from the first bit stream through an entropy encoder; the first sub-data and the second sub-data are used to restore first target data.

[0090] In a possible implementation, the code receiving module is specifically configured to:

[0091] Obtain the prior distribution of latent variables;

[0092] decompressing the latent variable from the second bitstream using an entropy encoder according to the prior distribution to obtain a fourth bitstream;

[0093] According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0094] decompressing the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream;

[0095] Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational self-decoder according to the first sub-data;

[0096] According to the approximate posterior distribution, the latent variable is compressed into the third bit stream by the entropy encoder to obtain a first bit stream.

[0097] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or,

[0098] The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or

[0099] The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or

[0100] The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

[0101] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0102] In a possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and the decompression module is specifically configured to:

[0103] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0104] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0105] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0106] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0107] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0108] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0109] In a possible implementation, the apparatus further includes:

[0110] a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0111] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0112] In a fifth aspect, the present application provides a data compression device, comprising a storage medium, a processing circuit, and a bus system; wherein the storage medium is used to store instructions, and the processing circuit is used to execute instructions in the memory to execute any data compression method described in the first aspect above.

[0113] In a sixth aspect, the present application provides a data compression device comprising a storage medium, a processing circuit, and a bus system; wherein the storage medium is used to store instructions, and the processing circuit is used to execute instructions in the memory to execute any data compression method described in the second aspect above.

[0114] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes any data compression method described in the first aspect.

[0115] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data compression method described in any one of the second aspects above.

[0116] In a ninth aspect, an embodiment of the present application provides a computer program, which, when executed on a computer, enables the computer to execute any of the data compression methods described in the first aspect.

[0117] In a tenth aspect, an embodiment of the present application provides a computer program, which, when executed on a computer, enables the computer to execute any of the data compression methods described in the second aspect.

[0118] In the eleventh aspect, the present application provides a chip system, which includes a processor for supporting an execution device (such as a data compression device or a data decompression device) or a training device to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above method. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the execution device or training device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0119] An embodiment of the present application provides a data compression method, comprising: obtaining first target data, the first target data including first sub-data and second sub-data; obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, the first probability distribution being used as a conditional probability distribution for the second sub-data; compressing the second sub-data based on the first probability distribution through an entropy encoder to obtain a first bit stream; and compressing the first sub-data into the first bit stream to obtain a second bit stream. Compared to the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiment of the present application does not require additional initial bits, can achieve compression of a single data point, and reduces the compression ratio during parallel compression. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 A structural diagram of the main framework of artificial intelligence;

[0121] Figure 2 This is an illustration of the application scenario of the embodiment of the present application;

[0122] Figure 3 This is an illustration of the application scenario of the embodiment of the present application;

[0123] Figure 4 This is a diagram of a data processing process based on CNN;

[0124] Figure 5 This is a diagram of a data processing process based on CNN;

[0125] Figure 6 An example of a system architecture method provided in an embodiment of the present application is shown;

[0126] Figure 7 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0127] Figure 8 A schematic diagram of a data compression method provided in an embodiment of the present application;

[0128] Figure 9This is a schematic diagram of an embodiment of a pixel replacement operation provided in an embodiment of the present application;

[0129] Figure 10 A processing flow diagram of a decoder provided in an embodiment of the present application;

[0130] Figure 11 A schematic diagram of the structure of the decoder provided in the embodiment of the present application;

[0131] Figure 12 A schematic diagram of a data compression method provided in an embodiment of the present application;

[0132] Figure 13 A schematic diagram of a data compression method provided in an embodiment of the present application;

[0133] Figure 14 A schematic diagram of a data decompression method provided in an embodiment of the present application;

[0134] Figure 15 A schematic diagram of the structure of a data compression device provided in an embodiment of the present application;

[0135] Figure 16 A schematic diagram of the structure of a data decompression device provided in an embodiment of the present application;

[0136] Figure 17 A schematic diagram of the structure of the execution device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0137] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0138] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0139] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0140] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 The following diagram illustrates a structural diagram of the AI framework. This framework is explained below from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it encompasses the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed progression from "data-information-knowledge-wisdom." The "IT value chain," encompassing the entire process from the underlying infrastructure of human intelligence, information (provided and processed by technology), to the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0141] (1) Infrastructure

[0142] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0143] (2) Data

[0144] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0145] (3) Data processing

[0146] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0147] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0148] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0149] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0150] (4) General ability

[0151] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0152] (5) Smart products and industry applications

[0153] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0154] This application can be applied to the field of data compression in the field of artificial intelligence. The following will introduce multiple application scenarios of multiple products.

[0155] 1. Image Compression Process Applied to Terminal Devices

[0156] The image compression method provided in the embodiment of the present application can be applied to the image compression process in the terminal device, specifically, it can be applied to the photo album, video monitoring, etc. on the terminal device. Figure 2 , Figure 2 This is an illustration of the application scenario of the embodiment of the present application, such as Figure 2As shown in , the terminal device can obtain the image to be compressed, wherein the image to be compressed can be a photo taken by a camera or a frame captured from a video. The terminal device can extract features of the acquired image to be compressed through the artificial intelligence (AI) coding unit in the embedded neural network (neural-network processing unit, NPU), transform the image data into output features with lower redundancy, and generate probability estimates of each point in the output features. The central processing unit (CPU) performs arithmetic coding on the extracted output features through the probability estimates of each point in the output features, reduces the coding redundancy of the output features, further reduces the amount of data transmitted during the image compression process, and saves the encoded data in the form of a data file in the corresponding storage location. When the user needs to obtain the file saved in the above storage location, the CPU can obtain and load the above-mentioned saved file in the corresponding storage location, and obtain the decoded feature map based on arithmetic decoding, and reconstruct the feature map through the AI decoding unit in the NPU to obtain a reconstructed image.

[0157] 2. Image Compression Process on the Cloud Side

[0158] The image compression method provided in the embodiment of the present application can be applied to the image compression process on the cloud side, and specifically, can be applied to functions such as cloud albums on the cloud side server. Figure 3 , Figure 3 This is an illustration of the application scenario of the embodiment of the present application, such as Figure 3As shown in , the terminal device can obtain the image to be compressed, wherein the image to be compressed can be a photo taken by a camera or a frame captured from a video. The terminal device can perform lossless coding compression on the image to be compressed through the CPU to obtain coded data, for example but not limited to any lossless compression method based on the prior art. The terminal device can transmit the coded data to the server on the cloud side, and the server can perform corresponding lossless decoding on the received coded data to obtain the image to be compressed. The server can extract features of the obtained image to be compressed through the AI coding unit in the graphics processing unit (GPU), transform the image data into output features with lower redundancy, and generate probability estimates of each point in the output features. The CPU performs arithmetic coding on the extracted output features through the probability estimates of each point in the output features, reduces the coding redundancy of the output features, further reduces the amount of data transmission in the image compression process, and saves the coded data obtained by coding in the form of a data file in the corresponding storage location. When the user needs to obtain the file saved in the above storage location, the CPU can obtain and load the above saved file in the corresponding storage location, and obtain the decoded feature map based on arithmetic decoding, and reconstruct the feature map through the AI decoding unit in the NPU to obtain a reconstructed image. The server can use the CPU to perform lossless encoding on the compressed image to obtain encoded data, such as but not limited to any lossless compression method based on the existing technology. The server can transmit the encoded data to the terminal device, and the terminal device can perform corresponding lossless decoding on the received encoded data to obtain the decoded image.

[0159] Since the embodiments of the present application involve the application of a large number of neural networks, for ease of understanding, the relevant terms and concepts of the neural networks that may be involved in the embodiments of the present application are first introduced below.

[0160] (1) Neural Network

[0161] A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be:

[0162]

[0163] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of Xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0164] (2) Deep Neural Networks

[0165] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. Based on the location of the different layers, the neural network within a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. Each layer is fully connected, meaning that any neuron in layer i is connected to any neuron in layer i+1.

[0166] Although DNN looks complicated, the work of each layer is actually not complicated. In simple terms, it can be expressed as the following linear relationship: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The number of these parameters is also relatively large. The definitions of these parameters in DNN are as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0167] In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as

[0168] It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0169] (3) Convolutional neural network (CNN) is a deep neural network with a convolutional structure. Convolutional neural network contains a feature extractor composed of convolution layer and subsampling layer, which can be regarded as a filter. Convolution layer refers to the neuron layer in convolutional neural network that performs convolution processing on the input signal. In the convolution layer of convolutional neural network, a neuron can only be connected to some neurons in the adjacent layer. A convolution layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way of extracting features is independent of position. Convolution kernels can be formalized as matrices of random size, and during the training process of convolutional neural network, convolution kernels can obtain reasonable weights through learning. In addition, the direct benefit of shared weights is to reduce the connections between the layers of convolutional neural network, while reducing the risk of overfitting.

[0170] CNN is a very common neural network. Figure 4 This section focuses on a detailed introduction to the structure of CNNs. As mentioned in the previous basic concepts, a convolutional neural network is a deep neural network with a convolutional structure and a deep learning architecture. Deep learning architectures use machine learning algorithms to perform multiple levels of learning at different levels of abstraction. As a deep learning architecture, a CNN is a feed-forward artificial neural network in which individual neurons respond to input images.

[0171] like Figure 4 As shown, the convolutional neural network (CNN) 200 may include an input layer 210 , a convolutional layer / pooling layer 220 (wherein the pooling layer is optional), and a fully connected layer 230 .

[0172] Convolutional layer / pooling layer 220:

[0173] Convolutional layer:

[0174] like Figure 4 The convolutional layer / pooling layer 220 shown may include layers 221-226. For example, in one implementation, layer 221 is a convolutional layer, layer 222 is a pooling layer, layer 223 is a convolutional layer, layer 224 is a pooling layer, layer 225 is a convolutional layer, and layer 226 is a pooling layer. In another implementation, layers 221 and 222 are convolutional layers, layer 223 is a pooling layer, layers 224 and 225 are convolutional layers, and layer 226 is a pooling layer. That is, the output of a convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

[0175] The following will take convolutional layer 221 as an example to introduce the internal working principle of a convolutional layer.

[0176] Convolution layer 221 can include multiple convolution operators, also known as kernels. In image processing, a convolution operator acts as a filter that extracts specific information from an input image matrix. A convolution operator is essentially a weight matrix, which is typically predefined. During the convolution operation, the weight matrix is typically applied horizontally to the input image, pixel by pixel (or two pixels by two pixels, depending on the stride), thereby extracting specific features from the image. The size of the weight matrix should be related to the image size. It is important to note that the depth dimension of the weight matrix is the same as the depth dimension of the input image. During the convolution operation, the weight matrix extends across the entire depth of the input image. Therefore, convolution with a single weight matrix produces a convolution output with a single depth dimension. However, in most cases, a single weight matrix is not used. Instead, multiple weight matrices of the same size (rows × columns) are applied, i.e., multiple homogeneous matrices. The outputs of each weight matrix are stacked to form the depth dimension of the convolved image, where the dimension is determined by the "multiple" mentioned above. Different weight matrices can be used to extract different features from an image. For example, one weight matrix can be used to extract edge information, another weight matrix can be used to extract specific colors, and yet another weight matrix can be used to blur unwanted noise in the image. The weight matrices have the same size (rows × columns), and the feature maps extracted by these weight matrices are also the same size. The extracted feature maps of the same size are then merged to form the output of the convolution operation.

[0177] The weight values in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions.

[0178] When the convolutional neural network 200 has multiple convolutional layers, the initial convolutional layer (for example, 221) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 increases, the features extracted by the later convolutional layers (for example, 226) become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0179] Pooling layer:

[0180] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer, such as Figure 4 Each layer 221-226 illustrated in Figure 220 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In the image processing process, the sole purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer can include an average pooling operator and / or a maximum pooling operator to sample the input image to obtain a smaller image. The average pooling operator can calculate the pixel values in the image within a specific range to produce an average value as the result of average pooling. The maximum pooling operator can take the pixel with the largest value within a specific range as the result of maximum pooling. In addition, just as the size of the weight matrix used in the convolutional layer should be related to the image size, the operators in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer can be smaller than the size of the image input to the pooling layer. Each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0181] Fully connected layer 230:

[0182] After being processed by the convolution layer / pooling layer 220, the convolution neural network 200 is still not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 220 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 200 needs to use the fully connected layer 230 to generate one or a group of outputs of the required number of classes. Therefore, the fully connected layer 230 may include multiple hidden layers (such as Figure 4 As shown in 231, 232 to 23n), the parameters contained in the multi-layer hidden layer can be pre-trained based on relevant training data of a specific task type, for example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0183] After the multiple hidden layers in the fully connected layer 230, the last layer of the entire convolutional neural network 200 is the output layer 240, which has a loss function similar to the classification cross entropy, specifically for calculating the prediction error. Once the forward propagation of the entire convolutional neural network 200 (such as Figure 4 The propagation from 210 to 240 is forward propagation) and the reverse propagation (such as Figure 4 The propagation from 240 to 210 is called back propagation) and then the weight values and biases of the aforementioned layers will be updated to reduce the loss of the convolutional neural network 200 and the error between the result output by the convolutional neural network 200 through the output layer and the ideal result.

[0184] It should be noted that if Figure 4 The convolutional neural network 200 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, for example, only including Figure 4 For example, the convolutional neural network used in the embodiment of the present application may only include the input layer 210, the convolution layer / pooling layer 220 and the output layer 240.

[0185] It should be noted that if Figure 4 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, such as Figure 5 The multiple convolutional layers / pooling layers shown are operated in parallel, and the features extracted from each layer are input to the fully connected layer 230 for processing.

[0186] (4) Loss function

[0187] During the training of a deep neural network, because we want the output of the deep neural network to be as close as possible to the desired value, we can compare the current network's predicted value with the desired value and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the deep neural network.) For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the deep neural network can predict the desired value or a value very close to the desired value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the actual value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the actual value. For example, the loss function output value (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss.

[0188] (5) Backpropagation algorithm

[0189] Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, the forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0190] (6) Lossless compression: A technique for compressing data so that the length of the compressed data is less than the original data. When the compressed data is decompressed, the recovered data must be exactly the same as the original data.

[0191] (7)Compressed length: the storage space occupied by the compressed data.

[0192] (8) Compression ratio: The ratio of the original data length to the compressed data length. If there is no compression, the value is 1. The larger the value, the better.

[0193] (9) Bits per dimension: The average bit length of each dimension (byte) of the compressed data. The calculation formula is: 8 / compression ratio. If there is no compression, this value is 8. The smaller the value, the better.

[0194] (10) Throughput: The average amount of data processed per second.

[0195] (11) Hidden variable: A type of data with a specific probability distribution. By establishing the conditional probability of these data and the original data, the probability distribution of the original data can be obtained.

[0196] (12) Encoding / Decoding: The process of compressing data is encoding, and the process of decompressing data is decoding.

[0197] (13) Inverse encoding: A special encoding technique that uses additional binary data stored in the system to generate specific data through decoding.

[0198] The following combination Figure 6 The system architecture provided in the embodiments of the present application is introduced in detail. Figure 6 This is a schematic diagram of the system architecture provided by an embodiment of the present application. Figure 6 As shown, the system architecture 500 includes an execution device 510 , a training device 520 , a database 530 , a client device 540 , a data storage system 550 , and a data collection system 560 .

[0199] The execution device 510 includes a calculation module 511, an I / O interface 512, a pre-processing module 513, and a post-processing module 514. The calculation module 511 may include the target model / rule 501, and the pre-processing module 513 and the post-processing module 514 are optional.

[0200] As an example, the execution device 510 can be a mobile phone, a tablet, a laptop computer, a smart wearable device, etc., and the terminal device can compress the acquired image. As another example, the terminal device can be a virtual reality (VR) device. As another example, the embodiment of the present application can also be applied to intelligent monitoring, and a camera can be configured in the intelligent monitoring, so that the intelligent monitoring can obtain the image to be compressed through the camera, etc. It should be understood that the embodiment of the present application can also be applied to other scenarios requiring image compression, and other application scenarios are not listed here one by one.

[0201] The data collection device 560 is used to collect training data. After collecting the training data, the data collection device 560 stores the training data in the database 530. The training device 520 trains the target model / rule 501 based on the training data maintained in the database 530.

[0202] The above-mentioned target model / rule 501 (such as the variational autoencoder, entropy encoder, etc. in the embodiment of the present application) can be used to implement data compression and decompression tasks, that is, the data to be processed (such as the first target data in the embodiment of the present application) is input into the target model / rule 501, and the compressed data (such as the second bit stream in the embodiment of the present application) can be obtained. It should be noted that in actual applications, the training data maintained in the database 530 does not necessarily all come from the collection of the data acquisition device 560, and may also be received from other devices. It should also be noted that the training device 520 does not necessarily train the target model / rule 501 entirely based on the training data maintained by the database 530, and may also obtain training data from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0203] The target model / rule 501 obtained by training the training device 520 can be applied to different systems or devices, such as Figure 6 The execution device 510 shown in the figure can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an augmented reality (AR) / virtual reality (VR) device, a vehicle terminal, etc. It can also be a server or a cloud. Figure 6 In the embodiment, the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with an external device. A user can input data to the I / O interface 512 through a client device 540 .

[0204] Preprocessing module 513 and preprocessing module 514 are used to preprocess the input data received by I / O interface 512. It should be understood that preprocessing module 513 and preprocessing module 514 may be absent or only one preprocessing module may be present. If preprocessing module 513 and preprocessing module 514 are absent, computing module 511 may be used directly to process the input data.

[0205] When the execution device 510 preprocesses the input data, or when the computing module 511 of the execution device 510 performs calculations and other related processing, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 550.

[0206] Finally, the I / O interface 512 presents the processing results to the client device 540 for providing to the user.

[0207] exist Figure 6In the illustrated case, the user can manually input data, and this "manual input data" can be operated through the interface provided by I / O interface 512. In another case, client device 540 can automatically send input data to I / O interface 512. If the automatic transmission of input data by client device 540 requires user authorization, the user can set the corresponding permissions in client device 540. The user can view the results output by execution device 510 on client device 540, and the specific presentation form can be a display, sound, action, etc. Client device 540 can also serve as a data acquisition terminal, collecting input data input into I / O interface 512 and output results from I / O interface 512 as new sample data and storing them in database 530. Of course, collection can also be performed without client device 540, and instead the I / O interface 512 directly stores the input data input into I / O interface 512 and output results from I / O interface 512 as new sample data in database 530.

[0208] It is worth noting that Figure 6 This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 6 In the embodiment, the data storage system 550 is an external memory relative to the execution device 510. In other cases, the data storage system 550 can also be placed in the execution device 510.

[0209] The following describes a chip hardware structure provided by an embodiment of the present application.

[0210] Figure 7 This is a chip hardware structure diagram provided in an embodiment of the present application, which includes a neural network processor 700. The chip can be set as follows Figure 6 The execution device 510 shown in FIG. 5 is used to complete the calculation work of the calculation module 511. The chip can also be set in Figure 6 The training device 520 shown in FIG. 5 is used to complete the training work of the training device 520 and output the target model / rule 501. Figure 6 The algorithms of each layer in the image processing model shown can be Figure 7 is implemented in the chip shown.

[0211] The neural processing unit (NPU) 700 is mounted as a coprocessor on the host central processing unit (CPU), with tasks assigned by the host CPU. The core of the NPU is the arithmetic circuit 703, which is controlled by a controller 704 to extract data from memory (weight memory 702 or input memory 701) and perform calculations.

[0212] In some implementations, the arithmetic circuit 703 includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 703 is a two-dimensional systolic array. The arithmetic circuit 703 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 703 is a general-purpose matrix processor.

[0213] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. Operation circuit 703 retrieves the corresponding data of matrix B from weight memory 702 and caches it on each PE in operation circuit 703. Operation circuit 703 retrieves the data of matrix A from input memory 701 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 708.

[0214] The vector calculation unit 707 can further process the output of the operation circuit 703, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector calculation unit 707 can be used for network calculations of non-convolutional / non-FC layers in a neural network, such as pooling, batch normalization, local response normalization, etc.

[0215] In some implementations, the vector calculation unit 707 can store the processed output vector to the unified memory 706. For example, the vector calculation unit 707 can apply a nonlinear function to the output of the operation circuit 703, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 707 generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 703, for example, for use in a subsequent layer in a neural network.

[0216] The unified memory 706 is used to store input data and output data.

[0217] The weight data is directly transferred from the external memory to the input memory 701 and / or the unified memory 706 through the direct memory access controller (DMAC) 705, the weight data in the external memory is stored in the weight memory 702, and the data in the unified memory 706 is stored in the external memory.

[0218] The bus interface unit (BIU) 710 is used to implement interaction between the main CPU, DMAC and instruction fetch memory 709 through the bus.

[0219] An instruction fetch buffer 709 connected to the controller 704 is used to store instructions used by the controller 704 .

[0220] The controller 704 is used to call the instructions cached in the instruction fetch memory 709 to control the working process of the computing accelerator.

[0221] Generally, the unified memory 706, the input memory 701, the weight memory 702, and the instruction fetch memory 709 are all on-chip memories, and the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM), or other readable and writable memory.

[0222] Lossless data compression is a fundamental and important area of information technology. Its goal is to establish a bijection from the original data space to the coding space, so that more frequently occurring and longer data is represented by a shorter code, resulting in a shorter data representation length on average. This bijection can also be used to achieve a one-to-one conversion between the original data space and the coding space. According to the Shannon source coding theorem, the optimal lossless compression length of data is determined by the Shannon information entropy of the data's probability distribution; the more accurate the estimate of the data's probability distribution, the closer the optimal lossless compression length is likely to be.

[0223] AI-based lossless compression schemes leverage the ability of deep generative models to more accurately estimate the probability distribution of data than traditional approaches, achieving compression ratios far superior to those of traditional lossless compression schemes. Among AI-based lossless compression schemes, widely used deep generative models include autoregressive models, variational autoencoders (VAEs), and normalizing flows. Generally speaking, autoregressive models are well compatible with arithmetic encoders and Huffman codes; variational autoencoders, combined with a bits-back mechanism, are well compatible with asymmetric digital systems; and flow models are compatible with the three aforementioned entropy encoders. In addition to compression ratio, throughput is also a metric used to evaluate lossless compression solutions. AI-based lossless compression solutions, due to their significantly larger model size, exhibit lower overall throughput than traditional solutions. Furthermore, considering both compression ratio and throughput, there is currently no absolute superiority among lossless compression solutions based on different generative models. Current research is still at the stage of exploring the Pareto frontier of compression schemes based on different generative models.

[0224] Among them, unlike full-observation models (such as autoregressive models), the variational autoencoder model is a latent variable model. This type of model does not directly model the data itself, but instead introduces one (or more) latent variables, and then models the prior distribution, likelihood function, and approximate posterior distribution. Since the marginal distribution of the data cannot be directly obtained from the variational autoencoder, the traditional entropy coding method cannot be directly used. In order to use the variational autoencoder for lossless data compression, a variational autoencoder lossless compression scheme based on the inverse coding mechanism was proposed. Bits-back ANS is the original form of this scheme, which is suitable for variational autoencoder models containing only one latent variable and can be generalized to variational autoencoder models containing multiple latent variables.

[0225] Taking Bits-Back ANS and a variational autoencoder with a single latent variable as examples, the model can be divided into three modules: the prior module, the variational encoder module, and the decoder module. These three modules can be used to determine the parameters of the following three distributions: the prior distribution of the latent variable, the likelihood function of the latent variable (the conditional probability distribution of the data), and the approximate posterior distribution of the latent variable.

[0226] The data compression steps in this technical solution are:

[0227] 1. Get the data to be compressed;

[0228] 2. Get additional initial bit data (bitstream 1);

[0229] 3. Input the data to be compressed into the variational encoder to obtain the approximate posterior distribution of the latent variable; based on the approximate posterior distribution, use the entropy encoder to decode a sample of the latent variable from bitstream 1 to obtain bitstream 2;

[0230] 4. Input the decompressed latent variable samples into the decoder to obtain the conditional probability distribution of the data; based on the conditional probability distribution of the data, the entropy encoder is used to compress the data into bitstream 2, thereby obtaining bitstream 3;

[0231] 5. Obtain a prior distribution of the latent variable from the prior module; compress the samples of the latent variable into bitstream 3 using an entropy encoder based on the prior distribution of the latent variable, thereby obtaining bitstream 4;

[0232] 6. Output bit stream 4 as the final compressed bit data.

[0233] The steps for decompressing data in this technical solution are:

[0234] 1. Obtain the bit data to be decompressed (bit stream 4);

[0235] 2. Obtaining a prior distribution of the latent variable from the prior module; decompressing the latent variable samples used in the compression stage from the fourth bitstream using an entropy encoder based on the prior distribution of the latent variable, thereby obtaining bitstream 3;

[0236] 3. Input the above hidden variable samples into the decoder to obtain the conditional probability distribution of the data; use the entropy encoder to decompress the compressed data from bitstream 3 based on the conditional probability distribution of the data to obtain bitstream 2;

[0237] 4. Input the decompressed data into the variational encoder to obtain the approximate posterior distribution of the latent variable; according to the approximate posterior distribution, use the entropy encoder to compress the above latent variable samples into bitstream 2, thereby obtaining bitstream 1;

[0238] 5. Output bitstream 1 as the additional initial bits for restoration;

[0239] 6. Output the decompressed data.

[0240] Existing lossless compression schemes for variational autoencoders based on inverse encoding mechanisms all require additional initial bits to decompress the latent variable samples. These additional initial bits are randomly generated data, and their size must be factored into the compression cost. The additional average cost is high when the number of data points to be compressed serially is small. Furthermore, because the required additional initial bits are proportional to the number of data points to be compressed, efficient parallel compression is not possible.

[0241] Based on the above technical background, the present invention is an improvement to an artificial intelligence lossless compression scheme based on variational autoencoders. This invention addresses two key pain points in this niche area: first, by introducing a special autoregressive structure, the number of parameters required to achieve the same compression ratio in the variational autoencoder is reduced, thereby improving throughput; second, by introducing a special variational encoder and decoder structure and proposing a new inverse encoding algorithm, the random initial bits required before the variational autoencoder lossless compression scheme are removed, thereby achieving single-data-point compression and decompression as well as efficient parallel compression and decompression.

[0242] Reference Figure 8 , Figure 8 This is an example of a data compression method provided in the present application. Figure 8 As shown, a data compression method provided by an embodiment of the present application includes:

[0243] 801. Acquire first target data, where the first target data includes first sub-data and second sub-data.

[0244] In one possible implementation, the first target data may be image data for compression or other data (such as text, video, etc.), wherein the first target data may be an image (or a portion of an image) captured by the terminal device through a camera, or the first target data may also be an image obtained from within the terminal device (for example, an image stored in an album of the terminal device, or a picture obtained by the terminal device from the cloud). It should be understood that the first target data may be data requiring image compression, and this application does not impose any limitation on the source of the first target image.

[0245] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block.

[0246] In one possible implementation, the first sub-data and the second sub-data are obtained by segmenting the image block in a spatial dimension or a channel dimension, wherein the image data includes one channel dimension (C) and two spatial dimensions (width W and height H).

[0247] For example, the first target data may include 6 channels, the first sub-data may be data of the first three channels in the first target data, and the second sub-data may be data of the last three channels in the first target data.

[0248] For example, the size of the first target data in the spatial dimension is N*N, the first sub-data can be the data in the first target data with a spatial dimension of (0 to N / 2)*N, and the second sub-data can be the data in the first target data with a spatial dimension of (N / 2 to N)*N.

[0249] It should be understood that the present application is not limited to the segmentation rules for performing data segmentation on the first target data.

[0250] 802. Obtain a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data.

[0251] In one possible implementation, the variational autoencoder may include a variational encoder, a decoder (such as the first decoder and the second decoder in the embodiment of the present application), and a prior distribution of latent variables.

[0252] In one possible implementation, the decoder can be composed of decoder layers (e.g., the first convolutional neural network and the second convolutional neural network in the embodiment of the present application), and the number of decoder layers is the same as the number of latent variables in the variational autoencoder. The decoder layer is used to input the latent variables of the deeper layer and output the conditional probability distribution of the current layer data (the current layer data can be the latent variables of the shallower layer or the data data).

[0253] In the existing variational autoencoder model, the variational encoder needs to input the entire data to predict the approximate posterior distribution of the latent variable, and the latent variable is input into the decoder to directly predict the conditional probability distribution of the entire data. In an embodiment of the present application, the data to be compressed is divided into at least two parts, namely: the first sub-data and the second sub-data. Unlike the existing method of inputting all the data into the variational encoder, in the embodiment of the present application, only a part of the data (the first sub-data) is input into the variational encoder to predict the approximate posterior distribution of the latent variable, and the conditional probability distribution of the first sub-data is predicted after the latent variable is input into the first decoder; the conditional probability distribution of the second sub-data depends on the first sub-data, which can be specifically determined by inputting the first sub-data into the first decoder.

[0254] Next, the structure of the decoder in the embodiment of the present application is introduced:

[0255] In one possible implementation, the decoder may implement a pixel reset operation.

[0256] Next, we introduce the pixel resetting operation from the spatial dimension to the channel dimension in the embodiment of the present application:

[0257] In one possible implementation, a parameter (denoted as k) and two reversible operations (denoted as spatial-to-channel and channel-to-spatial) can be configured to resize pixels in the channel and spatial dimensions. The parameter k is a positive integer that determines the ratio of the change in the spatial dimensions of the input and output tensors in the two reversible operations. For the same k, the spatial-to-channel and channel-to-spatial operations are inverses of each other.

[0258] For images, image data can be represented as a vector, which contains a channel dimension (C) and two spatial dimensions (width W and height H). Due to the characteristics of data batch processing in deep learning technology, the corresponding tensor representation has an additional batch dimension (N), that is, the image data tensor contains four dimensions (NCHW or NHWC). Taking HCHW as an example, a tensor of size n1*c1*h1*w1 can be converted to a tensor of size n1*k through a spatial-to-channel operation with parameter k. 2 A tensor of size c1*h1 / k*w1 / k. Here, both h1 and w1 are required to be divisible by k. A tensor of size n2*c2*h2*w2 is converted to a tensor of size n2*c2 / k by a channel-to-space operation with parameter k. 2 *kh2*kw2 tensor. It can be seen that the above two operations do not change the total number of elements in the tensor, but only change the position of the elements in the tensor. Different pixel reset rules can be used to obtain different pixel reset devices. The pixel reset operation used in the embodiment of the present application adopts a channel priority method. Since the space-to-channel and channel-to-space operations are mutually inverse when k is fixed, Figure 9 It shows the effect of space-to-channel operation when n is 1, h and w are 4, c is 3, and k is 2.

[0259] Taking the first decoder as an example, in a possible implementation, the first decoder may include a first convolutional neural network and a second convolutional neural network. Obtaining the first probability distribution based on the first sub-data through the first decoder of the variational autoencoder may specifically include: performing a pixel resetting operation from the spatial dimension to the channel dimension on the second target data including the second sub-data to obtain the third sub-data, where the second target data and the first target data have the same size, and the third sub-data and the first sub-data have the same size in the spatial dimension;

[0260] The second target data including the second sub-data may be data of the same size as the first target data, wherein, in the first target data, elements other than the second sub-data may be set to zero (or other preset values) to obtain the second target data, and after performing a pixel reset operation on the second target data, it may be converted into third sub-data of the same size as the first sub-data in the spatial dimension. For example, you can refer to Figure 10, wherein the current layer variable can be the above-mentioned second target data, and a pixel reset operation can be performed on the current layer variable.

[0261] In one possible implementation, fourth sub-data can be obtained based on the first sub-data through the first convolutional neural network, and the fourth sub-data has the same size as the third sub-data in the channel dimension. In other words, the first sub-data can be subjected to feature extraction and size transformation through the first convolutional neural network to obtain fourth sub-data having the same size as the third Hull sub-data in the channel dimension.

[0262] In a possible implementation, the third sub-data and the fourth sub-data may be fused to obtain fused sub-data. Optionally, the fusion method may be data replacement of corresponding channels.

[0263] In a possible implementation, the fusing of the third sub-data and the fourth sub-data may specifically include: replacing data of some channels in the fourth sub-data with data of corresponding channels in the third sub-data to obtain fused sub-data.

[0264] When calculating the probability distribution of the i+1th channel pixel of the second sub-data, the third sub-data z′ can be i-1 The first i channels replace the fourth sub-data z″ i The first i channels of .

[0265] In a possible implementation, the first probability distribution can be obtained by passing the fused sub-data through the second convolutional neural network.

[0266] In a possible implementation, the fused sub-data and the first sub-data may be concatenated along the channel dimension to obtain concatenated sub-data; and further, obtaining the first probability distribution through the second convolutional neural network based on the fused sub-data may specifically include obtaining the first probability distribution through the second convolutional neural network based on the concatenated sub-data.

[0267] For example, refer to Figure 10 , the deeper hidden variable is z i , the output is the current layer variable z i-1 The input of neural network 1 is z i , the output is the same as z i-1 A tensor z′ of the same size (containing the same number of elements) i The tensor z′ i After the space-to-channel pixel resetting operation with parameter k, it becomes z iTensor z″ of the same size in spatial dimensions i Since the tensor z i With the tensor z″ i The spatial dimensions are the same, so we can perform splicing operations according to the channel dimensions to obtain the spliced tensor z″′ i . Let tensor z be i-1 After the space-to-channel pixel resetting operation with parameter k, it becomes z i Tensor z′ with the same spatial dimensions i-1 The decoder layer introduces the autoregressive structure by converting the tensor z″′ i Input neural network 2 and get tensor z′ i-1 The probability distribution parameters of the first channel pixel; tensor z' i-1 Replace z″′ with the first i channels i From z″ i The first i channels of the neural network are input into the second neural network to obtain the tensor z′ i-1 Probability distribution parameters of the i+1th channel pixel.

[0268] Reference Figure 11 , Figure 11 This is an illustration of a variational autoencoder in an embodiment of the present application, wherein decoder 1 is equivalent to the second decoder in the embodiment of the present application, and decoder 2 is equivalent to the first decoder in the embodiment of the present application.

[0269] For example, as shown in Table 1 below, Table 1 shows an exemplary process for a variational autoencoder containing one latent variable. The compression process is shown on the left of Table 1, and the decompression process is shown on the right. Figure 1 As shown on the right. The approximate posterior distribution of its latent variable q(z 1 The parameters of |x) are given by the variational encoder after the input of the compressed data. The prior distribution of its deepest latent variables is The parameters of are directly given by the parameters of the deepest hidden variable prior distribution module in the model. The parameters of the remaining conditional probability distributions are output by the corresponding decoder layer through the value of the input conditional data. The structure of each decoder layer involved can refer to the description of the decoder in the above embodiment. 12 For data x (containing 3 channels), input the 12 channels obtained by transforming the spatial dimension to the channel dimension through channel-first pixel resetting (parameter k is 2). Same thing.

[0270] Table 1

[0271]

[0272] The embodiment of the present application fully utilizes the correlation between image pixels by using an encoder layer with an autoregressive structure defined based on channel-priority pixel reset, thereby significantly reducing the number of parameters required for the model while obtaining a lower coding length, thereby improving the compression throughput and reducing the space cost of model storage.

[0273] 803. Compress the second sub-data using an entropy encoder according to the first probability distribution to obtain a first bit stream.

[0274] In one possible implementation, the second sub-data can be compressed using an entropy encoder based on the first probability distribution to obtain a first bitstream. The first bitstream can serve as the initial bitstream for compressing the first sub-data. Compared to the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiments of the present application do not require additional initial bits, can achieve compression of a single data point, and significantly reduce the compression ratio during parallel compression.

[0275] 804. Compress the first sub-data into the first bit stream.

[0276] In a possible implementation, using the first bit stream as the initial bit stream and compressing the first sub-data may specifically include: obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational autoencoder based on the first sub-data; obtaining the latent variable from the first bit stream through the entropy encoder based on the approximate posterior distribution to obtain a third bit stream; obtaining a second probability distribution through a second decoder of the variational autoencoder based on the latent variable; the second probability distribution is used as a conditional probability distribution of the first sub-data; compressing the first sub-data to the third bit stream through the entropy encoder based on the second probability distribution to obtain a fourth bit stream; and compressing the latent variable to the fourth bit stream through the entropy encoder based on the prior distribution of the latent variable to obtain a second bit stream.

[0277] Exemplarily, on the encoding side, the following steps may be performed:

[0278] 1. Acquire the first target data;

[0279] 2. Splitting the first target data into first sub-data and second sub-data, inputting the first sub-data into a first decoder to obtain a conditional probability distribution of the second sub-data; and compressing the second sub-data using the conditional probability distribution of the second sub-data to obtain initial bit data (a first bit stream).

[0280] 3. Input the first sub-data into a variational encoder to obtain an approximate posterior distribution of the latent variable; decode a sample of the latent variable from the first bitstream using an entropy encoder based on the approximate posterior distribution to obtain a third bitstream;

[0281] 4. Input the decompressed latent variable sample into the second decoder to obtain the conditional probability distribution of the data; use the entropy encoder to compress the first sub-data into the third bit stream based on the conditional probability distribution of the data, thereby obtaining a fourth bit stream;

[0282] 5. Obtaining a prior distribution of the latent variable from the prior module; compressing the samples of the latent variable into a third bit stream using an entropy encoder based on the prior distribution of the latent variable, thereby obtaining a second bit stream;

[0283] 6. Output the second bit stream as the final compressed bit data.

[0284] Correspondingly, in the decoding stage, the decompression steps can be:

[0285] 1. Obtain the bit data to be decompressed (second bit stream);

[0286] 2. Obtaining a prior distribution of the latent variable from the prior module; decompressing the latent variable samples used in the compression phase from the second bitstream using an entropy encoder based on the prior distribution of the latent variable, thereby obtaining a fourth bitstream;

[0287] 3. Input the latent variable sample into decoder 1 to obtain the conditional probability distribution of sub-data 1; decompress the compressed sub-data 1 from the fourth bitstream using an entropy encoder based on the conditional probability distribution of sub-data 1, and obtain a third bitstream;

[0288] 4. Input the decompressed sub-data into the variational encoder to obtain an approximate posterior distribution of the latent variable; compress the latent variable samples into the third bit stream using the entropy encoder based on the approximate posterior distribution, thereby obtaining the first bit stream;

[0289] 5. Sub-data 1 is input into decoder 2 to obtain the conditional probability distribution of sub-data 2. The conditional probability distribution of sub-data 2 is decompressed from the first bit stream to obtain sub-data 2. At this point, the bit stream is completely consumed.

[0290] 6. Reverse the separation method of sub-data 1 and sub-data 2 during compression to obtain the original data, and output the decompressed original data (first target data).

[0291] Reference Figure 12 , Figure 12 : This is an illustration of the data compression process when the number of latent variables is 1, where S is the first target data, S1 is the first sub-data, and S2 is the second sub-data.

[0292] In order to compare the differences between the embodiment of the present application and the existing solution, the processes are shown in Figure 13 . Figure 13 The left side shows the core compression and decompression process of the existing solution; Figure 13 The right side shows the core compression and decompression process of the embodiment of the present application without the need for additional initial bits.

[0293] Refer to Table 2, which shows the core method flow of the decoder layer of the autoregressive structure defined based on channel-first pixel resetting and the variational autoencoder lossless compression solution without additional initial bits.

[0294] Table 2

[0295]

[0296] Next, the beneficial effects of the embodiments of the present application are described in conjunction with experimental results.

[0297] In the embodiment of the present application, since the encoder used makes good use of the correlation between pixels of the image data, it can reduce the model size by 100 times while providing a better coding length than the lossless compression scheme of the same type of model.

[0298] Table 3 shows the average bits per dimension (bpd) of this solution (SHVC) compared to other industry-leading solutions on public datasets. This shows that this solution performs optimally or nearly optimally among all compared solutions (including traditional solutions, VAE models, and stream models). Among similar solutions, the VAE-based solution is the best.

[0299] Table 4 shows that, in addition to the advantage in encoding length, this solution significantly reduces inference time due to the reduced number of model parameters, thereby improving compression and decompression throughput. The data set used is 10,000 CIFAR10 images, a batch size of 100, and a V100 graphics card.

[0300] Table 3

[0301]

[0302] Table 4

[0303]

[0304] In addition, the embodiments of the present application can achieve compression and efficient parallel compression of single data points while avoiding the additional initial bits required by the current inverse coding mechanism. Table 5 shows the average per-dimension coding length when taking into account the additional initial bits for the embodiment of the present application (SHVC), the use of (SHVC-ARIB), and the scheme using a deterministic posteriors (essentially an autoencoder model) and no inverse coding mechanism (SHVC-Det). As can be seen from Table 5, the use of this scheme can reduce the additional space cost by up to 30 times compared to the current inverse coding compression algorithm.

[0305] Table 5

[0306] SHVC SHVC-ARIB SHVC-Det CIFAR10 4.18 3.19 3.37 ImageNet32 5.03 4.00 4.17 ImageNet64 4.57 3.71 3.90

[0307] It should be understood that the embodiments of the present application can also be used in lossy compression of data.

[0308] An embodiment of the present application provides a data compression method, comprising: obtaining first target data, the first target data including first sub-data and second sub-data; obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, the first probability distribution being used as a conditional probability distribution for the second sub-data; compressing the second sub-data based on the first probability distribution through an entropy encoder to obtain a first bit stream; and compressing the first sub-data using the first bit stream as an initial bit stream to obtain a second bit stream. Compared to the additional initial bits required by the inverse encoding mechanism in the prior art, the embodiment of the present application does not require additional initial bits, can achieve compression of a single data point, and greatly reduces the compression ratio during parallel compression.

[0309] Reference Figure 14 , Figure 14 A data decompression method according to an embodiment of the present invention is shown in FIG. Figure 14 As shown, the data decompression method provided in the embodiment of the present application includes:

[0310] 1401. Obtain a second bit stream and a prior distribution of latent variables;

[0311] 1402. Decompress the latent variable from the second bitstream using an entropy encoder according to the prior distribution to obtain a fourth bitstream.

[0312] 1403. Obtain a second probability distribution based on the latent variable through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0313] 1404. Decompress the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream.

[0314] 1405. Obtain an approximate posterior distribution of a latent variable using a variational encoder in the variational self-decoder according to the first sub-data;

[0315] 1406. Compress the latent variable into the third bit stream using the entropy encoder according to the approximate posterior distribution to obtain a first bit stream.

[0316] 1407. Obtain a first probability distribution based on the first sub-data through a first decoder of the variational autoencoder, where the first probability distribution is used as a conditional probability distribution of the second sub-data.

[0317] 1408. Decompress the first bit stream to obtain second sub-data through the entropy encoder according to the first probability distribution; the first sub-data and the second sub-data are used to determine first target data.

[0318] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block.

[0319] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0320] In one possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and obtaining the first probability distribution through the first decoder of the variational autoencoder according to the first sub-data includes:

[0321] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0322] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0323] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0324] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0325] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0326] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0327] In one possible implementation, the method further includes:

[0328] Performing a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0329] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0330] For a description of the data decompression method, please refer to Figure 8 The descriptions about data decompression in the corresponding embodiments will not be repeated here.

[0331] exist Figures 1 to 14 On the basis of the corresponding embodiment, in order to better implement the above solution of the embodiment of the present application, the following also provides related equipment for implementing the above solution. Figure 15 , Figure 15 This is a schematic structural diagram of a data compression device 1500 provided in an embodiment of the present application. The data compression device 1500 may be a terminal device or a server. The data compression device 1500 includes:

[0332] An acquisition module 1501 is configured to acquire first target data, where the first target data includes first sub-data and second sub-data;

[0333] For a detailed description of the acquisition module 1501 , reference may be made to the description of step 801 in the above embodiment, which will not be repeated here.

[0334] Compression module 1502, configured to obtain a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data;

[0335] compressing the second sub-data using an entropy encoder according to the first probability distribution to obtain a first bitstream;

[0336] The first bit stream is used as an initial bit stream, and the first sub-data is compressed to obtain a second bit stream.

[0337] For a detailed description of the compression module 1502 , reference may be made to the description of steps 802 to 804 in the above embodiment, which will not be repeated here.

[0338] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block.

[0339] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0340] In a possible implementation, the compression module is specifically configured to:

[0341] Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational self-decoder according to the first sub-data;

[0342] Obtaining the latent variable from the first bitstream through the entropy encoder according to the approximate posterior distribution to obtain a third bitstream;

[0343] According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0344] compressing the first sub-data into the third bit stream by the entropy encoder according to the second probability distribution to obtain a fourth bit stream;

[0345] According to the prior distribution of the latent variable, the latent variable is compressed into the fourth bit stream by the entropy encoder to obtain a second bit stream.

[0346] In one possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and the compression module is specifically configured to:

[0347] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0348] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0349] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0350] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0351] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0352] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0353] In a possible implementation, the apparatus further includes:

[0354] a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0355] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0356] See Figure 16 , Figure 16 This is a schematic diagram of the structure of a data decompression device 1600 provided in an embodiment of the present application. The data decompression device 1600 may be a terminal device or a server. The data decompression device 1600 may include:

[0357] An acquisition module 1601 is configured to acquire a second bit stream and a prior distribution of latent variables;

[0358] For a detailed description of the acquisition module 1601 , reference may be made to the description of step 1401 in the above embodiment, which will not be repeated here.

[0359] a decompression module 1602 configured to decompress the latent variable from the second bit stream using an entropy encoder according to the prior distribution to obtain a fourth bit stream;

[0360] According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data;

[0361] decompressing the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream;

[0362] Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational self-decoder according to the first sub-data;

[0363] compressing the latent variable into the third bit stream by the entropy encoder according to the approximate posterior distribution to obtain a first bit stream;

[0364] Obtaining a first probability distribution through a first decoder of the variational autoencoder according to the first sub-data, where the first probability distribution is used as a conditional probability distribution of the second sub-data;

[0365] According to the first probability distribution, second sub-data are decompressed from the first bit stream by the entropy encoder; the first sub-data and the second sub-data are used to determine first target data.

[0366] For a detailed description of the decompression module 1602 , reference may be made to the description of steps 1402 to 1408 in the above embodiment, which will not be repeated here.

[0367] In a possible implementation, the first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block.

[0368] In a possible implementation, the first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

[0369] In a possible implementation, the first decoder includes a first convolutional neural network and a second convolutional neural network, and the decompression module is specifically configured to:

[0370] performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension;

[0371] Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension;

[0372] fusing the third sub-data and the fourth sub-data to obtain fused sub-data;

[0373] The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

[0374] In a possible implementation, the fusing the third sub-data and the fourth sub-data includes:

[0375] The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

[0376] In a possible implementation, the apparatus further includes:

[0377] a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data;

[0378] The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

[0379] Next, we will introduce an execution device provided by the embodiment of the present application. Figure 17 , Figure 17 This is a structural diagram of the execution device provided in the embodiment of the present application. The execution device 1700 can be specifically manifested as a virtual reality VR device, a mobile phone, a tablet, a laptop, a smart wearable device, a monitoring data processing device, etc., which is not limited here. Among them, the execution device 1700 can be deployed with Figure 15 The data compression device described in the corresponding embodiment, or Figure 16 The data decompression device described in the corresponding embodiment. Specifically, the execution device 1700 may include: a receiver 1701, a transmitter 1702, a processor 1703 and a memory 1704 (wherein the number of the processor 1703 in the execution device 1700 may be one or more, Figure 15 (taking one processor as an example), the processor 1703 may include an application processor 17031 and a communication processor 17032. In some embodiments of the present application, the receiver 1701, the transmitter 1702, the processor 1703 and the memory 1704 may be connected via a bus or other means.

[0380] Memory 1704 may include read-only memory and random access memory, and provides instructions and data to processor 1703. A portion of memory 1704 may also include non-volatile random access memory (NVRAM). Memory 1704 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0381] Processor 1703 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0382] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1703. Processor 1703 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 1703 or software instructions. The above processor 1703 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1703 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1704, and processor 1703 reads the information in memory 1704 and, in conjunction with its hardware, completes the steps of the above method.

[0383] Receiver 1701 can be used to receive input digital or character information and generate signal input related to executing device-related settings and function control. Transmitter 1702 can be used to output digital or character information through the first interface. Transmitter 1702 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack. Transmitter 1702 can also include a display device such as a display screen.

[0384] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the aforementioned Figure 8 The steps performed by the method described in the embodiment shown, or the computer is made to perform the above Figure 14 The illustrated embodiment describes the steps performed by the method.

[0385] The present application also provides a computer-readable storage medium in which a program for signal processing is stored. When the program is run on a computer, the computer executes the above-mentioned Figure 8The steps performed by the method described in the embodiment shown, or the computer is made to perform the above Figure 14 The illustrated embodiment describes the steps performed by the method.

[0386] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0387] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0388] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0389] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive (SSD)).

Claims

1. A data compression method, characterized in that: include: Acquire first target data, where the first target data includes first sub-data and second sub-data; Obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution of the second sub-data; compressing the second sub-data by an entropy encoder according to the first probability distribution to obtain a first bit stream; The first sub-data is compressed into the first bit stream to obtain a second bit stream.

2. The method according to claim 1, characterized in that The first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or, The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or, The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

3. The method according to claim 2, characterized in that The first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

4. The method according to any one of claims 1 to 3, characterized in that: The compressing the first sub-data into the first bit stream includes: Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational autoencoder according to the first sub-data; Decoding the latent variable from the first bit stream using the entropy encoder according to the approximate posterior distribution to obtain a third bit stream; According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data; compressing the first sub-data into the third bit stream by the entropy encoder according to the second probability distribution to obtain a fourth bit stream; According to the prior distribution of the latent variable, the latent variable is compressed into the fourth bit stream by the entropy encoder to obtain a second bit stream.

5. The method according to any one of claims 1 to 3, characterized in that: The first decoder includes a first convolutional neural network and a second convolutional neural network, and obtaining a first probability distribution through the first decoder of the variational autoencoder according to the first sub-data includes: performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension; Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension; fusing the third sub-data and the fourth sub-data to obtain fused sub-data; The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

6. The method according to claim 5, characterized in that The fusing the third sub-data and the fourth sub-data includes: The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

7. The method according to claim 5, characterized in that The method further comprises: Performing a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data; The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

8. A data decompression method, characterized in that: include: Obtaining a second bitstream; decoding the first sub-data from the second bit stream to obtain a first bit stream; Obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data; According to the first probability distribution, second sub-data are decompressed from the first bit stream through an entropy encoder; the first sub-data and the second sub-data are used to restore first target data.

9. The method according to claim 8, characterized in that The decoding the first sub-data from the second bit stream to obtain the first bit stream includes: Obtain the prior distribution of latent variables; decompressing the latent variable from the second bitstream using an entropy encoder according to the prior distribution to obtain a fourth bitstream; According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data; decompressing the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream; Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational autoencoder according to the first sub-data; According to the approximate posterior distribution, the latent variable is compressed into the third bit stream by the entropy encoder to obtain a first bit stream.

10. The method according to claim 8 or 9, characterized in that The first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or, The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

11. The method according to claim 10, characterized in that The first sub-data and the second sub-data are obtained by dividing the image block in a spatial dimension or a channel dimension.

12. The method according to claim 8 or 9, characterized in that The first decoder includes a first convolutional neural network and a second convolutional neural network, and obtaining a first probability distribution through the first decoder of the variational autoencoder according to the first sub-data includes: performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension; Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension; fusing the third sub-data and the fourth sub-data to obtain fused sub-data; The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

13. The method according to claim 12, characterized in that The fusing the third sub-data and the fourth sub-data includes: The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

14. The method according to claim 12, characterized in that The method further comprises: Performing a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data; The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

15. A data compression device, characterized in that: include: An acquisition module, configured to acquire first target data, where the first target data includes first sub-data and second sub-data; a compression module, configured to obtain, based on the first sub-data, a first probability distribution through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution of the second sub-data; compressing the second sub-data by an entropy encoder according to the first probability distribution to obtain a first bit stream; The first sub-data is compressed into the first bit stream to obtain a second bit stream.

16. The device according to claim 15, characterized in that The first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or, The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or, The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

17. The device according to claim 15 or 16, characterized in that The compression module is specifically used to: Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational autoencoder according to the first sub-data; Decoding the latent variable from the first bit stream using the entropy encoder according to the approximate posterior distribution to obtain a third bit stream; According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data; compressing the first sub-data into the third bit stream by the entropy encoder according to the second probability distribution to obtain a fourth bit stream; According to the prior distribution of the latent variable, the latent variable is compressed into the fourth bit stream by the entropy encoder to obtain a second bit stream.

18. The device according to claim 15 or 16, characterized in that The first decoder includes a first convolutional neural network and a second convolutional neural network, and the compression module is specifically configured to: performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension; Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension; fusing the third sub-data and the fourth sub-data to obtain fused sub-data; The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

19. The device according to claim 18, characterized in that The fusing the third sub-data and the fourth sub-data includes: The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

20. The device according to claim 18, characterized in that The device further comprises: a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data; The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

21. A data decompression device, characterized in that: include: An acquisition module, configured to acquire a second bit stream; a decompression module, configured to decode the first sub-data from the second bit stream to obtain a first bit stream; Obtaining a first probability distribution based on the first sub-data through a first decoder of a variational autoencoder, where the first probability distribution is used as a conditional probability distribution for the second sub-data; According to the first probability distribution, second sub-data are decompressed from the first bit stream through an entropy encoder; the first sub-data and the second sub-data are used to restore first target data.

22. The device according to claim 21, characterized in that The decompression module is specifically used to: Obtain the prior distribution of latent variables; decompressing the latent variable from the second bitstream using an entropy encoder according to the prior distribution to obtain a fourth bitstream; According to the latent variable, a second probability distribution is obtained through a second decoder of the variational autoencoder; the second probability distribution is used as a conditional probability distribution of the first sub-data; decompressing the first sub-data from the fourth bitstream using the entropy encoder according to the second probability distribution to obtain a third bitstream; Obtaining an approximate posterior distribution of a latent variable through a variational encoder in the variational autoencoder according to the first sub-data; According to the approximate posterior distribution, the latent variable is compressed into the third bit stream by the entropy encoder to obtain a first bit stream.

23. The device according to claim 21 or 22, characterized in that The first target data is an image block, and the first sub-data and the second sub-data are obtained by segmenting the image block; or The first target data is a text sequence, and the first sub-data and the second sub-data are obtained by segmenting the text sequence; or, The first target data is a binary stream, and the first sub-data and the second sub-data are obtained by segmenting the binary stream; or The first target data is a video, and the first sub-data and the second sub-data are obtained by segmenting a plurality of image frames of the video.

24. The device according to claim 21 or 22, characterized in that The first decoder includes a first convolutional neural network and a second convolutional neural network, and the decompression module is specifically configured to: performing a pixel resetting operation from a spatial dimension to a channel dimension on second target data including the second sub-data to obtain third sub-data, wherein the second target data has the same size as the first target data, and the third sub-data has the same size as the first sub-data in the spatial dimension; Obtain fourth sub-data through the first convolutional neural network based on the first sub-data, where the fourth sub-data has the same size as the third sub-data in the channel dimension; fusing the third sub-data and the fourth sub-data to obtain fused sub-data; The first probability distribution is obtained through the second convolutional neural network based on the fused sub-data.

25. The device according to claim 24, characterized in that The fusing the third sub-data and the fourth sub-data includes: The data of some channels in the fourth sub-data are replaced with the data of corresponding channels in the third sub-data to obtain fused sub-data.

26. The device according to claim 24, characterized in that The device further comprises: a splicing module, configured to perform a splicing operation on the fused sub-data and the first sub-data along a channel dimension to obtain spliced sub-data; The obtaining of the first probability distribution according to the fused sub-data through the second convolutional neural network includes: obtaining the first probability distribution according to the spliced sub-data through the second convolutional neural network.

27. A data compression device, characterized in that: The system comprises a storage medium, a processing circuit and a bus system; wherein the storage medium is used to store instructions, and the processing circuit is used to execute the instructions in the memory to perform the steps of the method according to any one of claims 1 to 14.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

29. A computer program product, characterized in that The computer program product comprises codes for implementing the steps of the method according to any one of claims 1 to 14 when the codes are executed.

Images