Compression method of heterogeneous data, computer device and computer readable storage medium

Abstract

The application provides a heterogeneous data compression method, a computer device and a computer readable storage medium, which comprises the following steps: inputting heterogeneous data of a system on chip into a compression network architecture; performing macro classification on the heterogeneous data by a physical domain router to obtain classified data; multiple encoders are used to extract features of the classified data corresponding to the encoders to obtain multiple groups of potential features, and the multiple groups of potential features are mapped to the same dimension; a hierarchical grouping vector quantization bottleneck layer is used to cluster the multiple groups of potential features to generate multiple codebooks; a loss function comprises a reconstruction loss, a vector quantization loss and a hardware bit width constraint term; the compression network architecture is trained to generate multiple optimal codebooks; the closest target centroid of the multiple optimal codebooks to the to-be-compressed data is queried, and the to-be-compressed data is compressed according to the optimal centroid to obtain compressed data. The application can ensure the compression efficiency and reduce the use of chip area.

Description

Technical Field

[0001] This invention relates to the field of data compression technology, specifically a method for compressing heterogeneous data, a computer device, and a computer-readable storage medium. Background Technology

[0002] Existing System-on-Chip (SoC) architectures are becoming increasingly complex. Current intelligent vision SoCs often integrate multiple computing units such as ISP, AI ISP, general-purpose NPU, codec, GPU, DPU, and CPU. During system operation, each unit needs to concurrently read weights, feature maps, image data, and instruction code from memory, resulting in tight bus bandwidth and high power consumption. Existing solutions often compress the data of each module independently, but this leads to the following drawbacks: First, if a single dictionary is used to forcibly fit data with completely different physical meanings (such as sparse weights in the tensor domain and spatially smooth images in the pixel domain), it will make it extremely difficult for the network to converge and result in huge quantization errors.

[0003] Second, existing deep learning compression (such as VQ-VAE) typically uses mean squared error (MSE) as the loss function, which fails to consider the bit width limitations of the SoC's underlying hardware adder, making it easy for the generated residuals to overflow during hardware decompression.

[0004] Third, different types of heterogeneous data use different variable-length entropy encoding algorithms, making it impossible to share a single, simplified decompression pipeline at the SoC hardware level, resulting in a sharp increase in chip area. Summary of the Invention

[0005] The first objective of this invention is to provide a method for compressing heterogeneous data that reduces chip area and data overflow.

[0006] A second objective of the present invention is to provide a computer device for implementing the above-described method for compressing heterogeneous data.

[0007] A second objective of this invention is to provide a computer-readable storage medium for implementing the above-described method for compressing heterogeneous data.

[0008] To achieve the first objective of this invention, the present invention provides a method for compressing heterogeneous data. The method includes: inputting heterogeneous data from an on-chip system into a compression network architecture; wherein the compression network architecture includes a physical domain router, an encoder, a hierarchical grouping vectorization bottleneck layer, and a loss function; the physical domain router performs macro-classification on the heterogeneous data to obtain categorized data; multiple encoders extract features from the categorized data corresponding to the encoders to obtain multiple sets of latent features, which are mapped to the same dimension; the hierarchical grouping vectorization bottleneck layer clusters the multiple sets of latent features to generate multiple codebooks, each codebook corresponding to one set of latent features; the loss function includes reconstruction loss, vector quantization loss, and a hardware bit-width constraint term, the hardware bit-width constraint term outputting the maximum value between zero and the calculated value, the calculated value being the difference between the residual value and a preset value, the residual value being obtained after compressing the centroid of the latent feature and the corresponding codebook; training the compression network architecture to generate multiple optimal codebooks; acquiring the data to be compressed, querying the closest target centroid between the data to be compressed and the multiple optimal codebooks, and compressing the data to be compressed based on the optimal centroid to obtain compressed data.

[0009] As can be seen from the above scheme, multiple sets of latent data are generated by multiple encoders, and all sets of latent data are mapped to the same dimension. Each set of latent data generates a codebook for latent data, thereby reducing the error caused by forcibly fitting a single codebook during subsequent compression. Since all sets of latent data are mapped to the same dimension, the data to be compressed is compressed into a single size, allowing for a shared decompression engine. Compared to traditional schemes where images, weights, and code each rely on a variable-length entropy decoder, resulting in hardware logic redundancy, this invention reduces the use of on-chip hardware resources and the area of ​​the on-chip system. Furthermore, by including a hardware bit-width constraint term in the loss function, the network output can be forced to adapt to the specific physical circuit bit width, reducing the chip's computational resources.

[0010] In a further scheme, the classification data includes at least one of the following: AI tensor domain, image domain, and code domain; the encoder includes at least one of the following: tensor encoder, visual encoder, and code encoder; and the latent features include at least one of the following: channel distribution features, visual features, and temporal features.

[0011] In a further proposed solution, the steps for the encoder to extract features from the classification data corresponding to the encoder include: a tensor encoder extracting features from the AI ​​tensor domain to obtain channel distribution features; a visual encoder extracting features from the image domain to obtain visual features; and a code encoder extracting features from the code domain to obtain temporal features.

[0012] This demonstrates that by capturing the latent features of different data domains during the training phase using different encoders, the most suitable codebook for that data domain can be generated.

[0013] A further proposed scheme includes an optimal codebook comprising a tensor domain codebook, an image domain codebook, and a code domain codebook; the tensor domain codebook includes channel distribution primitives, the image domain codebook includes visual primitives, and the code domain codebook includes temporal primitives.

[0014] This shows that different codebooks contain different information data, which reduces the error caused by forcibly fitting a single codebook during subsequent compression.

[0015] In a further scheme, the steps of compressing the data to be compressed based on the highest quality center to obtain compressed data include: compressing the data to be compressed based on the highest quality center to obtain a target residual value; determining whether the target residual value is less than a preset value; if so, outputting the target residual value and the index information of the highest quality center as compressed data; if the target residual value is greater than the preset value, discarding the target residual value and outputting the data to be compressed as outlier data.

[0016] This demonstrates that by setting outlier data, a high compression rate can be achieved with extremely low logic gate overhead.

[0017] In a further scheme, after obtaining compressed data or outlier data, the following steps are performed: the address information of the compressed data or the address information of the outlier data is recorded in a mapping table, which also includes a compression flag bit for the compressed data or the outlier data.

[0018] Therefore, the mapping table uses a compression flag to determine whether a data segment has been compressed.

[0019] In a further scheme, after obtaining the compressed data, the following steps are performed: A decompression step is executed; the decompression step includes: obtaining a mapping table, determining whether the compression flag is a compression flag based on the mapping table; if the compression flag is a compression flag, obtaining the compressed data from the address information corresponding to the compression flag; obtaining index information from the compressed data, querying the highest quality core based on the index information, and calculating the decompressed data based on the target residual value of the compressed data and the highest quality core; if the compression flag is not a compression flag, obtaining the decompressed data from the outlier data; synchronizing the compression step using a multiplexer based on the compression flag, and outputting the decompressed data.

[0020] This shows that during decompression, all data domains share a single decompression engine, thus enabling the use of on-chip system hardware resources.

[0021] To achieve the second objective, the present invention provides a computer device comprising a memory and a processor, characterized in that the memory stores a computer program, which, when executed, implements the above-described method for compressing heterogeneous data.

[0022] To achieve the third objective, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described method for compressing heterogeneous data. Attached Figure Description

[0023] Figure 1 This is a structural diagram of the compression network architecture of an embodiment of the heterogeneous data compression method of the present invention.

[0024] Figure 2 This is a flowchart of the compression network architecture of an embodiment of the heterogeneous data compression method of the present invention.

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0026] This invention classifies heterogeneous data using a physical domain router, then extracts features through various encoders to obtain multiple sets of latent features. These latent features are mapped to the same dimension, resulting in multiple codebooks of the same dimension. Multiple codebooks reduce errors caused by forced fitting from a single codebook during subsequent compression. Furthermore, because the codebooks are of the same dimension, the data to be compressed is compressed into a single size, allowing for a shared decompression engine. Compared to traditional solutions where images, weights, and code each rely on a variable-length entropy decoder, leading to hardware logic redundancy, this invention reduces the use of on-chip system hardware resources.

[0027] Examples of methods for compressing heterogeneous data: See Figure 1 Before compressing heterogeneous data, an optimal codebook must first be generated. Generating the optimal codebook involves... Figure 1 The compressed network architecture is generated. See also: Figure 2 First, step S11 is executed to input the heterogeneous data of the on-chip system into the compression network architecture. Before inputting the heterogeneous data of the on-chip system into the compression network architecture, each heterogeneous data is divided into fixed-size micro-blocks of fixed length, each with a size of 4×4 bytes.

[0028] The compressed network architecture includes a physical domain router 12, an encoder, a hierarchical packet vectorization bottleneck layer 16, and a loss function 17. The encoder includes at least one of the following: a tensor encoder 13, a visual encoder 14, and a code encoder 15.

[0029] After inputting the heterogeneous data from the on-chip system into the compressed network architecture, step S12 is executed. The physical domain router 12 performs macro-classification on the heterogeneous data to obtain classified data. The classified data includes at least one of the following: AI tensor fields, image fields, and code fields. The AI ​​tensor fields include the weights and feature maps of the AI ​​ISP and general-purpose NPU. This type of data exhibits a multidimensional Gaussian distribution and structured sparsity. The image fields include various image data (such as LUTs, reference frames, and rendered textures) from the ISP / Codec / GPU / DPU. This type of data exhibits strong spatial correlation, smooth gradients, and local high-frequency corners. The code fields include the specified executable code of the CPU. This type of data exhibits strong logical jumps and high-frequency instruction set combinations. The macro-classification is a coarse-grained classification mechanism. Macro-classification directly divides the heterogeneous data into major categories (AI tensor fields, image fields, and code fields) based on the physical manifold location and generation source of the heterogeneous data in the SoC chip, thereby decoupling the heterogeneous data at the very beginning of compression.

[0030] After obtaining the classification data, step S13 is executed, where multiple encoders extract features from the classification data corresponding to the encoders, obtaining multiple sets of latent features. The encoders include at least one of the following: a tensor encoder 13, a visual encoder 14, and a code encoder 15. The latent features include at least one of the following: channel distribution features, visual features, and temporal features. The steps for the encoders to extract features from the classification data corresponding to the encoders include: the tensor encoder extracts features from the AI ​​tensor domain to obtain channel distribution features; the visual encoder extracts features from the image domain to obtain visual features; and the code encoder extracts features from the code domain to obtain temporal features. Channel distribution features represent the homogeneity or extreme value sparse distribution among the AI ​​data channels. Visual features represent the directional gradient or geometric topology of the image data. Temporal features represent the high-frequency combination patterns of the instruction set. The tensor encoder can be a PCA encoder or a block-based compression scheme. The visual encoder can be a two-dimensional convolutional neural network or a graph neural network (GNN). The code encoder can be a sequence model. After extracting high-order statistical features through nonlinear transformation within each encoder, the number of neurons in its output layer is uniformly set to the codebook dimension, thereby forcing input data with different physical meanings to be mapped to the same dimension to share the latent space and generate corresponding latent features.

[0031] Then, step S14 is executed, where the hierarchical grouping and vectorization bottleneck layer clusters multiple groups of latent features to generate multiple codebooks. Each group of latent features is clustered separately, thus generating a different codebook. This codebook is not trained using a loss function and is not the optimal codebook.

[0032] The loss function includes reconstruction loss, vector quantization loss, and hardware bit-width constraint term, i.e., formula ,in, This is the reconstruction loss. The preset weighting coefficients, This is the Vector Quantization Loss. middle, To preset weights, The residual value, The hardware bit-width constraint output is the maximum value between zero and the calculated value. The calculated value is the difference between the residual value and the preset value. The residual value is obtained by compressing the centroids of the latent feature and its corresponding codebook. The centroid of the codebook corresponding to the latent feature is found by calculating the L2 norm distance between each centroid of the latent feature and its corresponding codebook, and then finding the centroid with the smallest L2 norm distance. The centroid compression operation between the latent feature and its corresponding codebook can use a subtractor to subtract the latent feature from its corresponding centroid to obtain the residual value.

[0033] Then, step S15 is executed, where the compressed network architecture is trained and optimized using a loss function to generate multiple optimal codebooks. A publicly available simulator is used for optimization training.

[0034] After obtaining the optimal codebook, compression can be performed on any heterogeneous data of the on-chip system. Specifically, step S16 is executed to obtain the data to be compressed, query the target centroid closest to multiple optimal codebooks, and compress the data to be compressed based on the closest centroid to obtain compressed data. The data to be compressed is the heterogeneous data to be compressed, forcibly packaged and divided into fixed-size, fixed-length micro-blocks, each 4×4 bytes in size.

[0035] The target centroid that is closest to the data to be compressed and multiple optimal codebooks can be found by calculating multiple L2 norm distances between the data to be compressed and the optimal codebooks. The centroid with the smallest L2 norm distance is the target centroid.

[0036] The steps for compressing data based on the optimal center of gravity include: compressing the data based on the optimal center of gravity to obtain target residual values. There can be multiple target residual values. For example, an image with R, G, and B channels has almost identical gradient directions and edge contours at the same micro-block location, with only slight deviations in specific brightness and color values. Therefore, an image with R, G, and B channels corresponds to one optimal center of gravity, but the residual values ​​are different, resulting in three target residual values ​​for each optimal center of gravity.

[0037] Determine if the target residual value is less than the preset value. If so, output the target residual value and the index information of the highest quality core as compressed data. If the target residual value is greater than the preset value, discard the target residual value and output the data to be compressed as outlier data.

[0038] After obtaining compressed data or outlier data, the following steps are performed: record the address information of the compressed data or outlier data in a mapping table, which also includes a compression flag for the compressed data or outlier data.

[0039] After obtaining the compressed data, a decompression step can be performed. Specifically, the decompression steps include: obtaining a mapping table; determining whether the compression flag is valid based on the mapping table; and if it is valid, retrieving the compressed data from the address information corresponding to that flag. For example, if the compression flag is a single binary digit, the compression flag is represented by a binary 1, and the uncompressed flag by a binary 0. Next, obtaining index information from the compressed data; querying the highest quality center based on the index information; and calculating the decompressed data based on the target residual value and the highest quality center. The decompressed data can be calculated using an adder, adding the target residual value to the highest quality center. When the data to be decompressed is image data with three channels (R, G, and B), there are three target residual values. After obtaining the highest quality center, three adders are used to calculate the R, G, and B data for the highest quality center.

[0040] If the compression flag is not a compression flag, decompressed data is obtained from the outlier data. The compression steps are synchronized using a multiplexer based on the compression flag, and the decompressed data is output. Since data compression involves multiple steps, while outlier data is processed in a single step, the compression flag is delayed by four clock cycles before being input to the multiplexer. The multiplexer then receives the data selected by the compression flag. Because the compressed data and outlier data are stored in different locations, the multiplexer obtains the decompressed data from these different locations using the compression flag. The four-clock-cycle delay of the compression flag ensures synchronized data output.

[0041] This embodiment generates multiple sets of latent data through multiple encoders, all of which are mapped to the same dimension. Each set of latent data generates a codebook for the latent data, thereby reducing the error caused by forced fitting from a single codebook during subsequent compression. Since all sets of latent data are mapped to the same dimension, the data to be compressed is compressed into a single size, such as 8-bit index information and 8-bit target residual value. This allows for a shared decompression engine. Compared to traditional solutions where image, weights, and code each rely on a variable-length entropy decoder, resulting in hardware logic redundancy, this invention reduces the use of on-chip hardware resources and the area of ​​the on-chip system. Furthermore, by including a hardware bit-width constraint term in the loss function, the network output can be forced to adapt to a specific physical circuit bit width, reducing chip computational resources.

[0042] Computer device embodiment: The computer device in this embodiment includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the above-described method for compressing heterogeneous data.

[0043] A computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that a computer device may include more or fewer components, or a combination of certain components, or different components; for example, a computer device may also include input / output devices, network access devices, buses, etc.

[0044] Examples of computer-readable storage media: The method for compressing heterogeneous data in a computer device described in the above embodiments can be stored in a computer-readable storage medium in the form of a computer program. When the computer program is executed by a processor, it can complete the steps of the above embodiments of the method for compressing heterogeneous data in a computer device. The computer-readable storage medium can be a computer-readable signal medium or a storage medium. The storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0045] The above are merely preferred embodiments of the present invention, but the design concept of the invention is not limited thereto. Without departing from the concept of the present invention, many other equivalent embodiments may be included. Those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the protection scope of the present invention.

Claims

1. A method for compressing heterogeneous data, the method comprising: Input heterogeneous data from the on-chip system into the compressed network architecture; The compressed network architecture includes physical domain routers, encoders, hierarchical packet vectorization bottleneck layers, and loss functions. The physical domain router performs macro-classification on the heterogeneous data to obtain classified data; There are multiple encoders, and the multiple encoders extract features from the classification data corresponding to the encoder to obtain multiple sets of latent features. The multiple sets of latent features are mapped to the same dimension. The hierarchical grouping vectorization bottleneck layer clusters the multiple sets of potential features to generate multiple codebooks, with each codebook corresponding to a set of potential features. The loss function includes reconstruction loss, vector quantization loss and hardware bit width constraint term. The hardware bit width constraint term outputs the maximum value between zero and the calculated value. The calculated value is the difference between the residual value and the preset value. The residual value is obtained by compressing the centroid of the latent feature and the codebook corresponding to the latent feature. The compressed network architecture is trained to generate multiple optimal codebooks; Obtain the data to be compressed, query the target centroid that is closest to the data to be compressed among multiple optimal codebooks, compress the data to be compressed based on the highest centroid, and obtain the compressed data.

2. The method for compressing heterogeneous data according to claim 1, characterized in that: The classification data includes at least one of the following: AI tensor domain, image domain, and code domain; The encoder includes at least one of the following: a tensor encoder, a visual encoder, and a code encoder; The potential features include at least one of the following: channel distribution features, visual features, and temporal features.

3. The method for compressing heterogeneous data according to claim 2, wherein the method comprises: The steps for the encoder to extract features from the classification data corresponding to the encoder include: The tensor encoder extracts features from the AI ​​tensor domain to obtain channel distribution features; The visual encoder extracts features from the image domain to obtain visual features; The code encoder extracts features from the code domain to obtain temporal features.

4. The method for compressing heterogeneous data according to claim 3, wherein the method comprises: The optimal codebook includes a tensor-domain codebook, an image-domain codebook, and a code-domain codebook; The tensor domain codebook includes channel distribution primitives, the image domain codebook includes visual primitives, and the code domain codebook includes temporal primitives.

5. The method for compressing heterogeneous data according to any one of claims 1 to 4, the method comprising: The steps for compressing the data to be compressed based on the highest quality data include: The data to be compressed is compressed according to the highest quality core to obtain the target residual value; Determine whether the target residual value is less than the preset value. If so, output the target residual value and the index information of the highest quality core as compressed data. If the target residual value is greater than the preset value, the target residual value is discarded, and the data to be compressed is output as outlier data.

6. The method for compressing heterogeneous data according to claim 5, wherein the method comprises: After obtaining the compressed data or the outlier data, the following is also performed: The address information of the compressed data or the address information of the outlier data is recorded in a mapping table, which also includes a compression flag bit for the compressed data or the outlier data.

7. The method for compressing heterogeneous data according to claim 6, wherein the method comprises: After obtaining the compressed data, the following steps are performed: Decompression. The decompression step includes: Obtain the mapping table, and determine whether the compression flag is a compression flag based on the mapping table; If the compression flag is set to compression, then the compressed data is retrieved from the address information corresponding to that compression flag. The index information is obtained from the compressed data, the highest quality core is queried according to the index information, and the decompressed data is calculated based on the target residual value and the highest quality core of the compressed data. If the compression flag is not a compression flag, then the decompressed data is obtained from the outlier data; The compression steps are synchronized using a multiplexer based on the compression flag, and the decompressed data is output.

8. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program that, when executed, implements the method for compressing heterogeneous data as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed, implements the method for compressing heterogeneous data as described in any one of claims 1 to 7.

Images