Neural network parameter compression method based on bit plane run length coding

Abstract

The invention belongs to the technical field of artificial intelligence and data compression. In particular to a neural network parameter compression method based on bit-plane run-length coding, which aims at bit-level distribution characteristics of quantization parameters, and realizes efficient lossless compression through sign bit separation, negative non-sign bit flipping, high-bit-plane expansion and extended run-length coding. The method comprises the following steps: separating sign bits and flipping negative non-sign bits to optimize high-order sparsity; selecting the maximum m bit planes (m ranges from 2 to 5 and can be adaptively selected) from the next bit of the sign bit, and expanding the m bit planes into a one-dimensional bit stream according to columns; extended run length types of all zero segments, all one segment, a zero segment ending with a single 1 and a zero segment ending with two continuous 1 are adopted for coding, and the run length supports multi-gear self-adaptive bit width. A code stream formed by combining the low-order transparent transmission part, the sign bit and the run length code is output, and the efficiency can be further improved by combining numerical-level sparse coding. By optimizing bit plane distribution and a run-length mode, the compression rate is remarkably improved, hardware is easy and convenient to implement, and the method is suitable for storage and transmission of neural network weights and intermediate features and particularly suitable for end-side equipment with limited resources and efficient cloud storage scenes.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and data compression technology, and relates to a lossless compression method for quantized neural network model parameters. Specifically, it relates to a neural network parameter compression method based on bit-plane run-length encoding, falling within the technical scope of neural network model deployment optimization. This invention is particularly suitable for the efficient storage and transmission of weight parameters (such as convolutional layer weights and fully connected layer weights) and intermediate activation features (such as feature map data output by each network layer) of quantized neural networks. Background Technology

[0002] According to available data, deep neural networks have been widely used in computer vision, speech recognition, and natural language processing. As model size continues to increase, the number of parameters in neural networks reaches hundreds of millions or even trillions. For example, large language models like Llama 3.1 have 405 billion parameters, and DeepSeek V3 reaches 671 billion parameters. In image and video processing, convolutional neural networks (CNNs) are also massive in scale; for example, ResNet-50 has approximately 25 million parameters, while ConvNeXt V2 has variants with hundreds of millions of parameters. This high computational and storage requirement means that models in the cloud rely on high-performance GPU clusters, resulting in high energy costs. Meanwhile, for edge deployment, lightweight models such as EfficientNet-B0 (approximately 5.3 million parameters), MobileNetV3 (approximately 3-5 million parameters), and MoViNet (approximately 3-10 million parameters) are widely used in image classification, object detection, and video analysis, and can run on smartphones or IoT devices, but still require optimization to cope with limited resources.

[0003] To reduce storage and bandwidth pressure, existing technologies commonly employ parameter quantization (such as INT8 and INT4) and lossless compression methods (such as Huffman coding, arithmetic coding, and run-length encoding). Furthermore, model sparsity pruning techniques are widely used for further optimization. For example, L1 / L2 regularization can induce weights to approach zero during training, or iterative pruning methods can be used to remove unimportant connections (such as threshold pruning based on magnitude or gradient), thereby increasing the proportion of zero in the parameter matrix to over 50%, or even higher. While these sparsity methods effectively reduce computation, they primarily target value-level sparsity, ignoring the bit-level distribution characteristics of the quantized parameters. For instance, for model parameters that approximate a Gaussian distribution after quantization, the values ​​are concentrated near zero, and high-bit values ​​are highly compressible.

[0004] Bit-level compression methods address this characteristic by decomposing quantization parameters into bit planes, fully utilizing bit-level sparsity to further improve compression ratios. This approach not only maintains hardware friendliness but also significantly improves memory access requirements: compressed data volumes are reduced, decreasing memory access frequency and bandwidth usage, thereby lowering latency and power consumption. It is particularly suitable for high-efficiency cloud storage and real-time edge deployment scenarios.

[0005] Based on the current state of the technology, the inventors of this application intend to provide a compression method that maintains hardware friendliness while optimizing for the distribution characteristics of quantized weights and activation data. Specifically, this provides a neural network parameter compression method based on bit-plane run-length encoding. Summary of the Invention

[0006] The purpose of this invention is to provide a neural network parameter compression method based on bit-plane run-length encoding, building upon existing technologies. This invention achieves efficient and lossless compression by addressing the bit-level characteristics of quantization parameters. The method combines column-direction bit-plane expansion, extended run-length types, and sparse aggregation strategies to effectively improve the compression ratio.

[0007] To achieve the above objectives, the technical solution proposed by this invention is as follows: A neural network parameter compression method based on bit-plane run-length encoding includes: 1. Sign Bit Processing and Negative Number Flipping: Define the bit plane as a two-dimensional plane after the quantization parameters are split into bits, where the k-th bit plane consists of the k-th bits of all parameters. Separate the sign bit from the input quantization parameters and flip (i.e., invert) the remaining bits of negative numbers. This is because in two's complement representation, the higher bits of negative numbers often appear as consecutive "1"s, and direct compression would result in a large number of redundant run-length encoding segments. By flipping, the negative part can be converted into a form closer to the distribution of positive numbers, resulting in more consecutive "0"s in the higher bit plane, thereby improving run-length encoding efficiency.

[0008] 2. High-level bit plane expansion: Starting from the bit below the sign bit, select the highest m bit planes and expand them into a one-dimensional bit stream along the column direction, which serves as the object to be encoded. Expanding along the column direction maintains the continuity of bits within the same bit plane, which is beneficial for forming long runs. The remaining low-level bits and the sign bit are directly retained as the pass-through part. The parameter m can be a fixed value, or it can be adaptively selected by analyzing and comparing the compression ratio under different m values. For example, for the input model parameters, calculate the compression ratio for m=2, 3, 4, and 5 respectively. If m=4 is found to be the best, then m=4 is selected for compression to maximize the compression efficiency.

[0009] 3. Extended mode run-length encoding: Perform custom run-length encoding on the high-bit plane stream.

[0010] a. Run Types: The run types include: all-zero segments, all-single segments, zero segments ending with a single "1" (0…01), and zero segments ending with two consecutive "1"s (0…011). Existing run-length encoding only supports all-zero / all-single segments, which cannot efficiently encode bit-level patterns of 'all zeros followed by one or two 1s' (such as the high bits of small numerical parameters), resulting in limited compression ratio. This extended mode can efficiently represent the sparse small-value characteristics commonly found in quantization parameter distributions, avoiding splitting short, abruptly changing bits into multiple run segments. A 2-bit index is used to identify the run type: {00} corresponds to all-zero segments, {01} corresponds to zero segments ending with a single "1", {10} corresponds to zero segments ending with two "1"s, and {11} corresponds to all-single segments.

[0011] b. Run Recording Method: Each run segment is represented by two parts: a run type field and a run length field. The run type field is used to distinguish the four types of run patterns mentioned above, and the run length field is used to indicate the number of consecutive bits.

[0012] c. Adaptive Bit Width: The bit width for the run length can be either fixed or adaptive. In the adaptive scheme, the run length can be divided into multiple intervals, for example: 1-2 uses 1 bit, 3-6 uses 2 bits, 7-22 uses 4 bits, and 23-149 uses 8 bits. A 2-bit index is used to identify the length bit width: {00} corresponds to 1 bit, {01} corresponds to 2 bits, {10} corresponds to 4 bits, and {11} corresponds to 8 bits. The encoder selects the corresponding bit width based on the interval to which the current run length belongs, avoiding wasted bits.

[0013] 4. Sparse Coding: Before bit-plane unpacking, sparse coding can be performed on the input parameters. Zero values ​​are compressed using numerical-level sparse coding, and then non-zero values ​​are fed into bit-plane run-length encoding, thus forming a hierarchical compression structure. Typical methods include: a. Sparse masking method: Generate a binary mask for each parameter (0 represents zero value, 1 represents non-zero value), and then perform bit-plane run-length encoding compression on the non-zero values; b. Sparse run-length encoding: Run-length encoding is performed on zero-value or non-zero-value segments, and the length and type (zero-value or non-zero-value) of each segment are recorded. Only the non-zero-value sequence is input into the bit-plane run-length encoding. 5. Output bitstream: Combine the low-order bits, sign bit, sparse index, and run-length encoding results into the final output bitstream.

[0014] Compared with the prior art, the present invention has the following advantages: 1. Improve compression ratio by leveraging bit-level characteristics. Design the bit-plane distribution of quantization parameters to fully exploit the continuous "0" characteristics and sparse small value pattern of the high bit plane, thus making up for the shortcomings of existing value-level compression. 2. It can be used to compress neural network weight parameters and intermediate activation features, covering the needs of cloud storage and real-time transmission on the edge; 3. Extended run types can efficiently represent the distribution characteristics of sparse small values ​​and reduce the number of run segments; 4. The adaptive bit-width encoding mechanism avoids wasting bits in the run length; 5. Supports fixed or adaptive selection of m value, and can be combined with sparse coding to adapt to high sparsity scenarios; 6. Run-length encoding has simple logic, requiring only shift registers, counters, and a small amount of logic control circuitry, making it suitable for FPGAs / ASICs. Attached Figure Description

[0015] Figure 1 The flowchart of the method of the present invention includes five steps: input quantization parameters (101), sign bit processing and negative number flipping (102), high-bit plane expansion (103), run-length encoding (104), and output bit stream (105); Figure 2 The flowchart of the method of the present invention combined with sparse coding is shown in Figure 1. The steps of sparse detection (202) and sparse index generation (203) are added. Subsequent bit-plane run-length coding is only performed on non-zero data (204). Figure 3 This is a comparison diagram of negative numbers before and after flipping. The left side is the original complement of -4 (11111100), and the right side is the result of flipping the unsigned bits after separating the sign bit (10000011), showing the optimization effect of consecutive "0" bits in the high bits after flipping. Figure 4 The diagram illustrates the selection process for the adaptive bit plane number m. By applying compression with the same input parameter at m=2, 3, 4, and 5 respectively, the size of each bitstream is calculated and compared, and the m value that generates the smallest bitstream is selected as the optimal parameter. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description is provided in conjunction with specific embodiments and accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. This implementation method is only used to explain this invention and does not limit the implementation scenarios of this invention.

[0017] like Figure 1 A flowchart illustrating a compression method provided for an embodiment of the present invention is shown, and the method flow is as follows.

[0018] Step 101: Input the quantized weights or activation values.

[0019] In this invention, the compressible quantized model parameters can be weights or activation values ​​quantized with precision such as INT8 or INT16. In this embodiment, INT8 weights {10,7,−1,−4,0,1,71,14,0,−2,−5,3,20,16,11,−4} are selected, where each value is an 8-bit weight parameter.

[0020] Step 102: Flip the non-sign bit of the negative number.

[0021] First, the sign bit is separated from the input data. For negative numbers, the remaining bits below the sign bit are flipped bit by bit to avoid the problem of long run length caused by redundant "1"s in the high bits of the two's complement. Figure 3 This is a diagram comparing the negative number before and after flipping. The left side represents the original two's complement form of the input data -4, and the right side represents the form after separating the sign bit and inverting the remaining bits. It can be seen that after flipping, the high-order bits are consecutively zero, which is more conducive to forming a run segment.

[0022] The processed data is shown in Table 1. Each row represents the bit-level representation of an INT8 data (after flipping the non-sign bit of a negative number). The first row in Table 1 represents the bit index, where S represents the sign bit, and the subsequent bits 7 to 1 represent the bits from the second most significant bit to the least significant bit. The first column in Table 1 represents the specific numerical value of the data in the illustrated example. For example, the second row of Table 1 records the bit representation of the first data 10 as 00001010. Another example is the fourth row of Table 1, which records the representation of -1 after flipping the non-sign bit. The two's complement representation of -1 is 11111111, and after flipping the non-sign bit, it becomes 10000000.

[0023] Table 1: Bit representation of a negative number after flipping the non-sign bits (S is the sign bit, 7-1 is the non-sign bits from high to low).

[0024]

[0025] Step 103: Expand the m-bit high-bit plane.

[0026] Starting from the bit after the sign bit, select the highest m bit planes and expand these m planes into a one-dimensional bit stream along the column direction (m is typically 2, 3, 4, or 5). For the data in the table, for example, for m=2, first connect all bits of the 7th bit plane, then immediately connect all bits of the 6th bit plane to form a long one-dimensional bit stream. The 7th bit column is {0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0}, and the 6th bit column is {0 0 0 0 0 0 0 0 0 0 0 0 0 0 0}. Concatenate and expand them along the column direction to form a one-dimensional representation {0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0}, obtaining the data for the next step of run-length encoding.

[0027] Step 104: Adaptive run-length encoding of the high-order bits.

[0028] Run-length encoding is performed on the bitstream. Run-length types include: As shown in Table 2: all zeros; all one; zeros ending with a single "1" (0…01); zeros ending with two consecutive "1"s (0…011). A 2-bit index is used to encode the run type: 00 corresponds to all zeros; 01 corresponds to a zero ending with a single "1"; 10 corresponds to a zero ending with two consecutive "1"s; 11 corresponds to an all one.

[0029] Table 2: Run Path Patterns and Index

[0030] Each run segment consists of a run type and a run length. The run type is represented by a fixed field, while the run length uses an adaptive bit width. One adaptive bit width setting is: 1 bit for length ≤ 2, 2 bits for lengths 3–6, 4 bits for lengths 7–22, and 8 bits for lengths 22–149. A 2-bit index is used to encode the run width: 00 corresponds to 1 bit width; 01 corresponds to 2 bits; 10 corresponds to 4 bits; and 11 corresponds to 8 bits.

[0031] In this embodiment, most of the bitstream consists of zero segments or short "0…01" segments, which significantly reduces the number of runs after encoding. Taking m=3, for this embodiment, the first run is six consecutive 0s ending with a single 1; the second run is 37 consecutive 0s ending with two consecutive 1s; and the third run is two 0s. This can be encoded as: run type {01, 10, 00}; run bit width {01, 11, 00}; run length {11, 00001010, 1}.

[0032] Step 105: Output the compressed bitstream.

[0033] The final output bitstream consists of the following parts: the sign bit and the low-order bits, and the result of run-length encoding. In this embodiment, for the compression mode with m=3, the compression ratio is 1.24x.

[0034] Table 3: Comparison of compression ratios under different m parameters

[0035] As shown in Table 3, the compression ratio is optimal when m=3. Figure 4 A schematic diagram of the selection process for the adaptive bit plane number m.

[0036] like Figure 4 As shown, this embodiment provides an adaptive bit plane number m selection process. Specifically, it includes: for the input parameter data, selecting m=2, m=3, m=4, and m=5 bit planes respectively to perform bit plane unrolling and run-length encoding to obtain the corresponding compressed bitstreams; calculating the length of the compressed bitstreams or the corresponding compression ratio for each; selecting the m with the highest compression ratio from the results corresponding to different m values, and outputting the optimal m compressed bitstream. In practical applications, different m values ​​can be tried on the data block, and the m with the highest compression ratio can be selected as the final solution, thereby achieving adaptive bit plane selection.

[0037] During the compression process, the above encoding process can be performed sequentially on each model parameter group starting from the beginning of the model parameters. After the encoding process of all model parameter groups is completed, the compressed bitstream described above can be obtained.

[0038] like Figure 2 As shown, the present invention can further introduce numerical-level sparse coding.

[0039] For example, the parameter input in step 201 is INT8 sparse weight {0 0 0 4 0 0 0 0 0 0 0 0 0 1 -1 2 0 9 0 0 0 0 -11 0 9 1 3 -2 0 0 0 0}, with a sparsity of 68.75%, which is suitable for sparse coding 202.

[0040] The encoding method in step 202 can be any sparse encoding method, such as bitmap sparse encoding, CSR encoding, run-length encoding, etc. For example, using bitmap sparse encoding, we will get sparse encoding index 203: {0 0 0 1 0 0 0 0 0 0 0 00 0 1 1 1 0 1 0 0 0 0 1 0 1 1 1 1 0 0 0 0} and non-zero data 204: {4 1 -1 2 9 -11 9 1 3-2}.

[0041] Step 205 continues by performing adaptive bit-plane run-length encoding on the non-zero data retained after sparse encoding, such as... Figure 1 As shown in the flowchart, by comparing the effects of different values ​​of m, the maximum compression ratio of 1.29x is obtained when m=3. The bit-plane run-length encoding index 206 is obtained: run type index {00}, run bit width index {11}, run length {00011110}, and uncompressed sign bit and low bit 207.

[0042] Finally, the sparse coding index 203, the bit-plane run-length coding index 206, and the sign bit and low bit 207 are integrated into the compressed bitstream 208. For the parameter {0 0 0 4 0 0 0 0 0 0 0 0 0 1 -1 2 0 9 00 0 0 -11 0 9 1 3 -2 0 0 0 0} in this embodiment, the compression ratio can reach 2.72x.

Claims

1. A neural network parameter compression method based on bit-plane run-length encoding, characterized in that, Includes the following steps: (1) Separate the sign bit of the input quantization parameter. The non-sign bit of the negative number is flipped bit by bit. Starting from the bit after the sign bit, select the highest m bit plane as the bit plane to be encoded, and the remaining low bits as the pass-through part. The value of m is 2, 3, 4 or 5. (2) Read the bit values ​​of the m bit planes sequentially in the column direction, and unfold the two-dimensional bit planes into a continuous one-dimensional bit stream; (3) Perform run-length encoding on the one-dimensional bit stream. The run-length types include: all zero segments (multiple consecutive 0s), all one segment (multiple consecutive 1s), zero segments ending with a single 1 (multiple consecutive 0s followed by 1s), and zero segments ending with two consecutive 1s (multiple consecutive 0s followed by 11s). Each run segment is represented by a run-length type identifier and run-length information. The run-length type identifier is a 2-bit index. (4) Output a symbol bit stream consisting of the low-order pass-through part and the sign bit, and a compressed bit stream formed by the combination of run-length encoding results.

2. The method according to claim 1, characterized in that, Before performing step (2), a sparse coding step is further included: the input quantization parameters are sparsely encoded by generating a sparse index, such as a binary mask or a zero-segment run record to mark the position of the zero value, and only the non-zero value part is extracted to perform subsequent bit-plane run-length encoding. The sparse index and the encoding result are included in the output bit stream.

3. The method according to claim 1 or 2, characterized in that, The run length adopts multi-level bit width adaptive encoding, which is as follows: the run length is divided into multiple numerical intervals, each interval corresponds to a preset bit width, and during encoding, the corresponding bit width is selected according to the interval to which the current run length belongs to record the length information, so as to reduce bit redundancy.

4. The method according to any one of claims 1 to 3, characterized in that, The compression method is applied to the storage and transmission of weight parameters or intermediate activation features of neural networks. The weight parameters include convolutional layer weights, fully connected layer weights, etc., and the intermediate activation features include feature map data output by each network layer.

5. The method according to claim 1, characterized in that: The value of m for selecting the highest m bit planes can be fixed or adaptively selected (range 2-5). When adaptively selected, the input parameters are bit plane run-length encoded by using m=2, 3, 4, and 5 respectively, and the size of the compressed bit stream corresponding to different m values ​​is calculated. The size of the compressed bit stream is obtained by performing steps (1)-(4) on the same input parameter under different m values. The value of m that generates the smallest compressed bit stream is selected as the optimal number of bit planes for the current parameter.

Images