An entropy coding method for quantized model parameters with high decoding efficiency

Abstract

The application discloses an entropy coding method for quantized model parameters, which is efficient in decoding, and comprises the following steps: training a convolutional neural network (CNN) model by using a standard deep learning method, including weight learning and optimization of each layer in the CNN model; adopting a bit allocation optimization method in the trained model; adopting a dead zone quantization technology to quantize the weights of the model; further encoding the quantized weights by using an improved Tunstall, adopting a fixed-length entropy coding technology to compress the weight data; loading the compressed model into a target hardware platform to perform reasoning; decoding the encoded weight data by using an improved Tunstall decoding; performing neural network operation on the decoded model on the hardware platform, and finally outputting a reasoning result; and the application optimizes the CNN model, and significantly improves the reasoning efficiency while maintaining the reasoning precision.

Description

Technical Field

[0001] This invention relates to the field of optimization of convolutional neural network (CNN) models, and in particular to an efficient entropy encoding method for decoding quantized model parameters. Background Technology

[0002] Despite significant progress in the application of deep learning models, especially convolutional neural network (CNN) models, some major shortcomings and limitations remain, particularly in efficient inference on resource-constrained devices. These shortcomings include: High computational and memory consumption: Existing CNN models typically contain millions to billions of parameters, requiring substantial computational resources and memory storage. On resource-constrained platforms such as embedded devices, mobile devices, and smart hardware, existing CNN models often struggle to run efficiently, resulting in slow inference speeds or even inability to process data in real time. Limited model compression effectiveness: Although many compression techniques (such as pruning, quantization, and low-rank decomposition) have been proposed, they have limitations in balancing compression ratio and inference speed. Existing techniques often require significant compromises between compression ratio and accuracy during model compression, resulting in compressed models failing to achieve an ideal balance between accuracy and inference efficiency. High system overhead: Most existing CNN model compression methods introduce significant system overhead during operations such as quantization and pruning, adding additional computational burden. These overheads may prevent the model from meeting real-time response requirements during inference, especially in applications that require fast processing and rapid decision-making, such as autonomous driving and intelligent transportation. Poor hardware accelerator adaptability: many current CNN model compression methods do not fully consider the characteristics of hardware accelerators, resulting in the compressed model not maximizing its inference performance on some hardware.

[0003] In scenarios such as embedded devices, mobile devices, the Internet of Things, and autonomous driving, convolutional neural network (CNN) models are often difficult to deploy efficiently due to their high computational complexity and memory consumption. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies in the efficient deployment of convolutional neural network (CNN) models, and to provide an efficient entropy encoding method for decoding quantized model parameters, which optimizes the bit allocation and quantization strategy of the CNN model, improves the compression ratio of the model, and significantly improves inference efficiency while maintaining inference accuracy.

[0005] To achieve the above objectives, this invention provides an efficient method for decoding entropy encoding of quantized model parameters, comprising the following steps:

[0006] Step S1: Model Training: Train the Convolutional Neural Network (CNN) model using standard deep learning methods to obtain the original uncompressed model, including weight learning and optimization for each layer in the CNN model;

[0007] Step S2: Bit allocation optimization: A bit allocation optimization method is used in the trained model; the bit width of each convolutional layer is dynamically adjusted according to the contribution of each channel to the output accuracy; this process is completed through rate-distortion optimization.

[0008] Step S3: Dead Zone Quantization: The model weights are quantized using dead zone quantization technology. During quantization, small weights are compressed to zero to reduce storage requirements. The quantization step size and dead zone interval are adaptively adjusted by the optimization algorithm.

[0009] Step S4: Improved Tunstall Encoding: The quantized weights are further encoded using improved Tunstall encoding, which uses fixed-length entropy encoding to compress the weight data; improved Tunstall encoding results in the encoded weight data occupying less storage space.

[0010] Step S5: Load the compressed model: The compressed model is loaded into the target hardware platform for inference;

[0011] Step S6: Real-time Decoding: During inference, the encoded weight data is first decoded using an improved Tunstall decoding method. The decoding process uses fixed-length encoding to ensure that the decoding process is efficient and real-time. The decoded weights are used for inference calculations in the CNN model.

[0012] Step S7: Inference Calculation: The decoded model performs neural network operations on the hardware platform and finally outputs the inference result.

[0013] As a preferred embodiment, the bit allocation method with rate-distortion optimization in step S2 dynamically adjusts the bit width according to the importance of each convolutional layer and channel, avoiding the shortcomings of uniform bit allocation in traditional compression methods, and achieving a higher compression ratio and lower computational overhead.

[0014] Preferably, step S2 further includes the following steps;

[0015] Step S21: Objective of bit allocation optimization; Define a deep convolutional neural network F, and give W i and N i Let and represent the weights and number of channels of the i-th convolutional layer, respectively; the goal is to optimize the bit allocation of each channel; by quantizing the weights of each layer, a modified output is obtained. Then Y and are measured by the expected error. The difference between them (i.e., distortion); this distortion value is the error generated during the quantization process, and the formula is:

[0016]

[0017] Where E represents the expectation of the distribution of the input image, Indicates the Euclidean distance;

[0018] Step S22: Bit allocation optimization; The goal of bit allocation optimization is to find the optimal bit allocation for each channel to minimize output distortion while satisfying the bit rate constraint of the compressed model. The optimization problem can be expressed as:

[0019]

[0020]

[0021] Where, r ij b represents the bit rate allocated to the j-th channel of the i-th layer. ij N is the quantization bit width of the channel. i L is the number of channels in the i-th layer, L is the total number of convolutional layers, and R is the constraint on the model size (i.e., the total bit rate).

[0022] Step S23: Optimize using the additive property. The additivity property is used to simplify the output distortion calculation, thereby solving the bit allocation problem. The output distortion of each channel can be viewed as the sum of the quantization errors of a single channel, i.e.:

[0023]

[0024] By utilizing Taylor series expansion, assuming the neural network is differentiable and has a small quantization error, the optimal bit allocation can be solved using the Lagrange optimization method.

[0025] Step S24: Optimal bit allocation. The optimal bit allocation problem is solved using the Lagrange formula; the Lagrange cost function expression is:

[0026]

[0027] Optimal bit allocation can be achieved by setting the Lagrange cost function with respect to each r. ij The partial derivative is zero, thus obtaining the optimal solution; this solution represents the bit width that should be allocated to each channel, thereby achieving the minimum output distortion.

[0028] Preferably, step S3 further includes: introducing adaptive dead zone quantization technology to force small weights to zero, reducing storage requirements and further improving the model compression ratio; at the same time, by finely adjusting the dead zone size and quantization step size, the inference accuracy of the model is maintained.

[0029] Step S31: Dead Interval: Within this interval, all quantized values ​​are forced to be set to zero; that is, any value less than a certain threshold is considered 0. This helps to eliminate small-amplitude noise and unimportant weights, and reduce storage requirements.

[0030] Step S32: Quantization step size: It controls the interval of quantization values ​​and is a fixed constant; during the quantization process, the weights are discretized according to the quantization step size;

[0031] Step S33: Formula:

[0032]

[0033] q(x) represents the quantized value;

[0034] θ is the size of the dead zone (if |x| < θ, then q(x) = 0);

[0035] Δ is the quantization step size;

[0036] b is the quantization bit width, which determines the precision of the quantization value;

[0037] sign(x) is used to preserve the sign, ensuring the correct handling of negative and positive values;

[0038] Step S34: Dead Zone Quantization: When the absolute value of a weight is less than the set dead zone size θ, the weight is quantized to 0, that is, small values ​​are discarded; this helps to reduce unimportant weight data and further compress the amount of data.

[0039] Step S35: Normal quantization: For weights larger than the dead zone, quantize by step size Δ and map them to the corresponding quantized values. If the weight x is within the dead zone, then q(x) = 0; otherwise, discretize by quantization step size Δ and ensure that the range of values ​​is within the effective bit width range.

[0040] Preferably, step S4 further includes the following steps: improved Tunstall encoding; using fixed-length entropy encoding, which enables the compressed weights to be stored and decoded in a more efficient manner, reducing the computational burden during the decoding process, and is especially suitable for hardware accelerator platforms.

[0041] Preferably, the improved Tunstall encoding uses a decomposition operation, which decomposes the input data before applying Tunstall encoding. This decomposition operation can further improve the compression ratio. The idea behind the decomposition operation is to decompose larger symbols into multiple smaller symbols, thereby optimizing the data encoding method.

[0042] Preferably, the improved Tunstall encoding is an improvement on the traditional Tunstall encoding. It reduces the value to be encoded by a base value and decomposes it. Improved Tunstall encoding is then performed on the decomposed characters. The improved Tunstall encoding adopts a tree structure and encodes based on the frequency of character occurrence, thereby maximizing the compression ratio. Since Tunstall encoding maps multiple characters to one code, multiple characters can be decoded simultaneously during the decoding process, achieving real-time and fast decoding.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] 1. This invention employs a rate-distortion optimized bit allocation method, introducing Lagrange optimization to ensure that different channels of each convolutional layer receive optimal bit allocation based on their contribution to the model output. This method improves compression ratio and inference accuracy by minimizing output distortion and precisely adjusting the quantization bit width of each channel within given storage and computational resource constraints, while avoiding excessive precision loss. It allocates bits more efficiently than traditional methods, significantly reducing storage requirements and computational complexity, and increasing the compression ratio. By optimizing bit allocation, the accuracy of the compressed model during inference is guaranteed, avoiding precision loss caused by over-compression.

[0045] 2. This invention innovatively combines dead-zone quantization with improved Tunstall coding, optimizing the dead-zone interval and quantization step size. By precisely selecting an appropriate dead-zone size, the number of small-amplitude weights is reduced, significantly improving the compression ratio. Furthermore, this method automatically selects the most suitable parameter configuration based on the weight distribution of different models by enumerating and optimizing the quantization step size and dead-zone interval, further reducing the storage requirements for redundant information. Dead-zone quantization forces small-amplitude weights to zero, effectively eliminating invalid weights and reducing storage requirements. Combined with improved Tunstall coding, this further improves the compression ratio. With the introduction of dead-zone quantization, only the more important weights are retained, thereby reducing the computational burden during inference and maintaining efficient inference speed.

[0046] 3. This invention employs improved Tunstall coding, a fixed-length entropy coding method. Unlike traditional variable-length coding (such as Huffman coding), improved Tunstall coding can encode multiple symbols into fixed-length bits, and the decoding process uses byte-oriented operations, making the decoding process more efficient. This method, combined with dead-zone quantization, further optimizes model compression, reduces computational latency during decoding, and improves real-time inference capabilities. Improved Tunstall coding, through fixed-length encoding, avoids complex variable-length decoding processes, significantly accelerating decoding speed, making it particularly suitable for applications with high real-time inference requirements. On hardware accelerators, the fixed-length encoding decoding process is more adaptable, fully leveraging the parallel computing advantages of hardware accelerators to improve inference efficiency. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the Tunstall encoding and the improved Tunstall encoding provided by the present invention. Detailed Implementation

[0049] The technical solution of this embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiment is one embodiment of the present invention, and not all embodiments thereof. Based on this embodiment of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Please refer to Figure 1 This invention provides an efficient method for decoding entropy encoding of quantized model parameters.

[0051] Example 1: A compression scheme for convolutional neural network (CNN) models based on bit allocation optimization and dead-zone quantization.

[0052] In this embodiment, the compression process of the Convolutional Neural Network (CNN) model is divided into two main stages: the training stage and the inference stage. The specific workflow and the transmission relationships between the components are as follows:

[0053] The training phase includes: step S1: model training, step S2: bit allocation optimization, step S3: dead zone quantization, and step S4: improved Tunstall encoding.

[0054] The inference stage includes: step S5: loading the compressed model, step S6: real-time decoding, and step S7: inference calculation.

[0055] Step S1: Model Training: Train the Convolutional Neural Network (CNN) model using standard deep learning methods to obtain the original uncompressed model, including weight learning and optimization for each layer in the CNN model.

[0056] Step S2: Bit allocation optimization: A bit allocation optimization method is used in the trained model; the bit width of each convolutional layer is dynamically adjusted according to the contribution of each channel to the output accuracy; this process is completed by rate-distortion optimization method; the goal is to minimize the output distortion after compression while ensuring low computational and storage overhead.

[0057] The bit allocation method optimized by rate distortion in step S2 dynamically adjusts the bit width according to the importance of each convolutional layer and channel, avoiding the shortcomings of uniform bit allocation in traditional compression methods, and achieving a higher compression ratio and lower computational overhead.

[0058] Step S2 further includes the following steps;

[0059] Step S21: Objective of bit allocation optimization; Define a deep convolutional neural network F, and give W i and N i Let and represent the weights and number of channels of the i-th convolutional layer, respectively; the goal is to optimize the bit allocation of each channel; by quantizing the weights of each layer, a modified output is obtained. Then Y and are measured by the expected error. The difference between them (i.e., distortion); this distortion value is the error generated during the quantization process, and the formula is:

[0060]

[0061] Where E represents the expectation of the distribution of the input image, Indicates the Euclidean distance;

[0062] Step S22: Bit allocation optimization; The goal of bit allocation optimization is to find the optimal bit allocation for each channel to minimize output distortion while satisfying the bit rate constraint of the compressed model. The optimization problem can be expressed as:

[0063]

[0064] Where, r ijb represents the bit rate allocated to the j-th channel of the i-th layer. ij N is the quantization bit width of the channel. i L is the number of channels in the i-th layer, L is the total number of convolutional layers, and R is the constraint on the model size (i.e., the total bit rate).

[0065] Step S23: Optimize using the additive property. The additivity property is used to simplify the output distortion calculation, thereby solving the bit allocation problem. The output distortion of each channel can be viewed as the sum of the quantization errors of a single channel, i.e.:

[0066]

[0067] By utilizing Taylor series expansion, assuming the neural network is differentiable and has a small quantization error, the optimal bit allocation can be solved using the Lagrange optimization method.

[0068] Step S24: Optimal bit allocation. The optimal bit allocation problem is solved using the Lagrange formula; the Lagrange cost function expression is:

[0069]

[0070] Optimal bit allocation can be achieved by setting the Lagrange cost function with respect to each r. ij The partial derivative is zero, thus obtaining the optimal solution; this solution represents the bit width that should be allocated to each channel, thereby achieving the minimum output distortion.

[0071] Step S3: Dead Zone Quantization: The model weights are quantized using dead zone quantization technology. During quantization, small weights are compressed to zero to reduce storage requirements. The quantization step size and dead zone interval are adaptively adjusted by the optimization algorithm to achieve the best compression effect.

[0072] Step S3 further includes: introducing adaptive dead zone quantization technology to force small weights to zero, reducing storage requirements and further improving the model compression ratio; at the same time, by finely adjusting the dead zone size and quantization step size, the inference accuracy of the model is maintained.

[0073] Step S31: Dead Interval: Within this interval, all quantized values ​​are forced to be set to zero; that is, any value less than a certain threshold is considered 0. This helps to eliminate small-amplitude noise and unimportant weights, and reduce storage requirements.

[0074] Step S32: Quantization step size: It controls the interval of quantization values ​​and is a fixed constant; during the quantization process, the weights are discretized according to the quantization step size;

[0075] Step S33: Formula:

[0076]

[0077] q(x) represents the quantized value;

[0078] θ is the size of the dead zone (if |x| < θ, then q(x) = 0);

[0079] Δ is the quantization step size;

[0080] b is the quantization bit width, which determines the precision of the quantization value;

[0081] sign(x) is used to preserve the sign, ensuring the correct handling of negative and positive values;

[0082] Step S34: Dead Zone Quantization: When the absolute value of a weight is less than the set dead zone size θ, the weight is quantized to 0, that is, small values ​​are discarded; this helps to reduce unimportant weight data and further compress the amount of data.

[0083] Step S35: Normal quantization: For weights larger than the dead zone, quantize by step size Δ and map them to the corresponding quantized values. If the weight x is within the dead zone, then q(x) = 0; otherwise, discretize by quantization step size Δ and ensure that the range of values ​​is within the effective bit width range.

[0084] Step S4: Improved Tunstall Encoding: The quantized weights are further encoded using improved Tunstall encoding, which uses fixed-length entropy encoding to compress the weight data; improved Tunstall encoding results in the encoded weight data occupying less storage space.

[0085] Step S4 further includes the following steps: improved Tunstall encoding; using fixed-length entropy encoding, which enables the compressed weights to be stored and decoded in a more efficient manner, reducing the computational burden during the decoding process, and is especially suitable for hardware accelerator platforms.

[0086] The improved Tunstall encoding uses a decomposition operation. Before applying Tunstall encoding, the input data is decomposed. This decomposition operation can further improve the compression ratio. The idea behind the decomposition operation is to decompose larger symbols into multiple smaller symbols, thereby optimizing the data encoding method.

[0087] The improved Tunstall encoding is an improvement on the traditional Tunstall encoding. It reduces the value to be encoded by a base value and decomposes it. Improved Tunstall encoding is then performed on the decomposed characters. The improved Tunstall encoding adopts a tree structure and encodes based on the frequency of character occurrence, thereby maximizing the compression ratio. Since Tunstall encoding maps multiple characters to one code, multiple characters can be decoded simultaneously during the decoding process, achieving real-time and fast decoding.

[0088] like Figure 1 As shown: First, a decomposition threshold (K) is set for the decomposition operation. If the value of a symbol is less than the threshold, the symbol will be decomposed into two symbols: one is K, and the other is uK.

[0089] Suppose we have a set of symbols U = {0, 1, 2, 3}, and a specified code length of 3 (i.e., each symbol is represented by 3 bits). Initially, the symbols in the set have uniform probabilities, and the encoding process maps them to a fixed 3-bit binary code. Then, a decomposition operation breaks down low-frequency symbols (e.g., small symbols) into higher-frequency symbols, thus allowing smaller symbols to occupy more coding space.

[0090] After the decomposition operation, the output code length is reduced from 27 bits to 21 bits, further optimizing the coding efficiency. This operation can be reversed and the compression efficiency is improved by optimizing the distribution of symbols.

[0091] Step S5: Load the compressed model: The compressed model (including the results of bit allocation optimization, dead zone quantization and improved Tunstall encoding) is loaded into the target hardware platform (such as smartphones, embedded devices, TPUs, etc.) for inference.

[0092] Step S6: Real-time Decoding: During inference, the encoded weight data is first decoded using an improved Tunstall decoding method. The decoding process uses fixed-length encoding to ensure that the decoding process is efficient and real-time. The decoded weights are used for inference calculations in the CNN model.

[0093] Step S7: Inference Calculation: The decoded model performs neural network operations on the hardware platform and finally outputs the inference result.

[0094] Example 2: CNN Compression and Inference on Edge Device Hardware

[0095] This embodiment is particularly suitable for performing deep CNN model inference on edge device hardware (such as TPU, Eyeriss, etc.). The steps are as follows:

[0096] 1. Training and Compression Phase:

[0097] Training the CNN model: Similarly, the original CNN model is first trained using standard methods.

[0098] Compression processing: The trained CNN model is compressed using bit allocation optimization, dead-zone quantization, and improved Tunstall encoding. Bit allocation optimization helps the model maintain high accuracy while ensuring a high compression ratio. Dead-zone quantization further reduces redundant weights, and improved Tunstall encoding effectively compresses model weights and improves decoding efficiency.

[0099] 2. Inference Phase (Edge Device Adaptation):

[0100] Load the compressed model onto the edge device: Load the compressed model onto the edge device. At this point, the compressed weight data is stored using improved Tunstall encoding.

[0101] Hardware Decoding: During inference, the edge device decodes the compressed weight data in real time using its built-in decoding module. Improved Tunstall encoding provides a fixed-length decoding method, reducing decoding time and ensuring that the edge device can quickly recover the weight data.

[0102] Inference computation: The decoded weight data is passed to the inference unit of the edge device for efficient neural network inference computation. Due to the efficiency of the improved Tunstall encoding, the hardware can complete the inference operation in a relatively short time.

[0103] The background of this invention stems from the resource constraints faced by deep learning models (especially CNNs) in practical applications. In scenarios such as embedded devices, mobile devices, the Internet of Things, and autonomous driving, deep learning models are often difficult to deploy efficiently due to their high computational complexity and memory consumption. This invention aims to improve the compression ratio of CNN models by optimizing bit allocation and quantization strategies through a novel compression method, while significantly improving inference efficiency while maintaining inference accuracy.

[0104] 1. Improve the compression ratio of CNN models: By introducing innovative bit allocation and quantization techniques, the compression effect of CNN models is significantly improved, reducing model storage and computational requirements.

[0105] 2. Maintain high inference efficiency and accuracy: While significantly compressing the model, ensure that the inference speed is not affected and minimize the loss of accuracy to meet the real-time requirements of practical applications.

[0106] 3. Reduce system overhead: By optimizing the encoding method, system-level overhead is reduced, the computational efficiency during the model compression process is improved, and the compressed model can be executed efficiently.

[0107] 4. Improved hardware compatibility: This invention takes into account the characteristics of hardware accelerators, enabling the compressed CNN model to be better adapted to some hardware platforms and maximize inference speed.

[0108] This embodiment is primarily applied to deep learning tasks requiring efficient deep learning model compression and low computational overhead, and is particularly suitable for resource-constrained devices, embedded systems, and various application scenarios requiring real-time processing and inference. Under conditions of limited computing resources, storage capacity, and power consumption, this embodiment significantly improves inference efficiency while reducing storage and computing costs through model optimization and compression, and can be widely applied in computer vision, mobile devices, smart hardware, and edge computing.

[0109] Specific application scenarios include:

[0110] 1. Security Monitoring: Security cameras utilize efficient model compression technology to achieve real-time processing and classification of video images, enabling rapid identification of abnormal behavior and monitoring of personnel activities. This is suitable for real-time security systems both indoors and outdoors.

[0111] 2. Smartphones: In smartphone cameras, compressed deep learning models can process and enhance images faster in real time, such as scene classification, automatic beautification, night mode optimization, and real-time background blurring, improving the user experience.

[0112] 3. Edge Devices and Embedded Systems: In edge devices (such as drones, IoT terminals, robots, etc.), this technology can improve the performance of image or video inference tasks, such as object detection, object tracking and environmental perception, enhance the intelligence of the device, and reduce dependence on the cloud.

[0113] Its application mainly involves optimizing the bit allocation and quantization techniques of convolutional neural network (CNN) models to improve the compression ratio while maintaining high inference efficiency and accuracy. This enables efficient deployment on resource-constrained devices, reduces memory and computational overhead, and enhances real-time inference capabilities.

[0114] The beneficial effects of this invention are as follows:

[0115] 1. Improve the balance between compression ratio and accuracy.

[0116] By utilizing the Lagrange multiplier optimization method, this invention achieves dynamic adjustment of bit width, resulting in a better balance between compression ratio and accuracy. Compared to traditional fixed bit width methods, this invention can precisely allocate bits according to the actual needs of each channel, thereby significantly improving the compression ratio while maximizing the preservation of model inference accuracy. Without affecting model accuracy, our invention achieves approximately 20 times compression on common CNN models ResNet-18, ResNet-34, and ResNet-50, with original model sizes of 46.7MB, 83MB, and 102MB, respectively, compressed to 2.33MB, 4.88MB, and 5.1MB. On the compact model MobileNet-v2, it achieves approximately 10 times compression, with the original model size of 13.2MB compressed to 1.2MB.

[0117] 2. Reduce system overhead and improve inference efficiency

[0118] This invention innovatively combines dead-zone quantization with improved Tunstall encoding, resulting in a more compact CNN model in terms of storage and significantly reduced computational burden during decoding. The fixed-length entropy encoding characteristic of improved Tunstall encoding makes decoding more efficient, thereby accelerating inference. By employing this invention, model inference speed is significantly accelerated and better matched with the parallel computing capabilities of hardware accelerators, thus greatly improving the inference speed of the hardware platform.

[0119] Hardware simulation results show that, on the Deep Learning Accelerator TPU (a dedicated accelerator designed and developed by Google specifically for accelerating machine learning workloads under the TensorFlow framework), the inference speed using the ResNet50 model is increased from the original 395 times per second to 1681 times per second after compression by this invention, which is a 4.3-fold improvement in inference speed.

[0120] On Eyeris (Eyeriss is a low-power deep learning accelerator designed by MIT, specifically optimized for computation of convolutional neural networks to reduce computation and storage overhead, suitable for edge computing and mobile devices), using the ResNet50 model for inference, the speed was increased from 6.6 times per second to 18.9 times per second after compression by this invention, a 2.8-fold improvement in inference speed.

[0121] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A highly efficient entropy encoding method for decoding quantized model parameters, characterized in that: Includes the following steps: Step S1: Model Training: Train the Convolutional Neural Network (CNN) model using standard deep learning methods to obtain the original uncompressed model, including weight learning and optimization for each layer in the CNN model; Step S2: Bit allocation optimization: A bit allocation optimization method is used in the trained model; the bit width of each convolutional layer is dynamically adjusted according to the contribution of each channel to the output accuracy; this process is completed through rate-distortion optimization. Step S3: Dead Zone Quantization: The model weights are quantized using dead zone quantization technology. During quantization, small weights are compressed to zero to reduce storage requirements. The quantization step size and dead zone interval are adaptively adjusted by the optimization algorithm. Step S4: Improved Tunstall Encoding: The quantized weights are further encoded using improved Tunstall encoding, which uses fixed-length entropy encoding to compress the weight data; improved Tunstall encoding results in the encoded weight data occupying less storage space. Step S5: Load the compressed model: The compressed model is loaded into the target hardware platform for inference; the target hardware platform is a security camera. Step S6: Real-time Decoding: During inference, the encoded weight data is first decoded using an improved Tunstall decoding method. The decoding process uses fixed-length encoding to ensure that the decoding process is efficient and real-time. The decoded weights are used for inference calculations in the CNN model. Step S7: Inference Calculation: The decoded model performs neural network operations on the target hardware platform, and achieves real-time processing and classification of video images through efficient model compression technology, and finally outputs the inference result; the inference result is: monitoring personnel activities and quickly identifying abnormal behavior.

2. The method for decoding efficient entropy encoding of quantized model parameters according to claim 1, characterized in that: The bit allocation method optimized by rate distortion in step S2 dynamically adjusts the bit width according to the importance of each convolutional layer and channel, avoiding the shortcomings of uniform bit allocation in traditional compression methods, and achieving a higher compression ratio and lower computational overhead.

3. The method for decoding efficient entropy encoding of quantized model parameters according to claim 2, characterized in that: Step S2 further includes the following steps; Step S21: Objective of bit allocation optimization; Define a deep convolutional neural network F, and give W i and N i , where represent the weights and number of channels of the i-th convolutional layer, respectively; The goal is to optimize the bit allocation of each channel; by quantizing the weights of each layer, a modified output is obtained. Then Y and are measured by the expected error. The difference between them is called distortion; this distortion value is the error generated during the quantization process, and the formula is: Where E represents the expected distribution of the input image, Indicates Euclidean distance; Step S22: Bit allocation optimization; The goal of bit allocation optimization is to find the optimal bit allocation for each channel to minimize output distortion while satisfying the bit rate constraint of the compressed model. The optimization problem is expressed as: Where, r ij b represents the bit rate allocated to the j-th channel of the i-th layer. ij N is the quantization bit width of the channel. i L is the number of channels in the i-th layer, L is the total number of convolutional layers, and R is the constraint on the model size, i.e., the total bit rate. Step S23: Optimize using the additive property. The additivity property is used to simplify the output distortion calculation, thereby solving the bit allocation problem. The output distortion of each channel is considered as the sum of the quantization errors of a single channel, i.e.: By utilizing Taylor series expansion, assuming the neural network is differentiable and has a small quantization error, the optimal bit allocation can be solved using the Lagrange optimization method. Step S24: Optimal bit allocation. The optimal bit allocation problem is solved using the Lagrange formula; the Lagrange cost function expression is: Optimal bit allocation is achieved by setting the Lagrange cost function with respect to each r. ij The partial derivative is zero, thus obtaining the optimal solution; this solution represents the bit width that should be allocated to each channel, thereby achieving the minimum output distortion.

4. The method for decoding efficient entropy encoding of quantized model parameters according to claim 1, characterized in that: Step S3 further includes: introducing adaptive dead zone quantization technology to force small weights to zero, reducing storage requirements and further improving the model compression ratio; at the same time, by finely adjusting the dead zone size and quantization step size, the inference accuracy of the model is maintained. Step S31: Dead Interval: Within this interval, all quantized values ​​are forced to be set to zero; that is, any value less than a certain threshold is considered 0. This helps to eliminate small-amplitude noise and unimportant weights, and reduce storage requirements. Step S32: Quantization step size: It controls the interval of quantization values ​​and is a fixed constant; during the quantization process, the weights are discretized according to the quantization step size; Step S33: Formula: q(x) represents the quantized value; θ is the size of the dead zone. If |x| < θ, then q(x) = 0. Δ is the quantization step size; b is the quantization bit width, which determines the precision of the quantization value; sign(x) is used to preserve the sign, ensuring the correct handling of negative and positive values; Step S34: Dead Zone Quantization: When the absolute value of a weight is less than the set dead zone size θ, the weight is quantized to 0, that is, small values ​​are discarded; this helps to reduce unimportant weight data and further compress the amount of data. Step S35: Normal quantization: For weights larger than the dead zone, quantize by step size Δ and map them to the corresponding quantized values. If the weight x is within the dead zone, then q(x) = 0; otherwise, discretize by quantization step size Δ and ensure that the range of values ​​is within the effective bit width range.

5. The method for decoding efficient entropy encoding of quantized model parameters according to claim 1, characterized in that: Step S4 further includes the following steps; Improved Tunstall encoding; Using fixed-length entropy encoding allows the compressed weights to be stored and decoded more efficiently, reducing the computational burden during the decoding process and making it suitable for hardware accelerator platforms.

6. The method for decoding efficient entropy encoding of quantized model parameters according to claim 5, characterized in that: The improved Tunstall encoding uses a decomposition operation. Before applying Tunstall encoding, the input data is decomposed. This decomposition operation further improves the compression ratio. The idea behind the decomposition operation is to decompose larger symbols into multiple smaller symbols, thereby optimizing the data encoding method.

7. The method for decoding efficient entropy encoding of quantized model parameters according to claim 6, characterized in that: The improved Tunstall encoding is an improvement on the traditional Tunstall encoding. It reduces the value to be encoded by a base value and decomposes it, and then performs improved Tunstall encoding on the decomposed characters. The improved Tunstall encoding uses a tree structure and encodes characters based on their frequency of occurrence, thereby maximizing the compression ratio. Since Tunstall encoding maps multiple characters to one code, it can decode multiple characters simultaneously during the decoding process, achieving real-time and fast decoding.

Images