Lightweight model adaptive compression system based on dynamic sparse activation

Abstract

The invention provides a lightweight model adaptive compression system based on dynamic sparse activation, and relates to the technical field of edge computing, the system comprises an edge side equipment operation monitoring module, the edge side equipment operation monitoring module is configured to obtain operation performance data of edge side equipment and hardware resource data of the edge side equipment; the compression execution module is configured to perform adaptive sparsity parameter adjustment on the to-be-compressed model according to the operation performance data and the hardware resource data, and perform model compression processing to obtain a compression model; the storage optimization module is configured to perform storage structure optimization processing on the compression model to obtain a target compression model; and the model evaluation feedback module is configured to judge whether the target compression model meets the performance requirements of the edge side equipment or not. Through the system, the problem of application limitation of an existing model compression technology is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge computing, and particularly relates to a lightweight model adaptive compression system based on dynamic sparse activation. BACKGROUND

[0002] At present, the deployment of deep learning models on edge devices mainly faces the problems of large model storage requirements and high computational overhead. Common model compression methods include pruning, quantization and knowledge distillation. Pruning technology reduces the model size by removing unimportant connections or neurons in the model, usually based on the size of the weights or their impact on the model output to determine the pruning strategy. Quantization technology converts high-precision weight representation to low-precision representation to reduce storage requirements and computational overhead. Knowledge distillation method uses a smaller model (student model) to learn the knowledge of a larger model (teacher model), thereby realizing the lightweight of the model.

[0003] However, these methods have certain limitations in practical application. Traditional pruning methods often need to manually set fixed pruning ratios or thresholds, which are difficult to dynamically adjust the compression degree according to the actual resource situation of the device. Quantization methods may cause significant decrease in model accuracy, especially at low bit quantization. Knowledge distillation requires a pre-trained teacher model, and the distillation process may be complex and have high computational cost. SUMMARY

[0004] The present application provides a lightweight model adaptive compression system based on dynamic sparse activation to solve the problem of application limitation of existing model compression technology.

[0005] The system comprises:

[0006] An edge device running monitoring module configured to obtain running performance data of an edge device and hardware resource data of the edge device;

[0007] A compression execution module configured to perform adaptive sparse parameter adjustment on a model to be compressed according to the running performance data and the hardware resource data, and perform model compression processing to obtain a compressed model;

[0008] A storage optimization module configured to perform storage structure optimization processing on the compressed model to obtain a target compressed model;

[0009] A model evaluation feedback module configured to determine whether the target compressed model meets the performance requirements of the edge device and generate evaluation information; the evaluation information is used to instruct the compression execution module to perform model compression iteration until the target compressed model meets the performance requirements of the edge device.

[0010] Preferably, the system further comprises:

[0011] a model pre-training module, configured to:

[0012] construct an initial model;

[0013] pre-train the initial model using a standard dataset to obtain the model to be compressed; the standard dataset is based on the model requirements of the edge-side device.

[0014] Preferably, the compression execution module is further configured to:

[0015] set an initial sparsity parameter of the DSACA algorithm to obtain a first DSACA algorithm;

[0016] perform sparse activation processing on the model to be compressed using the initial DSACA algorithm to obtain a first compressed model;

[0017] obtain local training data of the edge-side device, and update the neuron connection weights in the initial activation model using the local training data to obtain a second compressed model;

[0018] adaptively adjust the sparsity threshold of the first DSACA algorithm according to the running performance data and the hardware resource data to obtain a second DSACA algorithm;

[0019] perform quantization compression processing on the second compressed model using the second DSACA algorithm to obtain the compressed model. Preferably, the compression execution module comprises a DSACA algorithm unit, which is configured to:

[0020] set an initial sparsity threshold and a sparsity growth rate of the DSACA algorithm, and determine a quantization bit number to obtain the first DSACA algorithm; the sparsity threshold is used to limit the interval range of weight adjustment in the model, the sparsity growth rate is used to adjust the rate of the model compression process, and the quantization bit number is used to adjust the degree of weight compression in the model.

[0021] Preferably, the compression execution module further comprises a sparse activation unit, which is configured to:

[0022] traverse the neuron connection weights of each layer of the model through the DSACA algorithm, and calculate the absolute value of each neuron connection weight;

[0023] set all weights with an absolute value less than the sparsity threshold to zero to obtain the first compressed model.

[0024] Preferably, the compression execution module further comprises a weight updating module, which is configured to:

[0025] perform model training on the first compression model by using the local training data, so as to fine-tune the neuron connection weights in the first compression model and obtain the second compression model.

[0026] Preferably, the compression execution module further comprises a threshold adjusting unit, which is configured to:

[0027] calculate a new sparsity threshold according to the running performance data, the hardware resource data and the adaptive adjustment strategy, replace the original sparsity threshold in the first DSACA algorithm with the new sparsity threshold, and obtain the second DSACA algorithm.

[0028] Preferably, the compression execution module further comprises a quantization compression unit, which is configured to:

[0029] map the neuron connection weights to bit representations to obtain weight bits;

[0030] determine a quantization interval and a quantization parameter of the weight bits by using a uniform quantization method;

[0031] perform quantization compression processing on the quantization parameter by using the second DSACA algorithm and the quantization interval, and obtain the compression model.

[0032] Preferably, the storage optimization module is further configured to:

[0033] statistically analyze the frequency distribution of the neuron connection weights of the compression model, and construct a Huffman tree; the Huffman tree is used to represent the neuron connection weights after compression;

[0034] store the weight matrix in the Huffman tree in a sparse matrix storage format, and obtain the target compression model.

[0035] Preferably, the sparse matrix storage format comprises a compressed sparse row format and a compressed sparse column format.

[0036] From the above, the application provides a lightweight model adaptive compression system based on dynamic sparse activation, which comprises an edge device running monitoring module configured to acquire running performance data of an edge device and hardware resource data of the edge device; a compression execution module configured to perform adaptive sparse parameter adjustment on a to-be-compressed model according to the running performance data and the hardware resource data, and perform model compression processing to obtain a compressed model; a storage optimization module configured to perform storage structure optimization processing on the compressed model to obtain a target compressed model; and a model evaluation feedback module configured to judge whether the target compressed model meets the performance requirements of the edge device and generate evaluation information; the evaluation information is used to instruct the compression execution module to perform model compression iteration until the target compressed model meets the performance requirements of the edge device. The application solves the problem of application limitation of existing model compression technology through the above system. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0038] Figure 1 A schematic diagram of a lightweight model adaptive compression system based on dynamic sparse activation of the application;

[0039] Figure 2 A schematic diagram of a compression execution module in a lightweight model adaptive compression system based on dynamic sparse activation of the application;

[0040] Figure 3 An example flowchart of neuron connection weight updating;

[0041] Figure 4 An example flowchart of sparse threshold adjustment;

[0042] Figure 5 A principle flowchart of a lightweight model adaptive compression system based on dynamic sparse activation of the application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the application will be described clearly and completely in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0044] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described next, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0045] It should be noted that in the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, illustration or description. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0046] With the rapid development of artificial intelligence (AI) technology, especially deep learning (DL), the demand for its application on various intelligent edge devices (such as mobile terminals, Internet of Things nodes, embedded systems) is increasingly urgent. Edge computing aims to process data close to the data source, which can effectively reduce latency, save bandwidth and enhance privacy protection. However, deploying deep learning models, which are usually large in size and computationally intensive, on resource-constrained edge devices poses a serious challenge.

[0047] To achieve edge deployment of models, researchers have proposed a series of model compression techniques to balance between model storage requirements, computational overhead and model accuracy. The current mainstream compression methods mainly include:

[0048] Pruning: The core idea of this method is to remove connections (weights) or neurons (channels / layers) in the model that are judged to be redundant or unimportant, thereby reducing the size of the model and sparsifying the computation graph. Typical criteria may be the significance of the weight value itself or the evaluation of its contribution to the final output of the model.

[0049] Quantization: This technique aims to reduce the data representation precision of model weight parameters and activation values, for example, converting high-precision floating-point numbers to low-precision integer or fixed-point number representations with limited bit width. By reducing the bit width of data representation, the storage overhead of the model and the memory access and arithmetic operation burden in the calculation process can be significantly reduced.

[0050] Knowledge Distillation (KD): This method trains a smaller and lighter "student" model to imitate the behavior and output characteristics (not just the final prediction results) of a larger and usually more powerful "teacher" model. The goal is to transfer the "knowledge" (including internal representations of input data, decision boundary information, etc.) contained in the "teacher" model to the "student" model, so that the small model can achieve close-to-large-model performance while having lower deployment costs.

[0051] Although these techniques have achieved certain success in reducing model complexity and deployment barriers, they still have significant limitations and shortcomings in actual edge scenario applications:

[0052] Static and inadequate adaptability of compression strategies: Most existing methods use static compression strategies, i.e., pre-set compression ratios or intensities (such as fixed pruning rates, fixed quantization bit widths) before deployment. However, the resource conditions (such as available memory, computing power, energy budget, current task load) faced by edge devices in actual operating environments are often dynamically changing. Static strategies cannot perceive and adapt to such changes, resulting in the inability to perform deeper compression to ensure basic functions (such as real-time response) when resources are highly strained, and the inability to fully utilize idle resources to improve model performance (such as running larger models or higher precision versions) when resources are relatively abundant. This rigidity limits the overall efficiency and robustness of models on edge devices.

[0053] Balancing the compression rate and accuracy: In the pursuit of high compression rate (such as significant pruning or extremely low bit quantization), model accuracy often faces significant and unpredictable nonlinear decline risks. Especially when performing deep pruning, connections that are crucial for recognizing complex features or specific samples may be removed; when performing ultra-low precision quantization, the sharp decline in representation ability can easily cause feature information loss or decision boundary distortion. Existing technologies often struggle to stably and reliably maintain the original prediction accuracy and generalization ability of models under such deep compression conditions.

[0054] Lack of universality and generalization ability: The edge computing ecosystem is highly heterogeneous, encompassing a wide variety of hardware platforms (different architectures of CPU, GPU, NPU, etc.), performance indicators, and specific resource limitation conditions. Current mainstream compression methods are often optimized for specific types of model structures or target hardware, and their effectiveness and processes may fluctuate significantly between different model architectures or different edge devices. The lack of a universally applicable and highly automated model compression solution that can adapt to a wide range of deep learning model architectures and diverse edge hardware environments is one of the key bottlenecks hindering the large-scale deployment of edge AI.

[0055] Based on the above problems, the present application provides the following embodiments.

[0056] Figure 1 A schematic diagram of a lightweight model adaptive compression system based on dynamic sparse activation according to the present application.

[0057] Figure 5 A principle flowchart of a lightweight model adaptive compression system based on dynamic sparse activation according to the present application.

[0058] Referring to Figure 1 and Figure 5 It can be seen that the present embodiment provides a lightweight model adaptive compression system based on dynamic sparse activation, which comprises:

[0059] An edge-side device running monitoring module 100 configured to acquire running performance data of an edge-side device and hardware resource data of the edge-side device.

[0060] Specifically, in the present embodiment, the edge-side device running monitoring module 100 is deployed on an edge device to collect hardware resource data of the edge-side device and running performance data of the model running in real time.

[0061] The system further comprises:

[0062] A compression execution module 200 configured to perform adaptive sparse parameter adjustment on a model to be compressed according to the running performance data and the hardware resource data, and perform model compression processing to obtain a compressed model.

[0063] Specifically, in the present embodiment, the compression execution module 200 comprises the specific implementation logic of the DSACA algorithm, dynamically adjusts the sparsity threshold of the model according to the information fed back by the edge-side device running monitoring module 100, and performs compression operation.

[0064] The system further comprises:

[0065] A storage optimization module 300 configured to perform storage structure optimization processing on the compressed model to obtain a target compressed model.

[0066] Specifically, in the present embodiment, the storage optimization module 300 is used to perform storage structure optimization on the compressed model, adopts efficient data structure and storage strategy, and further reduces the storage overhead of the model.

[0067] The system further comprises:

[0068] The model evaluation feedback module 400 is configured to determine whether the target compressed model meets the performance requirements of the edge-side device and generate evaluation information; the evaluation information is used to instruct the model compression execution module 200 to perform model compression iteration until the target compressed model meets the performance requirements of the edge-side device.

[0069] Specifically, in the present embodiment, the model evaluation feedback module 400 is used to evaluate the compressed model, compare the performance indicators of the model before and after compression, and feed back the evaluation results to the model compression execution module 200, which is used to guide the subsequent compression operation.

[0070] Further, in some embodiments, the system further comprises:

[0071] The model pre-training module 500 is configured to construct an initial model; and pre-train the initial model using a standard data set to obtain the model to be compressed; the standard data set is determined based on the model requirements of the edge-side device.

[0072] Specifically, in the present embodiment, the model pre-training module 500 is responsible for pre-training the original deep learning model, and uses a standard data set to train the model to have basic business processing capability.

[0073] Specifically, in the present embodiment, the model pre-training module 500 is responsible for pre-training the original deep learning model, and uses a standard data set to train the model to have basic business processing capability.

[0074] During the pre-training process of the model pre-training module 500, a deep learning model is pre-trained using a standard data set. Taking a convolutional neural network (CNN) as an example, a network structure is constructed, including an input layer, multiple convolutional layers, a pooling layer, a fully connected layer, and an output layer. A loss function is set, and a stochastic gradient descent (SGD) or Adam optimization algorithm is used for training.

[0075] During the model pre-training phase, a training environment needs to be configured for a specific deep learning framework (TensorFlow). For image classification tasks, a data set usually consists of a large number of labeled images, such as the ImageNet data set, which contains more than 14 million images and is divided into 1000 categories. The goal of pre-training is to enable the model to learn general feature representations of images, so that it can quickly adapt to specific tasks in subsequent transfer learning or model compression.

[0076] At the beginning of the training process, the weights of the model are randomly initialized. Then, the predicted output of the model for the input image is calculated through forward propagation, and the loss value is calculated by comparing it with the true label. Then, the gradient is calculated using the backpropagation algorithm, and the weights are updated according to the selected optimization algorithm. This process continues for multiple epochs until the performance indicators (such as accuracy) of the model on the validation set tend to be stable.

[0076] Figure 2 This is a schematic diagram of the compression execution module in a lightweight adaptive compression system based on dynamic sparse activation, as described in this application.

[0077] See Figure 2 Furthermore, in some embodiments, the compression execution module 200 is further configured to:

[0078] By setting the initial sparsity parameters of the DSACA algorithm, the first DSACA algorithm is obtained.

[0079] Specifically, in this embodiment, the initial sparsity parameters of the DSACA algorithm are set by the DSACA algorithm unit 210. The DSACA algorithm unit 210 sets the initial sparsity threshold and sparsity growth rate of the DSACA algorithm and determines the number of quantization bits to obtain the first DSACA algorithm. The sparsity threshold is used to limit the range of weight adjustment in the model, the sparsity growth rate is used to adjust the rate of model compression, and the number of quantization bits is used to adjust the degree of weight compression in the model.

[0080] For example, the DSACA algorithm unit 210 sets the initial sparsity threshold (θ0) and sparsity growth rate (ρ) of the DSACA algorithm and determines the number of quantization bits (b).

[0081] Initializing the sparsity parameters is the starting point of the DSACA algorithm. The initial sparsity threshold (θ0) determines how many weights will initially be set to zero. The sparsity growth rate (ρ) controls how quickly the sparsity threshold increases as the compression process progresses. The number of quantization bits (b) determines the degree to which the weights are compressed during quantization; a lower number of bits results in less storage requirement but may lead to a greater loss of precision.

[0082] The selection of these parameters needs to consider the model size, task complexity, and resource constraints of the target device. For devices with extremely limited resources, such as some low-power IoT sensors, a higher initial sparsity threshold and a lower number of quantization bits may be required. For devices with relatively abundant resources, such as smartphones, the initial sparsity threshold can be appropriately reduced to balance compression ratio and model accuracy.

[0083] For example, in a heart rate prediction model running on a smartwatch, taking into account the smartwatch's storage and computing capabilities as well as the accuracy requirements of the heart rate prediction task, the initial sparsity threshold is set to θ0 = 0.01, the sparsity growth rate is set to ρ = 0.05, and the number of quantization bits is set to b = 8.

[0084] The compression execution module 200 is further configured to:

[0085] The initial DSACA algorithm is used to perform sparse activation processing on the to-be-compressed model to obtain a first compressed model.

[0086] Specifically, in the present embodiment, the sparse activation processing is completed by a sparse activation unit 220, which traverses the neuron connection weights of each layer of the model by the DSACA algorithm and calculates the absolute value of each neuron connection weight; weights with an absolute value less than the sparsity threshold are set to zero to obtain the first compressed model.

[0087] For example, for a pre-trained model, the DSACA algorithm starts to perform sparse activation processing on each layer of the model. The neuron connection weights of each layer of the model are traversed, the absolute value |w| of the weight is calculated, and compared with the current sparsity threshold θ. The weight with an absolute value less than the sparsity threshold is set to 0 to realize the sparsification of the weight.

[0088] In this process, the weight matrix of the model is traversed layer by layer. For each weight value, its absolute value is calculated and compared with the current sparsity threshold. If the absolute value of the weight is less than the threshold, it means that the weight has a small contribution to the model and can be safely set to zero, thereby reducing the number of non-zero elements in the model and realizing the sparsification of the model.

[0089] The sparsification processing can reduce the storage requirement of the model, because storing a sparse matrix only needs to save the non-zero elements and their position information, rather than the entire matrix. At the same time, the sparse matrix can also use special optimization algorithms in calculation to skip the operation of zero elements, thereby reducing the calculation overhead.

[0090] Algorithm formula:

[0091]

[0092] w: represents the weight parameter in the model, which can be the filter weight of the convolutional layer or the neuron connection weight of the fully connected layer.

[0093] θ: represents the current sparsity threshold, which is used to judge whether the weight is important enough to be retained.

[0094] w': represents the weight after sparse activation processing. If the absolute value of the weight is less than the threshold, it is set to 0, otherwise the original value is retained.

[0095] Suppose in a fully connected layer, the original weight matrix size is 1024x1024, and part of the weight values are shown in Table 1:

[0096] Table 1

[0097] weight value 0.005 -0.012 0.018 -0.009 0.021 absolute value 0.005 0.012 0.018 0.009 0.021

[0098] The current sparsity threshold is θ = 0.01. According to the formula, the weight values less than 0.01 in absolute value are set to 0, and the processed weight values are shown in Table 2 below:

[0099] Table 2

[0100] weight value 0 -0.012 0.018 0 0.02

[0101] The compression execution module 200 is further configured to:

[0102] Obtain local training data of the edge side device, and update the neuron connection weights in the initial activation model using the local training data to obtain a second compression model.

[0103] Specifically, in the present embodiment, the updating of the neuron connection weights is completed by the weight updating module 230, which performs model training on the first compression model using the local training data to fine-tune the neuron connection weights in the first compression model and obtain the second compression model.

[0104] Figure 3 An example flowchart for updating neuron connection weights.

[0105] Referring to Figure 3 It can be seen that, for example, after sparse activation, the model is retrained on a small scale using local data collected by the edge device to fine-tune the weights of the model. A local training data set is constructed, and the size of the data can be adjusted according to the storage capacity and computing power of the device. Gradient descent and other optimization algorithms are used to calculate the gradient of the loss function with respect to the weights and update the values of the non-zero weights according to the learning rate (η).

[0106] The weight updating and optimization steps aim to adapt the model to the new sparse structure through retraining and restore as much accuracy as possible that is lost due to sparsification. The local training data set should be able to represent the data distribution that the model may encounter in actual application. Only non-zero weights are updated during retraining, because zero weights represent unimportant connections in the sparse structure, and updating them can destroy the sparse pattern and increase storage and computing overhead.

[0107] Update formula:

[0108]

[0109] W1: represents the current weight value.

[0110] η: represents the learning rate, which controls the step size of each update.

[0111] represents the gradient of the loss function with respect to the weights, which reflects how the weights can be adjusted to reduce the loss.

[0112] Assuming that the heart rate prediction model is retrained on the smartwatch, using the heart rate data collected by the watch in the last week (a total of 500 samples) as local training data. Through experimental verification, setting the learning rate η = 0.0001 can achieve a good balance between training stability and convergence speed. Training is performed for 10 iteration cycles. Assuming that the weight gradient of a certain layer is The updated weight value is:

[0113] w1 = w1 - 0.0001 x 0.002.

[0114] The compression execution module 200 is also configured to:

[0115] According to the running performance data and the hardware resource data, the sparsity threshold of the first DSACA algorithm is adaptively adjusted to obtain a second DSACA algorithm.

[0116] Specifically, in this embodiment, the adaptive adjustment of the sparsity threshold is completed by the threshold adjustment unit 240, which calculates a new sparsity threshold according to the running performance data, the hardware resource data, and an adaptive adjustment strategy, and replaces the original sparsity threshold in the first DSACA algorithm with the new sparsity threshold to obtain the second DSACA algorithm.

[0117] Figure 4 An example flowchart for sparsity threshold adjustment.

[0118] Referring to Figure 4 It can be seen that, for example, the edge device running monitoring module sends the running state information of the device to the compression execution module in real time. The compression execution module calculates a new sparsity threshold using a pre-set adaptive adjustment strategy according to the received information.

[0119] Specifically, when it is monitored that the device resources are tight (such as the CPU usage is higher than the set threshold) or the model performance is degraded (such as the inference delay exceeds the allowed range), the algorithm will increase the sparsity threshold to further compress the model. Conversely, when the device resources are abundant and the model performance is good, the algorithm will reduce the sparsity threshold to allow the model to retain more weights to improve the prediction accuracy.

[0120] Algorithm formula:

[0121] θ new = θ + p x (a x (CPU_usage - CPU_threshold) + b x (Memory_usage - Memory_threshold) + g x (Latency - Latency_threshold));

[0122] θ: represents the current sparsity threshold.

[0123] ρ: represents the sparsity growth rate, controls the magnitude of each adjustment.

[0124] α, β, γ: respectively represent the weight coefficients of CPU usage, memory usage, and inference delay, for comprehensive consideration of the impact of different resources.

[0125] CPU_usage, Memory_usage, Latency: respectively represent the current CPU usage, memory usage, and inference delay.

[0126] CPU_threhshold, Memory_threshold, Latency_threshold: respectively represent the set CPU usage, memory usage, and inference delay thresholds.

[0127] For example:

[0128] The CPU usage of the smart watch is monitored to be 80%, the memory usage is 70%, and the inference delay is 100ms. According to the use scenario and performance requirements of the device,

[0129] The weight coefficients are set as α = 0.4, β = 0.3, and γ = 0.3; the CPU usage threshold is 70%, the memory usage threshold is 60%, and the inference delay threshold is 80ms.

[0130] Substitute the formula to calculate the new sparsity threshold:

[0131] θ new = 0.01 + 0.05 × (0.4 × (80% - 70%) + 0.3 × (70% - 60%) + 0.3 × (100ms - 80ms));

[0132] Assuming that 10% and 20ms correspond to 0.1 and 0.2 respectively after numerical conversion, then:

[0133] θ new = 0.01 + 0.05 × (0.4 × 0.1 + 0.3 × 0.1 + 0.3 × 0.2)

[0134] = 0.01 + 0.05 × (0.04 + 0.03 + 0.06) = 0.01 + 0.0065 = 0.0165.

[0135] The compression execution module 200 is further configured to:

[0136] quantize and compress the second compression model using the second DSACA algorithm to obtain the compression model.

[0137] Specifically, in this embodiment, the quantization compression unit 250 completes the quantization compression process, which maps the neuron connection weights to a bit representation, obtaining weight bits; a uniform quantization method is used to determine the quantization interval and quantization parameter of the weight bits; the second DSACA algorithm and the quantization interval are used to quantize and compress the quantization parameter, obtaining the compressed model.

[0138] For example, based on sparse activation and weight update, the model weight is quantized. The weight value is mapped to a lower bit representation (such as quantizing a 32-bit floating point to an 8-bit integer). A uniform quantization method is used to determine the quantization interval and quantization parameter.

[0139] Quantization compression reduces the storage requirements and computational overhead of the model by reducing the number of bits required to represent each weight. Uniform quantization uniformly maps weight values within a predetermined range. The number of quantization bits (b) determines the accuracy of quantization, and a lower number of bits means less storage space and faster computation speed, but may also result in a certain loss of accuracy.

[0140] The quantization process includes two main steps: determining the quantization parameter (such as quantization step Δ) and converting the floating point weight to an integer representation. The quantization step determines the interval size between the quantized weight values, which is calculated based on the maximum and minimum values of the weight and the number of quantization bits.

[0141] Quantization formula:

[0142] w q = round(w2 / Δ)

[0143]

[0144] W2: represents the current weight value.

[0145] Δ: represents the quantization step, which determines the interval between the quantized weight values.

[0146] w max , w min : represent the maximum and minimum values of the weight, respectively.

[0147] b: represents the number of quantization bits, which determines the accuracy of the quantized weight representation.

[0148] wq: represents the quantized weight value.

[0149] Assuming the quantized weight range is [-5, 5] and the number of quantization bits is 8, the quantization step is:

[0150]

[0151] After quantizing the weight value 3.25:

[0152] w q = round(3.25 / 0.0392) ≈ round(82.908) = 83.

[0153] Further, in some embodiments, the storage optimization module 300 is further configured to:

[0154] statistically analyze the frequency distribution of the neuron connection weights of the compression model, and construct a Huffman tree; the Huffman tree is used to represent the compressed neuron connection weights; and the weight matrix in the Huffman tree is stored in a sparse matrix storage format, to obtain the target compression model.

[0155] Specifically, in this embodiment, the weight data is further compressed using the Huffman coding algorithm. The frequency distribution of the weight values is statistically analyzed, and a Huffman tree is constructed. The sparse weight matrix is stored in a sparse matrix storage format (such as CRS, CCS, etc.).

[0156] The storage structure optimization aims to further reduce the storage requirement of the model by using a more efficient data representation method. Huffman coding is a lossless compression algorithm based on the frequency of weight values (prior art), which assigns a unique binary code to each weight value. The weight values with high frequency are assigned shorter codes, and vice versa. In this way, the total number of bits required for storage can be reduced without losing information.

[0157] Sparse matrix storage formats (such as compressed sparse row format CRS or compressed sparse column format CCS) are specifically optimized for sparse matrices, and only store non-zero elements and their position information, thereby greatly reducing the storage requirement.

[0158] The advantages of this embodiment include:

[0159] Through the dynamic sparse activation and quantization compression technology, combined with the adaptive threshold adjustment strategy, efficient compression and storage of deep learning models on edge devices are realized. The storage requirement and computational overhead of the model are significantly reduced, making the model more easily deployed on resource-constrained edge devices such as smart watches and Internet of Things sensors. At the same time, by dynamically adjusting the sparsity threshold, the model can dynamically balance the compression rate and performance according to the real-time resource state of the device, ensuring efficient operation when resources are tight and maintaining high precision when resources are abundant. In addition, the scheme incorporates a model evaluation and feedback mechanism, which effectively avoids the loss of precision caused by excessive compression through multiple rounds of iterative optimization, maintains the high precision of the model, and meets the actual application requirements. The wide applicability provides a general solution for model deployment and optimization in the fields of Internet of Things, smart wearables, smart homes, and smart medical care, reducing the cost of model development and deployment.

[0160] The foregoing description has been directed to specific embodiments. It is recognized that variations and modifications of the embodiments will occur to those skilled in the art. It is the objects of the disclosure to incorporate substitutions, permutations, additions and / or modifications into the practice of the application. Although specific terms have been employed herein, such terms are intended in a descriptive sense and not for purposes of limitation.

Claims

1. A lightweight model self-adaptive compression system based on dynamic sparse activation, characterized in that, The system comprises: An edge-side device running monitoring module (100) configured to acquire running performance data of an edge-side device and hardware resource data of the edge-side device; A compression execution module (200) configured to perform adaptive sparsity parameter adjustment on a to-be-compressed model according to the running performance data and the hardware resource data, and perform model compression processing to obtain a compressed model; A storage optimization module (300) configured to perform storage structure optimization processing on the compressed model to obtain a target compressed model; A model evaluation feedback module (400) configured to judge whether the target compressed model meets the performance requirements of the edge-side device and generate evaluation information; the evaluation information is used to instruct the compression execution module (200) to perform model compression iteration until the target compressed model meets the performance requirements of the edge-side device.

2. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 1, characterized in that, The system further comprises: A model pre-training module (500) configured to: Construct an initial model; Pre-train the initial model using a standard data set to obtain the to-be-compressed model; the standard data set is determined based on the model requirements of the edge-side device.

3. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 1, characterized in that, The compression execution module (200) is further configured to: Set an initial sparsity parameter of a DSACA algorithm to obtain a first DSACA algorithm; Perform sparse activation processing on the to-be-compressed model using the initial DSACA algorithm to obtain a first compressed model; Acquire local training data of the edge-side device, and update neuron connection weights in the initial activation model using the local training data to obtain a second compressed model; Adaptively adjust the sparsity threshold of the first DSACA algorithm according to the running performance data and the hardware resource data to obtain a second DSACA algorithm; Perform quantization compression processing on the second compressed model using the second DSACA algorithm to obtain the compressed model.

4. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 3, characterized in that, The compression execution module (200) comprises a DSACA algorithm unit (210) configured to: Set an initial sparsity threshold and a sparsity growth rate of a DSACA algorithm, and determine a quantization bit number to obtain the first DSACA algorithm; the sparsity threshold is used to limit the interval range of weight adjustment in the model, the sparsity growth rate is used to adjust the rate of the model compression process, and the quantization bit number is used to adjust the degree of weight compression in the model.

5. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 3, characterized in that, The compression execution module (200) further comprises a sparse activation unit (220) configured to: Iterate through neuron connection weights of each layer of the model through the DSACA algorithm, and calculate the absolute values of each neuron connection weight; Set all weights with absolute values less than the sparsity threshold to zero to obtain the first compressed model.

6. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 3, characterized in that, The compression execution module (200) further comprises a weight updating module (230), which is configured to: perform model training on the first compression model by using the local training data, so as to fine-tune the neuron connection weights in the first compression model and obtain the second compression model.

7. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 3, characterized in that, The compression execution module (200) further comprises a threshold adjustment unit (240), which is configured to: calculate a new sparsity threshold according to the running performance data, the hardware resource data and an adaptive adjustment strategy, replace the original sparsity threshold in the first DSACA algorithm with the new sparsity threshold, and obtain the second DSACA algorithm.

8. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 3, characterized in that, The compression execution module (200) further comprises a quantization compression unit (250), which is configured to: map the neuron connection weights to bit representations to obtain weight bits; determine a quantization interval and a quantization parameter of the weight bits by using a uniform quantization method; perform quantization compression processing on the quantization parameter by using the second DSACA algorithm and the quantization interval, and obtain the compression model.

9. The lightweight model adaptive compression system based on dynamic sparse activation of claim 1, wherein, The storage optimization module (300) is further configured to: statistically analyze the frequency distribution of the neuron connection weights of the compression model, and construct a Huffman tree; the Huffman tree is used to represent the neuron connection weights after compression; store the weight matrix in the Huffman tree in a sparse matrix storage format to obtain the target compression model.

10. The lightweight model self-adaptive compression system based on dynamic sparse activation according to claim 9, characterized in that, The sparse matrix storage format comprises a compressed sparse row format and a compressed sparse column format.

Images