Model quantization-aware training system on edge devices

Abstract

The application discloses to the technical field of edge device, specifically for model quantization perception training system on edge device, including: data acquisition module, for collecting data from various edge devices; data preprocessing module, for cleaning, normalizing, enhancing preprocessing operation to the collected data, to improve the quality and diversity of data, to provide better data basis for subsequent model training; quantization perception training module, for being responsible for applying quantization perception training technology in the training process, to simulate quantization operation in the forward propagation of the model, quantizing and dequantizing the weight and activation value, adopting gradient approximation method to calculate gradient in the back propagation, to optimize model parameters. The application can convert the weight and activation value of the model into low-precision representation through quantization perception training, greatly reducing the calculation amount of the model when running on the edge device.

Description

Technical Field

[0001] This invention relates to the field of edge device technology, specifically to a model quantization perception training system on an edge device. Background Technology

[0002] In today's digital age, edge devices are being used more and more widely in various fields, such as smart homes, smart security, and the Industrial Internet of Things. These devices typically need to run deep learning models to achieve intelligent functions.

[0003] However, due to the limited hardware resources (such as computing power, memory, and storage) of edge devices, deploying traditional high-precision deep learning models on edge devices faces many challenges. Specifically, the following problems exist:

[0004] 1. Computational Resource Limitations: Edge devices have relatively weak computing power, making it difficult to handle large-scale, highly complex deep learning models. For example, in intelligent security monitoring, real-time target detection and recognition of video streams captured by cameras is required. If an unoptimized high-precision model, such as a ResNet-101-based target detection model, is used, its computation speed on ordinary edge computing devices (such as a Raspberry Pi 4B) is far from meeting real-time requirements, resulting in severe video analysis delays and an inability to detect anomalies in a timely manner.

[0005] 2. Limited Memory Resources: The weights and intermediate data of deep learning models typically consume a large amount of memory. For edge devices with limited memory, such as some low-power IoT sensor nodes, it may be impossible to load the complete model. For example, a simple speech recognition model may require tens of MB of memory for its unquantized weight data, while some IoT sensor nodes may only have a few MB of memory, making it impossible for the model to function properly on such devices.

[0006] 3. Model performance degradation: Directly quantizing the model without considering its impact on model structure and parameters often leads to a significant drop in model performance. For example, in image classification tasks, directly quantizing a well-trained full-precision ResNet-50 model to INT8 format may cause the accuracy on the test set to plummet from 75% to around 50%, severely affecting the model's performance in practical applications.

[0007] Based on the above, a model quantization perception training system for edge devices is invented. Summary of the Invention

[0008] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0009] A model quantization-aware training system on an edge device, comprising:

[0010] The data acquisition module is used to collect data from various edge devices;

[0011] The data preprocessing module is used to clean, normalize, and enhance the preprocessing operations of the collected data to improve the quality and diversity of the data and provide a better data foundation for subsequent model training.

[0012] The quantization-aware training module is responsible for applying quantization-aware training technology during the training process. It simulates quantization operations in the forward propagation of the model, quantizes and dequantizes weights and activation values, and uses gradient approximation methods to calculate gradients in the backpropagation to optimize model parameters.

[0013] The model evaluation module is used to evaluate the model during or after training. It calculates the model's accuracy, recall, and F1 score on the validation set to determine whether the model's performance meets the requirements. If the model's performance is poor, it will be fed back to the quantization-aware training module to adjust the training parameters or quantization strategy.

[0014] The model deployment module is used to deploy models that have been trained by quantization and whose performance meets the requirements to edge devices. During the deployment process, the model will be further optimized and adapted according to the hardware characteristics of the edge device to ensure that the model can run efficiently on the edge device.

[0015] The dynamic scheduling module receives performance feedback from the model evaluation module, dynamically adjusts the training parameters of the quantization perception training module based on the feedback results, and determines the timing and method of model deployment based on the real-time resource status of edge devices.

[0016] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the quantization perception training module includes:

[0017] The simulated quantization configuration module is used to first determine the quantization parameters and then insert the quantization operation;

[0018] The model initialization and training parameter setting module is used to first initialize the model and then set the training parameters.

[0019] The forward propagation and analog quantization execution module is used to first perform data input and forward calculation, and then perform analog quantization to demonstrate the effect;

[0020] The backpropagation and gradient approximation module is used to first calculate the loss function and then perform gradient approximation calculation.

[0021] The model parameter optimization and update module is used to first update the parameters, and then perform multiple iterations of training.

[0022] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the simulated quantization configuration module includes:

[0023] The quantization parameter determination module is used to determine the quantization bit width, quantization method, and quantization granularity based on the hardware capabilities of the edge device and the performance requirements of the model.

[0024] The Insert Quantization Operation module is used to insert quantization and dequantization operations during the forward propagation of the model in a deep learning framework.

[0025] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the model initialization and training parameter setting module includes:

[0026] The model initialization module is used to initialize the selected deep learning model, setting the model's parameters and structure;

[0027] The module for setting training parameters is used to first determine key parameters in the training process, including learning rate, batch size, and number of iterations. Then, a dynamic learning rate adjustment strategy is adopted, using a larger learning rate in the early stage of training to speed up convergence and gradually reducing the learning rate as training progresses to improve model performance.

[0028] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the forward propagation and simulated quantization execution module includes:

[0029] The data input and forward computation module is used to input the preprocessed data into the model and perform calculations according to the conventional forward propagation process. During the calculation process, the weights and activation values ​​will undergo quantization and dequantization operations.

[0030] The simulated quantization effect module is used to simulate the effect of quantization in actual deployment through quantization and dequantization operations, so that the model can perceive the changes brought about by quantization during the training process, providing a basis for subsequent parameter optimization.

[0031] As a preferred embodiment of the model quantization-aware training system on the edge device described in this invention, the backpropagation and gradient approximation processing module includes:

[0032] The loss function calculation module is used to calculate the loss function value based on the model's output and the true labels, measuring the difference between the model's current prediction and the actual situation;

[0033] The gradient approximation calculation module is used to calculate the gradient using gradient approximation techniques. During backpropagation, the gradient is directly transmitted through the quantization operation without considering the non-differentiability of the quantization operation itself, thereby enabling the updating of model parameters.

[0034] As a preferred embodiment of the model quantization-aware training system on the edge device described in this invention, the model parameter optimization and update module includes:

[0035] The parameter update module is used to update the model parameters using an optimization algorithm based on the calculated gradient. During the update process, the model will adjust the parameters according to the quantized gradient information to adapt the model to the quantized representation and reduce the impact of quantization on model performance.

[0036] The multi-iteration training module is used to repeat the above forward propagation, back propagation and parameter update process to perform multiple iterations of training. During the training process, the loss value and accuracy index are continuously monitored so that the training parameters can be adjusted in a timely manner according to the changes in the index until the model performance reaches the expected target or tends to stabilize.

[0037] As a preferred embodiment of the model quantization-aware training system on the edge device described in this invention, the dynamic scheduling module includes:

[0038] The real-time equipment status monitoring module is used to first collect resource data, then monitor performance indicators, and then sense environmental parameters.

[0039] The dynamic adjustment module during training is used to first receive performance feedback, then optimize training parameters, and finally fine-tune the quantization strategy.

[0040] The model deployment intelligent decision-making module is used to first determine the deployment timing, then select the deployment scheme, and finally distribute the deployment tasks.

[0041] The continuous monitoring and feedback module is used to first track the running status, then handle abnormal situations, and then perform feedback optimization in a loop.

[0042] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the real-time device status monitoring module includes:

[0043] The resource data acquisition module is used to collect key resource data of the device in real time through the monitoring tools or system plugins built into the edge device. The key resource data includes computing resources, memory usage, remaining storage space, and network bandwidth.

[0044] The performance metrics monitoring module is used to monitor the performance metrics of the model when it runs on edge devices, including inference time, response latency, and throughput. It can also use performance analysis tools to analyze the model inference process and obtain detailed performance data on the computation time and data transmission time of each layer to evaluate the current running efficiency of the model.

[0045] The environmental parameter sensing module is used to sense the external environmental parameters of the edge device.

[0046] The dynamic adjustment module for the training process includes:

[0047] The performance feedback receiving module is used to establish a data interaction channel with the model evaluation module and receive performance feedback data in real time during the model training process.

[0048] The training parameter optimization module is used to dynamically adjust the training parameters of the quantization perception training module based on performance feedback and device resource status.

[0049] The quantization strategy fine-tuning module is used to dynamically fine-tune the quantization strategy based on device resources and model training performance.

[0050] As a preferred embodiment of the model quantization perception training system on the edge device described in this invention, the model deployment intelligent decision-making module includes:

[0051] The deployment timing judgment module is used to determine the best time to deploy the model by comprehensively considering the status of equipment resources and the results of model performance evaluation.

[0052] The deployment scheme selection module is used to select the most suitable model deployment scheme based on the hardware characteristics and resource conditions of different edge devices;

[0053] The deployment task distribution module is used to reasonably allocate the selected model deployment tasks to the target edge devices, and accurately transmit the deployment resources of model files and configuration parameters to the corresponding devices through the device management system or distributed deployment tools, and start the deployment process to ensure that the model can be correctly installed and run on the edge devices.

[0054] The continuous monitoring and feedback module for operational status includes:

[0055] The runtime status tracking module is used to continuously monitor the runtime status of the model after it is deployed to the edge device, collect resource usage and performance index changes in real time during the model's operation, and track the accuracy of the model's inference results and resource usage fluctuations through log recording and real-time data monitoring.

[0056] The exception handling module is used to take appropriate measures in a timely manner when an exception is detected in the model operation;

[0057] The feedback optimization loop module is used to feed back the model's running status and processing results to the dynamic scheduling module as a reference for the next round of scheduling decisions. By continuously collecting and analyzing model running data, it continuously optimizes training parameter adjustment strategies, quantization strategies, and deployment schemes, forming a closed-loop dynamic optimization process to improve the overall system's operating efficiency and model performance.

[0058] Compared with existing technologies:

[0059] 1. Reduced computational resource consumption: Quantization-aware training converts the model's weights and activation values ​​into low-precision representations, significantly reducing the computational load when running the model on edge devices. For example, running a quantized deep learning model on an ARM-based edge computing device consumes significantly less computational resources compared to an unquantized full-precision model, enabling complex models that would otherwise be unable to run on such devices to operate efficiently.

[0060] 2. Reduced memory footprint: The memory space occupied by the quantized model weights and intermediate data is significantly reduced. Taking a typical image recognition model as an example, the memory footprint of the model before quantization may be around 100MB, while after quantization it can be reduced to less than 20MB. This allows more edge devices to easily load and run the model, improving the feasibility of deploying the model on resource-constrained devices.

[0061] 3. Maintaining Model Performance: Despite quantization, the model retains high performance after quantization due to the use of quantization-aware training techniques during training. In common tasks such as image classification and object detection, the decrease in accuracy of the quantized model compared to the full-precision model can be controlled, meeting the needs of most practical applications.

[0062] 4. Improved Inference Speed: Quantization reduces computational load and memory access frequency, significantly improving inference speed on edge devices. In applications with high real-time requirements, such as intelligent security monitoring and autonomous driving assistance, quantized models can process data faster and provide timely decision support. For example, in intelligent security monitoring, a quantized target detection model can complete the detection of a frame of image within milliseconds, meeting the needs of real-time monitoring. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall framework of the present invention;

[0064] Figure 2 This is a schematic diagram of the framework of the quantitative perception training module of the present invention;

[0065] Figure 3 This is a schematic diagram of the dynamic scheduling module framework of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0067] This invention provides a model quantization-aware training system for edge devices. Please refer to [link / reference]. Figures 1-3 ,include:

[0068] The data acquisition module is used to collect data from various edge devices;

[0069] The data preprocessing module is used to clean, normalize, and enhance the preprocessing operations of the collected data to improve the quality and diversity of the data and provide a better data foundation for subsequent model training.

[0070] The quantization-aware training module is responsible for applying quantization-aware training technology during the training process. It simulates quantization operations in the forward propagation of the model, quantizes and dequantizes weights and activation values, and uses gradient approximation methods to calculate gradients in the backpropagation to optimize model parameters.

[0071] The model evaluation module is used to evaluate the model during or after training. It calculates the model's accuracy, recall, and F1 score on the validation set to determine whether the model's performance meets the requirements. If the model's performance is poor, it will be fed back to the quantization-aware training module to adjust the training parameters or quantization strategy.

[0072] The model deployment module is used to deploy models that have been trained by quantization and whose performance meets the requirements to edge devices. During the deployment process, the model will be further optimized and adapted according to the hardware characteristics of the edge device to ensure that the model can run efficiently on the edge device.

[0073] The dynamic scheduling module receives performance feedback from the model evaluation module, dynamically adjusts the training parameters of the quantization perception training module based on the feedback results, and determines the timing and method of model deployment based on the real-time resource status of edge devices.

[0074] By setting up a dynamic scheduling module, the system can dynamically adjust training strategies and deployment schemes based on real-time performance metrics during model training, as well as the computing resources and memory usage of edge devices. For example, when the edge device is under high load, the dynamic scheduling module can postpone model deployment or adjust the model's quantization strategy to reduce the resource requirements of deployment. During training, if the model is found to be improving slowly on certain metrics, training parameters such as learning rate and batch size can be automatically adjusted to improve training efficiency and model performance, thereby enhancing the system's adaptability to different scenarios and device states.

[0075] The quantitative perception training module includes:

[0076] The simulated quantization configuration module is used to first determine the quantization parameters and then insert the quantization operation;

[0077] The simulation quantization configuration module includes:

[0078] The quantization parameter determination module is used to determine the quantization bit width, quantization method, and quantization granularity based on the hardware capabilities of the edge device and the performance requirements of the model.

[0079] The module for inserting quantization operations is used in deep learning frameworks to insert quantization and dequantization operations during the forward propagation of the model. Taking TensorFlow as an example, the QuantizeAwareTraining API is used to set the quantization of weights and activation values ​​of each layer of the model. In the specific implementation, when quantizing weight data, the maximum value of its absolute value is first calculated to determine the quantization scale factor, and the weight data is mapped to the corresponding quantization interval. For activation value data, an appropriate quantization range is selected for mapping based on its distribution characteristics.

[0080] The model initialization and training parameter setting module is used to first initialize the model and then set the training parameters.

[0081] The model initialization and training parameter setting module includes:

[0082] The model initialization module is used to initialize the selected deep learning model, setting the model's parameters and structure;

[0083] The module for setting training parameters is used to first determine key parameters in the training process, including learning rate, batch size, and number of iterations. Then, a dynamic learning rate adjustment strategy is adopted, using a larger learning rate in the early stage of training to speed up convergence and gradually reducing the learning rate as training progresses to improve model performance.

[0084] The forward propagation and analog quantization execution module is used to first perform data input and forward calculation, and then perform analog quantization to demonstrate the effect;

[0085] The forward propagation and analog quantization execution module includes:

[0086] The data input and forward computation module is used to input the preprocessed data into the model and perform calculations according to the conventional forward propagation process. During the calculation process, the weights and activation values ​​will undergo quantization and dequantization operations.

[0087] The simulated quantization effect module is used to simulate the effect of quantization in actual deployment through quantization and dequantization operations, so that the model can perceive the changes brought about by quantization during the training process, providing a basis for subsequent parameter optimization.

[0088] The backpropagation and gradient approximation module is used to first calculate the loss function and then perform gradient approximation calculation.

[0089] The backpropagation and gradient approximation processing module includes:

[0090] The loss function calculation module is used to calculate the loss function value based on the model's output and the true labels, measuring the difference between the model's current prediction and the actual situation;

[0091] The gradient approximation calculation module is used to calculate the gradient using gradient approximation techniques, so that the gradient can be directly passed through the quantization operation during backpropagation, without considering the non-differentiability of the quantization operation itself, thereby realizing the update of model parameters.

[0092] The model parameter optimization and update module is used to first update the parameters and then perform multiple iterations of training.

[0093] The model parameter optimization and update module includes:

[0094] The parameter update module is used to update the model parameters using an optimization algorithm based on the calculated gradient. During the update process, the model will adjust the parameters according to the quantized gradient information to adapt the model to the quantized representation and reduce the impact of quantization on model performance.

[0095] The multi-iteration training module is used to repeat the above forward propagation, back propagation and parameter update process to perform multiple iterations of training. During the training process, the loss value and accuracy index are continuously monitored so that the training parameters can be adjusted in a timely manner according to the changes in the index until the model performance reaches the expected target or tends to stabilize.

[0096] The dynamic scheduling module includes:

[0097] The real-time equipment status monitoring module is used to first collect resource data, then monitor performance indicators, and then sense environmental parameters.

[0098] The real-time equipment status monitoring module includes:

[0099] The resource data acquisition module is used to collect key resource data of the device in real time through the monitoring tools or system plugins built into the edge device. The key resource data includes computing resources, memory usage, remaining storage space, and network bandwidth.

[0100] The performance metrics monitoring module is used to monitor the performance metrics of the model when it runs on edge devices, including inference time, response latency, and throughput. It can also use performance analysis tools to analyze the model inference process and obtain detailed performance data on the computation time and data transmission time of each layer to evaluate the current running efficiency of the model.

[0101] The environmental parameter sensing module is used to sense the external environmental parameters of the edge device.

[0102] The dynamic adjustment module during training is used to first receive performance feedback, then optimize training parameters, and finally fine-tune the quantization strategy.

[0103] The dynamic adjustment module for the training process includes:

[0104] The performance feedback receiving module is used to establish a data interaction channel with the model evaluation module and receive performance feedback data in real time during the model training process.

[0105] The training parameter optimization module is used to dynamically adjust the training parameters of the quantization perception training module based on performance feedback and device resource status. If the current CPU load of the edge device is too high, the batch size of training can be reduced to reduce the amount of data per training session and reduce computational pressure. If the model converges slowly during training, the learning rate can be increased or the learning rate decay strategy can be adjusted to speed up the training process.

[0106] The quantization strategy fine-tuning module is used to dynamically fine-tune the quantization strategy based on device resources and model training performance. When device memory resources are tight, the quantization bit width is further reduced or the quantization granularity is adjusted to reduce model memory usage. If the model performance degrades significantly after quantization, the quantization accuracy of some key layers can be restored, thus finding a new balance between resource consumption and model performance.

[0107] The model deployment intelligent decision-making module is used to first determine the deployment timing, then select the deployment scheme, and finally distribute the deployment tasks.

[0108] The model deployment intelligent decision-making module includes:

[0109] The deployment timing judgment module is used to determine the best time to deploy the model by comprehensively considering the device resource status and model performance evaluation results. If the edge device is currently under low load and the model's performance indicators on the test set meet the expected requirements, the model deployment process is triggered. If the device resources are insufficient or the model performance does not meet the standards, the deployment is postponed and the device and model status is continuously monitored.

[0110] The deployment scheme selection module is used to select the most suitable model deployment scheme based on the hardware characteristics and resource conditions of different edge devices. For devices with strong computing power but limited memory, the scheme combining model compression and quantization is preferred to reduce memory consumption. For devices with weak computing resources, lightweight models are selected and quantization strategies are optimized to ensure that the model can run efficiently on the device.

[0111] The deployment task distribution module is used to reasonably allocate the selected model deployment tasks to the target edge devices, and accurately transmit the deployment resources of model files and configuration parameters to the corresponding devices through the device management system or distributed deployment tools, and start the deployment process to ensure that the model can be correctly installed and run on the edge devices.

[0112] The continuous monitoring and feedback module is used to first track the running status, then handle abnormal situations, and then perform feedback optimization loop.

[0113] The continuous monitoring and feedback module for operational status includes:

[0114] The runtime status tracking module is used to continuously monitor the runtime status of the model after it is deployed to the edge device, collect resource usage and performance index changes in real time during the model's operation, and track the accuracy of the model's inference results and resource usage fluctuations through log recording and real-time data monitoring.

[0115] The exception handling module is used to take appropriate measures in a timely manner when an exception is detected in the model operation; for example, automatically restarting the model service, rolling back to the previous version of the model, or adjusting the model's running parameters according to the exception to ensure the stable operation of the model.

[0116] The feedback optimization loop module is used to feed back the model's running status and processing results to the dynamic scheduling module as a reference for the next round of scheduling decisions. By continuously collecting and analyzing model running data, it continuously optimizes training parameter adjustment strategies, quantization strategies, and deployment schemes, forming a closed-loop dynamic optimization process to improve the overall system's operating efficiency and model performance.

[0117] In practical use, the specific operating steps are as follows:

[0118] S1: Collect data from various edge devices via the data acquisition module;

[0119] S2: The data preprocessing module cleans, normalizes, and enhances the preprocessing of the collected data to improve the quality and diversity of the data, providing a better data foundation for subsequent model training.

[0120] S3: The quantization parameter determination module defines the quantization bit width, quantization method, and quantization granularity based on the hardware capabilities of the edge device and the performance requirements of the model. After this, the quantization operation insertion module inserts quantization and dequantization operations into the deep learning framework during the model's forward propagation process. Following insertion, the model initialization module initializes the selected deep learning model, setting its parameters and structure. After initialization, the training parameter setting module first determines key parameters for the training process, including learning rate, batch size, and number of iterations. Then, a dynamic learning rate adjustment strategy is adopted, using a larger learning rate in the early stages of training to accelerate convergence and gradually decreasing it as training progresses to improve model performance. Next, the data input and forward computation module inputs the preprocessed data into the model and performs calculations according to the standard forward propagation process. During the calculation, weights and activation values ​​undergo quantization and dequantization operations. After calculation, the quantization effect simulation module uses quantization and dequantization operations to simulate the effect of quantization in actual deployment, making the model... The model perceives the changes brought about by quantization during training, providing a basis for subsequent parameter optimization. After simulation, the loss function calculation module calculates the loss function value based on the model's output and the true label, measuring the difference between the model's current prediction and the actual situation. After calculation, the gradient approximation calculation module uses gradient approximation techniques to calculate the gradient, allowing the gradient to be directly transmitted through the quantization operation during backpropagation, without considering the non-differentiability of the quantization operation itself, thereby updating the model parameters. After calculation, the parameter update module updates the model parameters using optimization algorithms based on the calculated gradient. During the update process, the model adjusts the parameters according to the quantized gradient information, adapting the model to the quantized representation and reducing the impact of quantization on model performance. After the update, the multiple iteration training module repeats the above forward propagation, backpropagation, and parameter update process for multiple iterations of training. During training, the loss value and accuracy metrics are continuously monitored to adjust the training parameters in a timely manner according to changes in the metrics, until the model performance reaches the expected target or tends to stabilize.

[0121] S4: The model evaluation module evaluates the model during or after training by calculating the model's accuracy, recall, and F1 score on the validation set to determine whether the model's performance meets the requirements. If the model's performance is poor, it will be fed back to the quantization-aware training module to adjust the training parameters or quantization strategy.

[0122] S5: The model deployment module deploys the quantized perception-trained model that meets the performance requirements to the edge device. During the deployment process, the model is further optimized and adapted according to the hardware characteristics of the edge device to ensure that the model can run efficiently on the edge device.

[0123] S6: The resource data acquisition module utilizes the monitoring tools or system plugins built into the edge device to collect key resource data of the device in real time. This key resource data includes computing resources, memory usage, remaining storage space, and network bandwidth. After collection, the performance indicator monitoring module monitors the model's performance indicators on the edge device, including inference time, response latency, and throughput. Performance analysis tools can be used to analyze the model's inference process, obtaining detailed performance data on computation time and data transmission time for each layer to assess the model's current operating efficiency. After analysis, the environmental parameter perception module senses the external environmental parameters of the edge device. After sensing, a data interaction channel is established between the performance feedback receiving module and the model evaluation module to receive performance feedback data during model training in real time. After receiving the data, the training parameter optimization module dynamically adjusts the training parameters of the quantization perception training module based on performance feedback and device resource status. After adjustment, the quantization strategy fine-tuning module dynamically fine-tunes the quantization strategy based on device resource status and model training performance. After fine-tuning, the deployment timing judgment module determines the optimal time for model deployment by comprehensively considering device resource status and model performance evaluation results. After judgment, the deployment... The deployment scheme selection module chooses the most suitable model deployment scheme based on the hardware characteristics and resource availability of different edge devices. After selection, the deployment task distribution module allocates the selected model deployment tasks to the target edge devices and accurately transmits the deployment resources, including model files and configuration parameters, to the corresponding devices through the device management system or distributed deployment tools, initiating the deployment process to ensure the model can be correctly installed and run on the edge devices. After distribution, the running status tracking module continuously monitors the model's running status after deployment to the edge devices, collecting real-time data on resource usage and performance changes during model operation. It tracks the accuracy of inference results and resource usage fluctuations through log recording and real-time data monitoring. After tracking, the anomaly handling module takes timely action when anomalies are detected in the model's operation. After handling, the feedback optimization loop module feeds back the model's running status and processing results to the dynamic scheduling module as a reference for the next round of scheduling decisions. By continuously collecting and analyzing model running data, the module continuously optimizes training parameter adjustment strategies, quantization strategies, and deployment schemes, forming a closed-loop dynamic optimization process that improves the overall system's operating efficiency and model performance.

[0124] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A model quantization-aware training system on an edge device, the system comprising: Comprise: a data acquisition module for collecting data from various edge devices; a data preprocessing module for cleaning, normalizing, and enhancing the collected data to improve its quality and diversity, providing a better data foundation for subsequent model training; a quantization-aware training module for applying quantization-aware training techniques during the training process to simulate quantization operations in the forward propagation of the model, quantizing and dequantizing weights and activation values, and using gradient approximation methods to calculate gradients in the backward propagation to optimize model parameters; a model evaluation module for evaluating the model during or after training to determine whether the model's performance meets the requirements by calculating the accuracy, recall, and F1 value of the model on the validation set, and if the model's performance is not satisfactory, feeding back to the quantization-aware training module to adjust the training parameters or quantization strategy; a model deployment module for deploying the model that has undergone quantization-aware training and meets the performance requirements to edge devices, and during deployment, further optimizing and adapting the model according to the hardware characteristics of the edge devices to ensure efficient operation of the model on the edge devices; a dynamic scheduling module for receiving performance feedback from the model evaluation module, dynamically adjusting the training parameters of the quantization-aware training module based on the feedback, and determining the timing and method of model deployment based on the real-time resource status of the edge devices; The dynamic scheduling module comprises: a device state real-time monitoring module for collecting resource data, then monitoring performance indicators, and then sensing environmental parameters; a training process dynamic adjustment module for receiving performance feedback, then optimizing training parameters, and then fine-tuning quantization strategies; a model deployment intelligent decision-making module for determining the deployment timing, then selecting the deployment scheme, and then distributing the deployment tasks; a running state continuous monitoring and feedback module for tracking the running state, then handling abnormal situations, and then performing feedback optimization cycles; The device state real-time monitoring module comprises: a resource data acquisition module for collecting key resource data of the edge device in real time through built-in monitoring tools or system plugins, including computing resources, memory usage, storage remaining space, and network bandwidth; a performance indicator monitoring module for monitoring the performance indicators of the model running on the edge device, including inference time, response delay, and throughput, and using performance analysis tools to analyze the model inference process to obtain detailed performance data of each layer's computation time and data transmission time to evaluate the current running efficiency of the model; an environmental parameter sensing module for sensing the external environmental parameters of the edge device; The training process dynamic adjustment module comprises: a performance feedback receiving module for establishing a data interaction channel with the model evaluation module to receive performance feedback data in real time during the model training process; a training parameter optimization module for dynamically adjusting the training parameters of the quantization-aware training module based on performance feedback and device resource status; The quantization strategy fine-tuning module is configured to fine-tune the quantization strategy dynamically according to the device resource condition and the model training performance.

2. The model quantization perception training system on edge device of claim 1, wherein, The quantization-aware training module comprises: The analog quantization configuration module is configured to determine the quantization parameter first and then insert the quantization operation; The model initialization and training parameter setting module is configured to initialize the model first and then set the training parameter; The forward propagation and analog quantization execution module is configured to input data and perform forward calculation first and then embody the analog quantization effect; The backward propagation and gradient approximation processing module is configured to calculate the loss function first and then perform gradient approximation calculation; The model parameter optimization and updating module is configured to update the parameter first and then perform multiple iteration training.

3. The model quantization perception training system on edge device of claim 2, wherein, The analog quantization configuration module comprises: The quantization parameter determination module is configured to determine the quantization bit width, quantization method and quantization granularity according to the hardware capability of the edge device and the performance requirement of the model; The quantization operation insertion module is configured to insert the quantization and dequantization operation in the forward propagation process of the model in the deep learning framework.

4. The model quantization awareness training system on edge device of claim 2, wherein, The model initialization and training parameter setting module comprises: The model initialization module is configured to initialize the selected deep learning model, set the parameter and structure of the model; The training parameter setting module is configured to determine the key parameters in the training process first, including the learning rate, batch size and iteration number, and then adopt the dynamic learning rate adjustment strategy to use a larger learning rate in the early stage of training to accelerate the convergence speed and gradually reduce the learning rate as the training progresses to improve the model performance.

5. The model quantization awareness training system on edge device of claim 2, wherein, The forward propagation and analog quantization execution module comprises: The data input and forward calculation module is configured to input the preprocessed data into the model and perform calculation according to the conventional forward propagation process, and in the calculation process, the weight and activation value will undergo quantization and dequantization operation; The analog quantization effect embodiment module is configured to simulate the effect of quantization in actual deployment through quantization and dequantization operation, so that the model perceives the changes brought by quantization during the training process, providing a basis for subsequent parameter optimization.

6. The model quantization awareness training system on edge device of claim 2, wherein, The backward propagation and gradient approximation processing module comprises: The loss function calculation module is configured to calculate the loss function value according to the output of the model and the real label, to measure the difference between the current prediction result of the model and the real situation; The gradient approximation calculation module is configured to calculate the gradient using the gradient approximation technology, so that the gradient directly passes through the quantization operation during the backward propagation, without considering the non-differentiability of the quantization operation itself, thereby realizing the update of the model parameters.

7. The model quantization awareness training system on edge device of claim 2, wherein, The model parameter optimization and updating module comprises: The parameter updating module is configured to update the model parameters using the optimization algorithm according to the calculated gradient, and in the updating process, the model adjusts the parameters according to the quantized gradient information, so that the model adapts to the quantized representation and reduces the impact of quantization on the model performance; The multiple iteration training module is configured to repeat the above forward propagation, backward propagation and parameter updating process to perform multiple iteration training, and in the training process, the loss value and accuracy index are continuously monitored to adjust the training parameter in time according to the index change until the model performance reaches the expected target or tends to be stable.

8. The model quantization awareness training system on edge device of claim 1, wherein, The model deployment intelligent decision module comprises: A deployment timing judgment module for comprehensively judging the best timing of model deployment based on device resource status and model performance evaluation results; A deployment scheme selection module for selecting the most suitable model deployment scheme according to the hardware characteristics and resource conditions of different edge devices; A deployment task distribution module for reasonably distributing the selected model deployment task to the target edge device, accurately transmitting the model file, configuration parameter and deployment resource to the corresponding device through the device management system or the distributed deployment tool, and starting the deployment process to ensure that the model can be correctly installed and run on the edge device; The running state continuous monitoring and feedback module comprises: A running state tracking module for continuously monitoring the running state of the model after the model is deployed to the edge device, collecting the resource usage and performance index changes in the model running process in real time, and tracking the inference result accuracy and resource occupation fluctuation information of the model through log recording and real-time data monitoring; An abnormal situation processing module for taking corresponding processing measures in time when it is detected that the model running appears abnormal; A feedback optimization cycle module for feeding back the model running state and processing results to the dynamic scheduling module as the reference basis for the next round of scheduling decision, so as to continuously optimize the training parameter adjustment strategy, quantization strategy and deployment scheme through the continuous collection and analysis of model running data, form a closed-loop dynamic optimization process, and improve the running efficiency and model performance of the whole system.

Images