Model low-power-consumption reasoning method and related equipment
By performing structured pruning, knowledge distillation, and parameter quantization on the initial model, a lightweight model is generated. Combined with cloud-edge collaborative decision-making based on hardware characteristics and confidence scores, the problem of efficient inference and low-power operation on edge devices is solved, achieving efficient model adaptation and energy consumption optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 广州商研网络科技有限公司
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies struggle to achieve efficient inference and low-power operation of deep learning models on resource-constrained edge devices, and existing model compression technologies do not fully consider hardware architecture characteristics, resulting in low operating efficiency and energy waste.
By performing structured channel pruning, knowledge distillation compensation, and parameter fixed-point quantization on the initial model, a lightweight model is generated. An inference engine file is generated based on the performance of the hardware computing unit, the hardware operating frequency is dynamically adjusted, and a cloud-edge collaborative decision-making mechanism based on confidence is introduced.
Significantly reduces the number of model parameters and computational complexity, optimizes computation graph scheduling, dynamically matches computing power supply, reduces power consumption, and achieves a balance between inference accuracy, latency, and power consumption.
Smart Images
Figure CN121835902A_ABST
Abstract
Claims
1. A low-power inference method for a model, applied to computer devices, characterized in that, The method includes: The preset initial model is optimized to obtain a lightweight model corresponding to the initial model; Based on the lightweight model and the performance parameters of the hardware computing unit of the computer device, an inference engine file is generated. When the inference engine file is running on the computer device, the hardware operating frequency is dynamically adjusted according to the task load; Perform local inference and determine the confidence score of the local inference result. Based on the comparison result of the confidence score and the preset threshold, select to output the local inference result or trigger the collaborative inference between the computer device and the cloud server.
2. The low-power inference method for models according to claim 1, characterized in that, The optimization process for the preset initial model includes: The initial model is subjected to joint optimization processing of structured channel pruning, knowledge distillation compensation, and parameter fixed-point quantization.
3. The low-power inference method for models according to claim 2, characterized in that, The structured channel pruning includes: The convolutional layer topology of the initial model is determined, and the channel importance score is calculated based on the L1 norm of the channel weight tensor of each convolutional layer. A binary mask vector is generated based on a preset pruning rate. The corresponding convolution kernel is removed based on the binary mask vector, and the input channel dimension of the next level is updated.
4. The low-power inference method for models according to claim 2, characterized in that, The knowledge distillation compensation includes: The initial model is used as the teacher network, and the structured channel pruned model is used as the student network to construct a parallel training architecture. The logical vectors of the teacher network and the student network are softened based on temperature parameters to obtain soft labels and soft prediction probabilities. The weight parameters of the student network are updated based on the distillation loss function between the soft label and the soft prediction probability, and the cross-entropy loss function between the prediction result of the student network and the true label.
5. The low-power inference method for models according to claim 2, characterized in that, The fixed-point quantization of the parameters includes: Based on the dynamic range of the weight parameters and activation values in the lightweight model, the quantization scaling factor is determined. Based on the quantization scaling factor, the 32-bit floating-point parameter is mapped to an 8-bit integer parameter; The quantized parameters are then encapsulated into a model file.
6. The low-power inference method for models according to claim 1, characterized in that, The generation of the inference engine file based on the performance parameters of the lightweight model and the hardware computing unit of the computer device includes: Based on the computation graph of the lightweight model, determine the operator type attribute in the computation graph; Based on the matching relationship between the operator type attribute and the performance parameter, with the goal of minimizing inference latency, an allocation relationship is established between the operator in the computation graph and the corresponding hardware computing unit. The inference engine file is generated based on the allocation relationship and the model parameters of the lightweight model.
7. The low-power inference method for models according to claim 1, characterized in that, The dynamic adjustment of hardware operating frequency based on task load includes: Establish a task waiting queue, and obtain the absolute deadline and pre-calculated computational workload for each task in the task waiting queue; Based on the computational workload and remaining relaxation time, calculate the minimum processing frequency required to satisfy all task timing constraints. A target operating frequency is selected from the set of discrete frequencies supported by the hardware of the computer device, wherein the target operating frequency is the minimum value among one or more frequencies that are greater than or equal to the minimum processing frequency; The power supply voltage and clock frequency of the computer device are adjusted based on the target operating frequency.
8. The low-power inference method for models according to claim 7, characterized in that, The method further includes: If the task waiting queue is empty, the dynamic clock signal of the computing unit of the computer device is cut off based on the clock gating mechanism.
9. The low-power inference method for models according to claim 1, characterized in that, The triggering of collaborative reasoning between the computer device and the cloud server includes: The input data for the current inference task is losslessly compressed, and a transmission data packet is generated based on the compressed data. Establish a communication connection with the cloud server and send the transmission data packet to the cloud server, and control the computer device to enter a discontinuous reception mode; Upon receiving the response data packet returned by the cloud server, the response data packet is parsed to obtain the cloud inference result as the target inference result.
10. A computer device, characterized in that, include: Memory, and A processor that executes computer-readable instructions stored in the memory to implement the model low-power inference method as described in any one of claims 1 to 9.