A model pruning method for heterogeneous cloud-edge-device collaborative systems

Abstract

This invention discloses a model pruning method for heterogeneous cloud-edge-device collaborative systems, belonging to the field of model pruning technology in machine learning. It solves the problem of low resource utilization in traditional deep learning model-based pruning methods. The invention includes the following steps: S1. Initializing the model to be called, parsing the structure of the model to be called to form a structure set, and distributing it to computing nodes to generate an overhead predictor; S2. Calling the overhead predictor according to the pruning shape control strategy, using the pruning evaluation function to obtain the pruning evaluation function; S3. From the pruned model set at different split points, selecting the optimal deployment model based on the time overhead evaluation function and network conditions. This invention effectively improves the running efficiency of the pruned model on different hardware, and can control inference efficiency and resource utilization. It meets different task requirements through dynamic weight adjustment and can be applied to pruning models.

Description

Technical Field

[0001] This invention relates to a model pruning method, and more particularly to a model pruning method for heterogeneous cloud-edge-device collaborative systems, belonging to the field of model pruning technology in machine learning. Background Technology

[0002] Cloud-edge-device collaboration, also known as cloud-edge-device integration, is a technical architecture that combines cloud computing, edge computing, and terminal devices to optimize computing resource allocation, reduce latency, and improve overall system performance. This collaborative model is particularly suitable for scenarios requiring real-time response and large-scale data processing, such as the Internet of Things (IoT), smart manufacturing, and autonomous driving. For example, in the field of autonomous driving, vehicles need to process data from sensors in real time to ensure safe driving. In a cloud-edge-device collaborative architecture, data and computing tasks need to be dynamically allocated across different layers, which places high demands on network connection quality and bandwidth. Therefore, ensuring that the system can still operate normally in narrowband conditions, especially in critical application scenarios, is a problem that urgently needs to be solved.

[0003] In narrowband communication scenarios, model pruning, by simplifying deep learning models, can effectively improve the operating efficiency of edge devices and reduce the amount of data transmitted between the cloud and the edge. Firstly, pruning significantly reduces the number of model parameters, lowering the computational load and memory requirements of edge devices, enabling them to quickly process inference tasks and improve real-time response capabilities even in resource-constrained environments. Secondly, the pruned model is smaller, reducing the amount of data that needs to be transmitted during cloud-edge collaboration, thereby reducing communication bandwidth usage and latency. For example, in multi-mobile platform target tracking applications, edge devices can monitor and analyze the position and trajectory of multiple targets in real time. Through model pruning, the computational requirements of the target tracking model are greatly reduced, allowing edge devices to quickly process target information in video streams without frequently sending complete data to the cloud for analysis. Thus, pruning optimization not only improves the response speed of target tracking but also reduces the consumption of narrowband communication bandwidth, ensuring stable and efficient system operation even under bandwidth constraints. This highlights the urgency and importance of model pruning in solving practical problems.

[0004] Meanwhile, edge systems are generally heterogeneous. Different edge devices will have different performance when running the same model, and different edge devices will have different performance requirements such as real-time performance, accuracy, or size. For example, edge devices that perform target tracking have high real-time requirements, while those that perform face recognition focus more on accuracy. Deployed on resource-constrained devices such as STM32, they have higher size requirements. Current general pruning algorithms cannot fully consider the differences in these heterogeneous systems.

[0005] In the existing technology, pruning methods based on deep learning models are generally divided into the following categories: (1) Structured pruning techniques. In structured pruning, the pruning operation is concentrated at the level of the entire convolutional kernel, channel, or layer. Commonly used methods include pruning based on the importance of convolutional kernels. These methods selectively prune convolutional kernels or channels with smaller contributions by evaluating the contribution of convolutional kernels in the model. Depending on the evaluation source, they can be divided into parameter-driven and data-driven methods. Parameter-driven methods start from the weights of the model and evaluate the importance of convolutional kernels based on the parameters in L1 regularization, L2 regularization, batch normalization, and the spectral clustering of filters, and then perform pruning. Data-driven methods, on the other hand, are based on the intermediate layer characteristics generated after inputting data. Evaluation is carried out using the feature map. Specifically, there are methods such as the number of activation values ​​in the activation output value, the rank decomposition of the feature layer, subspace clustering of the feature map, principal component analysis, and network entropy before and after pruning; (2) Layer sensitivity analysis pruning technology. Layer sensitivity analysis is a method to evaluate the importance of each layer of the neural network during the model pruning process. By pruning layer by layer and observing the impact on model performance, it is possible to identify those layers that have a small impact on the overall performance of the model and can be pruned. The analysis uses gradient information, Hessian matrix, Taylor expansion or other heuristic methods to evaluate the sensitivity of each layer. The aim is to find pruning strategies that reduce computation and model size, while maintaining model performance as much as possible. Through layer sensitivity analysis , can optimize the efficiency and deployment capability of the model, especially on resource-constrained devices with practical application value; (3) Adaptive pruning technology, adaptive pruning technology is a method to dynamically adjust the degree of model pruning. In adaptive pruning, the model parameters are usually multiplied by a trainable mask or gating mechanism. The mask will be dynamically updated according to the gradient of the loss function or other indicators, and the pruning process is dynamic. As training progresses, the pruning decision will be adjusted according to the current state of the model, and will not depend on the pre-set pruning rules. The pruning strategy can be flexibly adjusted according to different datasets and task requirements; However, the above pruning methods have the following problems: (1) Most pruning methods only consider the model from the algorithm level. Pruning ignores the differences in computing resources between different nodes in a heterogeneous system, which makes it difficult for the pruning model in a heterogeneous system to adapt to all nodes and to maximize resource utilization; (2) The existing pruning method evaluation index is mainly FLOPs, which is used to measure the improvement of model inference speed, ignoring that inference time and memory size can more directly reflect the actual operation of the model in practical applications; (3) The current cloud-edge-device collaborative system will frequently transmit intermediate feature maps for collaborative computation during the collaborative process. Narrowband takes longer and is more difficult to bear than normal network transmission. Most existing pruning algorithms are designed to balance accuracy and model size without considering the control of model shape.

[0006] In summary, a model pruning method is needed that considers the computational resources of different nodes during pruning and is oriented towards heterogeneous cloud-edge-device collaborative systems. Summary of the Invention

[0007] A brief overview of the invention is given below to provide a basic understanding of certain aspects of it. It should be understood that this overview is not an exhaustive summary of the invention. It is not intended to identify key or essential parts of the invention, nor is it intended to limit the scope of the invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description that follows.

[0008] In view of this, in order to solve the problem of low resource utilization of traditional deep learning model-based pruning methods in the prior art, this invention provides a model pruning method for heterogeneous cloud-edge-device collaborative systems.

[0009] The technical solution is as follows: A model pruning method for heterogeneous cloud-edge-device collaborative systems, comprising the following steps:

[0010] S1. Initialize the model to be called, parse the structure of the model to be called to form a structure set, distribute it to the computing nodes, and generate the cost predictor;

[0011] S2. Based on the pruning shape control strategy, call the cost predictor and use the pruning evaluation function to obtain the pruning evaluation function.

[0012] S3. From the pruned model set at different split points, the optimal deployment model is selected based on the time cost evaluation function and network conditions.

[0013] Furthermore, in S1, a set A1 of models to be called is obtained, the structure of the models to be called is parsed to form a structure set A2, and distributed to computing nodes. The running speed of different structures on different computing nodes is measured, a record file is generated, the record file is transmitted back to the cloud, and a unified training is conducted to generate an overhead predictor for each computing node for subsequent model pruning.

[0014] Further, in S2, the model to be pruned is obtained, the segmentation points of the intermediate feature map are found, and the pruning model segmentation point set A3 is obtained. For each possible segmentation point position in the pruning model segmentation point set A3, its size is limited, that is, the channel of the segmentation point position is pruned first, and then pruning is performed.

[0015] The model to be pruned is divided into a set of structures that can be pruned. A pruning evaluation function is used to score each structure of the model to be pruned. The number of prunings N is controlled by the pruning rate. The structure with the lowest score is removed in reverse order through N loops to obtain the pruned model.

[0016] The pruning evaluation function L_total is expressed as:

[0017]

[0018] Where α, β, γ, and δ are the weight coefficients corresponding to the original task loss L_task, inference time prediction loss L_time, memory usage prediction loss L_mem, and device resource utilization loss L_util;

[0019] The original task loss L_task is represented as:

[0020]

[0021] Where y_i is the actual label, y_hat_i is the model output, and L is the loss function;

[0022] The inference time prediction loss is expressed as:

[0023] L_time = (T(θ) / T_max) ^ 2

[0024] Where T(θ) is the inference time prediction model, and T_max is the upper limit of inference time;

[0025] The memory usage prediction loss L_mem is expressed as:

[0026] L_mem = (M(θ) / M_max) ^ 2

[0027] Where M(θ) is the memory usage prediction model, and M_max is the set memory limit;

[0028] The equipment resource utilization loss L_util is represented as:

[0029] L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2

[0030] Where U(θ) is the equipment resource utilization prediction model, and U_ideal is the target utilization rate.

[0031] Furthermore, in S3, under the narrowband background, based on the communication bandwidth dt, the intermediate feature size of the pruned model transmission feature_out, and the communication overhead feature_out / dt, the structure before the segmentation point of the pruned model is called forward_layers, and the time overhead evaluation function Time is obtained through the time prediction function time_predict.

[0032] The time cost evaluation function Time is represented as:

[0033] Time = feature_out / dt + time_predict(forward_layers)

[0034] The pruned model set that meets the accuracy and resource constraints is selected, its time cost function value is calculated, and the model with the smallest time cost function value is selected as the optimal deployment model under the current network conditions.

[0035] The beneficial effects of this invention are as follows: This invention proposes a pruning method that reduces overhead by controlling the shape of the pruned model and the position of the intermediate feature maps during transmission. Considering the resource consumption of the model during the pruning stage allows for better adaptation to actual deployment scenarios. Regarding improving the adaptability of heterogeneous systems, traditional pruning algorithms often ignore the resource differences of different hardware devices. This invention, by introducing an overhead predictor and a device resource utilization loss function, can better adapt to heterogeneous systems, ensuring that the pruned model runs efficiently on different hardware. In terms of comprehensively optimizing model performance, by adding weights to inference time, memory consumption, and resource utilization to the loss function, model pruning no longer simply pursues accuracy and model size but can also control inference efficiency and resource utilization, achieving more comprehensive performance optimization. Regarding automated model segmentation and shape control, this invention analyzes the model structure, automatically finds segmentation points, and controls the model shape of different pruning schemes. It can automatically generate multiple pruned models and select the model with the best performance, avoiding the complexity of manual debugging. Regarding dynamic weight adjustment, the set weight coefficients allow users to flexibly adjust the model pruning strategy according to specific task requirements and hardware conditions, finding a balance point under different resource constraints. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0037] Figure 1 A flowchart illustrating a model pruning method for heterogeneous cloud-edge-device collaborative systems;

[0038] Figure 2 This is a schematic flowchart of an embodiment of a model pruning method for heterogeneous cloud-edge-device collaborative systems. Detailed Implementation

[0039] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0040] refer to Figure 1 and Figure 2 This embodiment describes a model pruning method for heterogeneous cloud-edge-device collaborative systems, specifically including the following steps:

[0041] S1. Initialize the model to be called, parse the structure of the model to be called to form a structure set, distribute it to the computing nodes, and generate the cost predictor;

[0042] S2. Based on the pruning shape control strategy, call the cost predictor and use the pruning evaluation function to obtain the pruning evaluation function.

[0043] S3. From the pruned model set at different split points, the optimal deployment model is selected based on the time cost evaluation function and network conditions.

[0044] Furthermore, in S1, a set A1 of models to be called is obtained, the structure of the models to be called is parsed to form a structure set A2, and distributed to computing nodes. The running speed of different structures on different computing nodes is measured, a record file is generated, the record file is transmitted back to the cloud, and a unified training is conducted to generate an overhead predictor for each computing node for subsequent model pruning.

[0045] Further, in S2, the model to be pruned is obtained, and the segmentation points of the intermediate feature map in the cloud-edge-device system are found to obtain the pruning model segmentation point set A3. For each possible segmentation point position in the pruning model segmentation point set A3, its size is limited, that is, the channel of the segmentation point position is pruned first, and then pruning is performed.

[0046] The model to be pruned is divided into a set of structures that can be pruned. A pruning evaluation function is used to score each structure of the model to be pruned. The number of prunings N is controlled by the pruning rate. The structure with the lowest score is removed in reverse order through N loops to obtain the pruned model.

[0047] The pruning evaluation function L_total is expressed as:

[0048]

[0049] Where α, β, γ, and δ are the weight coefficients corresponding to the original task loss L_task, inference time prediction loss L_time, memory usage prediction loss L_mem, and device resource utilization loss L_util;

[0050] The original task loss L_task is represented as:

[0051]

[0052] Where y_i is the actual label, y_hat_i is the model output, and L is the loss function, which is set according to the task requirements. The task-related loss function can be cross-entropy loss, mean squared error, etc.

[0053] The inference time prediction loss is expressed as:

[0054] L_time = (T(θ) / T_max) ^ 2

[0055] Where T(θ) is the inference time prediction model, which is used to estimate the inference time of the model, and T_max is the upper limit of the inference time;

[0056] The memory usage prediction loss L_mem is expressed as:

[0057] L_mem = (M(θ) / M_max) ^ 2

[0058] Where M(θ) is the memory usage prediction model, which is used to estimate the memory usage of the model, and M_max is the set memory limit;

[0059] The equipment resource utilization loss L_util is represented as:

[0060] L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2

[0061] Wherein, U(θ) is the equipment resource utilization prediction model, which is used to estimate resource utilization, and U_ideal is the target utilization.

[0062] Furthermore, in S3, under the narrowband background, based on the communication bandwidth dt, the intermediate feature size of the pruned model transmission feature_out, and the communication overhead feature_out / dt, the structure before the segmentation point of the pruned model is called forward_layers, and the time overhead evaluation function Time is obtained through the time prediction function time_predict.

[0063] The time cost evaluation function Time is represented as:

[0064] Time = feature_out / dt + time_predict(forward_layers)

[0065] The pruned model set that meets the accuracy and resource constraints is selected, its time cost function value is calculated, and the model with the smallest time cost function value is selected as the optimal deployment model under the current network conditions.

[0066] Specifically, in this invention, in order to achieve the optimal deployment of the cloud-edge-device collaborative inference system under narrowband network conditions, a time overhead evaluation function including transmission overhead and local inference overhead is constructed. After the system selects a set of pruned models that meet the accuracy and resource constraints, it calculates their time overhead function values ​​and selects the model with the smallest value as the optimal deployment model under the current network conditions. This evaluation method can dynamically adapt to changes in communication bandwidth and effectively improve the system's real-time response capability and robustness.

[0067] This embodiment takes the perception task of the forward-looking camera in an autonomous vehicle as an example to illustrate the practical application process of the model pruning method proposed in this invention in a heterogeneous cloud-edge-device collaborative system, as detailed below:

[0068] 1. Scene Description:

[0069] In autonomous driving scenarios, forward-looking cameras acquire image data in real time. The images are sent to lightweight object detection models (such as YOLOv5s or RT-DETR) deployed on edge computing units (such as vehicle-mounted SoCs) for preprocessing. The feature maps can be segmented in the middle according to system scheduling and uploaded to the cloud. The cloud model then performs fine-grained recognition or decision processing. Due to bandwidth and computing resource limitations, the model needs to be pruned at the segmentation points and optimally deployed while ensuring accuracy.

[0070] 2. Specific implementation process:

[0071] (1) Model structure analysis and cost predictor generation (corresponding to step S1):

[0072] - Obtain the YOLOv5s model to be deployed and divide it into multiple module structures (such as Backbone_0, Backbone_1, ..., Backbone_n, Neck, Head).

[0073] - Deploy the module structure on the vehicle edge node (such as NVIDIA Jetson Orin) and the cloud server respectively, record the running time, memory usage and resource utilization of different structure combinations, and generate sample records;

[0074] - Train a lightweight prediction model using sample records, including:

[0075] -Time predictor T(θ)

[0076] -Memory predictor M(θ)

[0077] -Resource occupancy predictor U(θ)

[0078] - The above predictor is used to dynamically evaluate the overhead of candidate model structure in subsequent pruning strategies;

[0079] (2) Pruning control and structural assessment (corresponding to step S2):

[0080] - Traverse multiple possible split points in the Backbone (such as after C3 or C4 layers) and perform channel-level pruning on the structure before the split point;

[0081] - During pruning, the number of channels in the region before the segmentation point is controlled to avoid generating excessively large intermediate feature maps (feature_out) and reduce communication costs;

[0082] -Evaluate each pruning structure using the following total loss function L_total:

[0083]

[0084] - Using mAP (mean accuracy) as the task loss, the inference time, memory, and resource utilization of the prediction model are given by T(θ), M(θ), and U(θ), ​​respectively;

[0085] -Under the constraint of keeping the accuracy decrease to no more than 3%, the structure is scored one by one and iteratively pruned to obtain multiple pruned model versions;

[0086] (3) Deployment point evaluation and optimal model selection (corresponding to step S3):

[0087] - In narrowband network environments (such as V2X communication), the bandwidth is set to dt = 1MB / s;

[0088] - Calculate the feature map size feature_out and upload time feature_out / dt for each pruned model;

[0089] - For each pruning model's split point forward_layers, call time_predict(forward_layers) to evaluate the time spent at the edge;

[0090] - The total time cost function Time is represented as:

[0091] Time=feature_out / dt+time_predict(forward_layers)

[0092] - Select the model with the shortest time from the set of models that meet the accuracy and resource constraints for deployment.

[0093] Although the invention has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A model pruning method for heterogeneous cloud-edge-device collaborative systems, characterized in that, Includes the following steps: S1. Initialize the model to be called, parse the structure of the model to be called to form a structure set, distribute it to the computing nodes, and generate the cost predictor; S2. Based on the pruning shape control strategy, call the cost predictor and use the pruning evaluation function to obtain the pruning evaluation function. S3. From the pruned model set at different split points, the optimal deployment model is selected based on the time cost evaluation function and network conditions; In step S2, the model to be pruned is obtained, the segmentation points of the intermediate feature map are found, and the pruning model segmentation point set A3 is obtained. For each possible segmentation point position in the pruning model segmentation point set A3, its size is limited, that is, the channel of the segmentation point position is pruned first, and then pruning is performed. The model to be pruned is divided into a set of structures that can be pruned. A pruning evaluation function is used to score each structure of the model to be pruned. The number of prunings N is controlled by the pruning rate. The structure with the lowest score is removed in reverse order through N loops to obtain the pruned model. The pruning evaluation function L_total is expressed as: L_total = α * L_task + β * L_time + γ * L_mem + δ * L_util Where α, β, γ, and δ are the weight coefficients corresponding to the original task loss L_task, inference time prediction loss L_time, memory usage prediction loss L_mem, and device resource utilization loss L_util; The original task loss L_task is represented as: L_task = (1 / N) * Σ L(y_i, y_hat_i) Where y_i is the actual label, y_hat_i is the model output, and L is the loss function; The inference time prediction loss is expressed as: L_time = (T(θ) / T_max) ^ 2 Where T(θ) is the inference time prediction model, and T_max is the upper limit of inference time; The memory usage prediction loss L_mem is expressed as: L_mem = (M(θ) / M_max) ^ 2 Where M(θ) is the memory usage prediction model, and M_max is the set memory limit; The equipment resource utilization loss L_util is represented as: L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2 Where U(θ) is the equipment resource utilization prediction model, and U_ideal is the target utilization rate; In S3, under the narrowband background, based on the communication bandwidth dt, the intermediate feature size of the pruned model transmission feature_out, and the communication overhead feature_out / dt, the structure before the segmentation point of the pruned model is called forward_layers, and the time overhead evaluation function Time is obtained through the time prediction function time_predict. The time cost evaluation function Time is represented as: Time = feature_out / dt + time_predict(forward_layers) The pruned model set that meets the accuracy and resource constraints is selected, its time cost function value is calculated, and the model with the smallest time cost function value is selected as the optimal deployment model under the current network conditions.

2. The model pruning method for heterogeneous cloud-edge-device collaborative systems according to claim 1, characterized in that, In step S1, a set A1 of models to be called is obtained, the structure of the models to be called is parsed to form a structure set A2, and distributed to computing nodes. The running speed of different structures on different computing nodes is measured, a record file is generated, the record file is transmitted back to the cloud, and a unified training is conducted to generate an overhead predictor for each computing node for subsequent model pruning.

Images