Model pruning method for heterogeneous cloud edge-end cooperative system

Abstract

The invention discloses a model pruning method for a heterogeneous cloud edge-end cooperative system, and belongs to the technical field of model pruning of machine learning. The problem that in the prior art, a traditional pruning method based on a deep learning model is low in resource utilization rate is solved. The method comprises the following steps: S1, initializing a to-be-called model, analyzing the structure of the to-be-called model to form a structure set, distributing the structure set to a computing node, and generating an overhead predictor; s2, according to a pruning shape control strategy, calling an overhead predictor, and using a pruning evaluation function to obtain a used pruning evaluation function; and S3, in the pruned model set of different segmentation points, according to a time overhead evaluation function and a network condition, screening to obtain an optimal deployment model. According to the method, the operation efficiency of the pruned model on different hardware is effectively improved, the reasoning efficiency and the resource utilization rate can be controlled, different task requirements are met through dynamic weight adjustment, and the method can be applied to the pruning model.

Description

TECHNICAL FIELD

[0001] The application relates to a model pruning method, in particular to a model pruning method for a heterogeneous cloud-edge-end collaborative system, and belongs to the technical field of model pruning of machine learning. BACKGROUND

[0002] Cloud-edge-end collaboration, also known as cloud-edge-end integration, is a technology architecture that combines cloud computing, edge computing and terminal devices to optimize the allocation of computing resources, reduce latency and improve the overall performance of the system. This collaborative model is particularly suitable for scenarios such as the Internet of Things (IoT), intelligent manufacturing and autonomous driving that require real-time response and large-scale data processing. For example, in the field of autonomous driving, vehicles need to process data from sensors in real time to ensure safe driving. In a cloud-edge-end collaborative architecture, data and computing tasks need to be dynamically allocated between different levels, which puts high demands on the connection quality and bandwidth of the network. Therefore, how to ensure that the system can still function normally in a narrowband situation, especially in critical application scenarios, is a problem that needs to be solved urgently.

[0003] In a narrowband communication scenario, model pruning can effectively improve the efficiency of edge devices and reduce the amount of data transmission between the cloud and the edge by simplifying deep learning models. First, the number of parameters in the pruned model is significantly reduced, which reduces the computational load and memory requirements of edge devices, allowing them to quickly process inference tasks in resource-limited environments and improve real-time response capabilities. Second, the pruned model is smaller in size, reducing the amount of data that needs to be transmitted during cloud-edge-end collaboration, thereby reducing the occupancy of communication bandwidth and latency. For example, in a multi-mobile platform target tracking application, edge devices can monitor and analyze the positions and motion trajectories of multiple targets in real time. Through model pruning, the computational requirements of the target tracking model are greatly reduced, allowing edge devices to quickly process target information in video streams without the need to frequently send complete data to the cloud for analysis. In this way, pruning optimization not only improves the response speed of target tracking, but also reduces the occupancy of narrowband communication bandwidth, ensuring that the system still functions stably and efficiently in a bandwidth-limited situation. This highlights the urgency and importance of model pruning in solving problems in practical applications.

[0004] At the same time, edge systems are generally heterogeneous, and different edge devices running the same model will have different performance, and different edge devices will have different performance requirements such as real-time performance, accuracy or size. For example, edge devices performing target tracking require high real-time performance, while edge devices performing face recognition focus more on accuracy, and edge devices deployed on resource-limited devices such as stm32 require high size. The current general pruning algorithm cannot fully consider the differences in these heterogeneous systems.

[0005] In the prior art, the pruning methods based on deep learning models are generally divided into the following categories: (1) structured pruning technology. In structured pruning, the pruning operation is concentrated on the level of the entire convolution kernel, channel or layer. Common methods include pruning based on the importance of the convolution kernel. This method selectively prunes the convolution kernel or channel with less contribution by evaluating the contribution of the convolution kernel in the model. According to the evaluation source, it can be divided into parameter-driven and data-driven. The parameter-driven is to start from the weights of the model, evaluate the importance of the convolution kernel according to the standards such as l1 regularization, l2 regularization, parameters in batch normalization and spectral clustering of filters, and then perform pruning. The data-driven is to evaluate the generated intermediate layer feature map after inputting the data. Specifically, there are methods such as the number of activation values in the activation output value, rank decomposition of the feature layer, subspace clustering of the feature map, principal component analysis, and network entropy value before and after pruning. (2) Layer sensitivity analysis pruning technology. Layer sensitivity analysis is a method for evaluating the importance of each layer of a neural network during model pruning. By pruning layers one by one and observing the impact on model performance, it can identify layers that have less impact on the overall performance of the model and can be pruned. The analysis uses gradient information, Hessian matrix, Taylor expansion or other heuristic methods to evaluate the sensitivity of each layer, aiming to find a pruning strategy that reduces computational load and model size while maintaining model performance as much as possible. Through layer sensitivity analysis, the efficiency and deployment capability of the model can be optimized, which has practical application value especially on resource-constrained devices. (3) Adaptive pruning technology. Adaptive pruning technology is a method of dynamically adjusting the degree of model pruning. In adaptive pruning, the parameters of the model are usually multiplied by a trainable mask or gating mechanism. The mask is dynamically updated according to the gradient of the loss function or other indicators, and the pruning process is dynamic. As the training progresses, the pruning decision is adjusted according to the current state of the model, rather than relying on pre-set pruning rules. The pruning strategy is flexibly adjusted according to different data sets and task requirements. However, the above pruning methods have the following problems: (1) Most pruning methods only consider model pruning from the algorithm level, ignoring the differences in computing resources between different nodes in a heterogeneous system, resulting in that the pruned model in a heterogeneous system is difficult to adapt to all nodes and cannot maximize resource utilization. (2) The existing pruning methods mainly evaluate the FLOPs as an indicator to measure the improvement of model inference speed, ignoring the fact that in practical applications, inference time and memory size can more directly reflect the actual running situation of the model. (3) The current cloud-edge-end collaborative system frequently transmits intermediate feature maps for collaborative computation during collaboration. Narrowband transmission takes longer than normal network transmission and is more difficult to bear. However, existing pruning algorithms mostly focus on balancing the accuracy and model size without considering controlling the model shape.

[0006] In summary, there is a need for a model pruning method for a heterogeneous cloud edge-end collaborative system that considers the computing resources of different nodes when pruning. SUMMARY

[0007] A brief summary of the application is presented below in order to provide a basic understanding of some aspects of the application. It should be appreciated that this summary is not an exhaustive overview of the application. It is not intended to identify key or critical elements of the application or to delineate the scope of the application. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

[0008] In view of this, to solve the problem of low resource utilization of the traditional deep learning model-based pruning method in the prior art, the application provides a model pruning method for a heterogeneous cloud edge-end collaborative system.

[0009] The technical scheme is as follows: a model pruning method for a heterogeneous cloud edge-end collaborative system, comprising the following steps:

[0010] S1. initializing the to-be-called model, parsing the structure of the to-be-called model to form a structure set, and distributing to the computing nodes to generate an overhead predictor;

[0011] S2. according to the pruning shape control strategy, calling the overhead predictor, using the pruning evaluation function, obtaining the pruning evaluation function using the pruning evaluation function;

[0012] S3. in the model set after pruning at different partition points, according to the time overhead evaluation function and the network condition, the optimal deployment model is screened.

[0013] Further, in S1, a set A1 of to-be-called models is obtained, the structure of the to-be-called model is parsed to form a structure set A2, and is distributed to the computing nodes, the running speed of different structures on different computing nodes is measured respectively, a record file is generated, the record file is transmitted back to the cloud, and an overhead predictor for each computing node is generated for subsequent model pruning calling.

[0014] Further, in S2, the to-be-pruned model is obtained, the partition point of the transmission intermediate feature map is found, the pruning model partition point set A3 is obtained, for each possible partition point position in the pruning model partition point set A3, the size is limited, that is, the channels of the partition point position are preferentially pruned, and then pruning is performed;

[0015] The to-be-pruned model is divided into a set of prunable structures, the pruning evaluation function is used to score each structure of the to-be-pruned model, the pruning number N is controlled through the pruning rate, and the lowest scoring structure is removed through N times of loop in reverse order, to obtain the pruned model;

[0016] The pruning evaluation function L_total is represented as:

[0017]

[0018] wherein α, β, γ, δ are weight coefficients corresponding to the original task loss L_task, the inference time prediction loss L_time, the occupied memory prediction loss L_mem, and the device resource utilization loss L_util.

[0019] The original task loss L_task is represented as:

[0020]

[0021] wherein y_i is an actual label, y_hat_i is a model output, and L is a loss function.

[0022] The inference time prediction loss is represented as:

[0023] L_time = (T(θ) / T_max) ^ 2

[0024] wherein T(θ) is an inference time prediction model, and T_max is an upper limit of inference time.

[0025] The occupied memory prediction loss L_mem is represented as:

[0026] L_mem = (M(θ) / M_max) ^ 2

[0027] wherein M(θ) is an occupied memory prediction model, and M_max is a set upper limit of memory.

[0028] The device resource utilization loss L_util is represented as:

[0029] L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2

[0030] wherein U(θ) is a device resource utilization prediction model, and U_ideal is a target utilization.

[0031] Further, in S3, in a narrowband context, according to a communication bandwidth dt, an intermediate feature size of a pruned model transmission feature_out, and a communication overhead feature_out / dt, a structure before a pruning point of the pruned model is referred to as forward_layers, and a time prediction function time_predict is used to obtain a time overhead evaluation function Time.

[0032] The time overhead evaluation function Time is represented as:

[0033] Time = feature_out / dt + time_predict(forward_layers)

[0034] Screening the pruned model set meeting the accuracy and resource constraint conditions, calculating the time overhead function value thereof, and selecting the model with the minimum time overhead function value as the optimal deployment model under the current network condition.

[0035] The beneficial effects of the present application are as follows: the present application proposes a pruning method that can reduce overhead by controlling the shape of the pruned model and the position of the intermediate feature map in transmission, and considering the resource occupation problem of the model in the model pruning stage can better adapt to the actual deployment scenario; in terms of improving the adaptability of the heterogeneous system, the traditional pruning algorithm usually ignores the resource differences of different hardware devices, while the present application can better adapt to the heterogeneous system by introducing the overhead predictor and the device resource usage loss function, and ensure that the pruned model runs efficiently on different hardware; in terms of comprehensive optimization of model performance, by adding the weights of inference time, memory occupation and resource utilization in the loss function, the model pruning no longer simply pursues accuracy and model size, but also can control the inference efficiency and resource usage, achieving more comprehensive performance optimization; in terms of automatic model segmentation and shape control, the present application can automatically generate multiple pruned models and select the model with the optimal performance by analyzing the structure of the model, automatically finding the segmentation point, and controlling the shape of the model in different pruning schemes, avoiding the complexity of manual debugging; in terms of dynamic weight adjustment, the set weight coefficient enables the user to flexibly adjust the model pruning strategy according to specific task requirements and hardware conditions, and find a balance point under different resource constraint conditions. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0037] Figure 1 It is a flowchart of a model pruning method for a heterogeneous cloud edge-end collaborative system;

[0038] Figure 2 It is a flowchart of an embodiment of a model pruning method for a heterogeneous cloud edge-end collaborative system. DETAILED DESCRIPTION

[0039] In order to make the technical solutions and advantages of the embodiments of the present application clearer, the exemplary embodiments of the present application are further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than an exhaustive enumeration of all embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0040] Reference Figure 1 and Figure 2 In detail, a model pruning method for a heterogeneous cloud edge-end collaborative system specifically includes the following steps:

[0041] S1. The to-be-called model is initialized, the structure of the to-be-called model is parsed to form a structure set, and is distributed to a computing node to generate an overhead predictor;

[0042] S2. According to a pruning shape control strategy, the overhead predictor is called, and a pruning evaluation function is used to obtain a pruning evaluation function using the pruning evaluation function;

[0043] S3. In the model set after pruning at different partition points, the optimal deployment model is selected according to a time overhead evaluation function and a network condition.

[0044] Further, in S1, a set A1 of to-be-called models is obtained, the structure of the to-be-called model is parsed to form a structure set A2, and is distributed to a computing node to measure the running speed of different structures on different computing nodes, a record file is generated, the record file is transmitted back to the cloud, and an overhead predictor for each computing node is trained to be used for subsequent model pruning.

[0045] Further, in S2, a to-be-pruned model is obtained, a partition point of a possible intermediate feature map in a cloud edge-end system is found, a set A3 of pruning model partition points is obtained, for each possible partition point position in the set A3 of pruning model partition points, the size of the partition point position is limited, that is, the channels of the partition point position are preferentially pruned, and then pruning is performed;

[0046] The to-be-pruned model is divided into a set of prunable structures, a pruning evaluation function is used to score each structure of the to-be-pruned model, the number N of pruning is controlled through a pruning rate, and the lowest scoring structure is removed through N times of loop reverse pruning to obtain a pruned model;

[0047] The pruning evaluation function L_total is expressed as:

[0048]

[0049] Wherein, alpha, beta, gamma, delta are the weight coefficients corresponding to the original task loss L_task, inference time prediction loss L_time, memory occupation prediction loss L_mem, and device resource utilization loss L_util.

[0050] The original task loss L_task is expressed as:

[0051]

[0052] Wherein, y_i is the actual label, y_hat_i is the model output, and L is the loss function. According to the task requirements, the task-related loss function can be cross-entropy loss, mean square error, etc.

[0053] The inference time prediction loss is expressed as:

[0054] L_time = (T(θ) / T_max) ^ 2

[0055] Wherein, T(θ) is the inference time prediction model, which is used to estimate the inference time of the model, and T_max is the upper limit of the inference time.

[0056] The memory occupation prediction loss L_mem is expressed as:

[0057] L_mem = (M(θ) / M_max) ^ 2

[0058] Wherein, M(θ) is the memory occupation prediction model, which is used to estimate the memory usage of the model, and M_max is the set memory upper limit.

[0059] The device resource utilization loss L_util is expressed as:

[0060] L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2

[0061] Wherein, U(θ) is the device resource utilization prediction model, which is used to estimate the resource utilization, and U_ideal is the target utilization.

[0062] Further, in S3, in the narrowband context, according to the communication bandwidth dt, the intermediate feature size of the pruned model transmission feature_out, and the communication overhead feature_out / dt, the structure before the pruning point of the pruned model is called forward_layers, and the time prediction function time_predict is used to obtain the time overhead evaluation function Time.

[0063] The time overhead evaluation function Time is expressed as:

[0064] Time = feature_out / dt + time_predict(forward_layers)

[0065] The pruned model set meeting the accuracy and resource constraint conditions is screened out, the time overhead function value is calculated, and the model with the minimum time overhead function value is selected as the optimal deployment model under the current network condition.

[0066] Specifically, in the present application, in order to realize the optimal deployment of the cloud-edge-end collaborative reasoning system under the narrowband network condition, a time overhead evaluation function including transmission overhead and local reasoning overhead is constructed, after the pruned model set meeting the accuracy and resource constraint conditions is screened out, the time overhead function value is calculated, and the model with the minimum value is selected as the optimal deployment model under the current network condition, and the evaluation method can dynamically adapt to the change of communication bandwidth, effectively improving the real-time response ability and robustness of the system.

[0067] The present embodiment takes the front-view camera perception task in an autonomous vehicle as an example to illustrate the actual application process of the model pruning method proposed in the present application in a heterogeneous cloud-edge-end collaborative system, which is as follows:

[0068] 1. Scene description:

[0069] In the autonomous driving scene, the front-view camera collects image data in real time, the image is sent to the lightweight target detection model (such as YOLOv5s or RT-DETR) deployed on the edge computing unit (such as vehicle-mounted SoC) for preprocessing, and the feature map can be uploaded to the cloud after being cut in the middle according to system scheduling, and the cloud model further completes fine-grained recognition or decision processing. Due to the limitation of bandwidth and computing resources, the model needs to be pruned according to the partition point under the premise of ensuring accuracy, and the optimal deployment is needed.

[0070] 2. Specific implementation process:

[0071] (1) Model structure analysis and overhead predictor generation (corresponding to step S1):

[0072] -Obtain the YOLOv5s model to be deployed, and divide it into multiple module structures (such as Backbone_0, Backbone_1, …, Backbone_n, Neck, Head).

[0073] -Deploy the module structures on the vehicle-mounted edge node (such as NVIDIA Jetson Orin) and the cloud server respectively, record the running time, memory usage and resource occupancy rate of different structure combinations, and generate sample records;

[0074] -Train a lightweight overhead prediction model using sample records, including:

[0075] - Time predictor T(θ)

[0076] - Memory predictor M(θ)

[0077] - Resource usage predictor U(θ)

[0078] - The above predictors are used to dynamically evaluate the overhead of candidate model structures in subsequent pruning strategies;

[0079] (2) Pruning control and structure evaluation (corresponding to step S2):

[0080] - Traverse multiple possible split points in Backbone (such as after C3 or C4 layer), and perform channel-level pruning on the structure before the split point;

[0081] - When pruning, first control the number of channels in the region before the split point to avoid generating excessively large intermediate feature maps (feature_out) and reduce communication costs;

[0082] - Evaluate each pruned structure using the following total loss function L_total:

[0083]

[0084] - Use mAP (mean average precision) as the task loss, and the prediction model inference time, memory, and resource usage are given by T(θ), M(θ), and U(θ), respectively;

[0085] - Under the constraint that the accuracy drop does not exceed 3%, score each structure and iteratively prune to obtain multiple pruned model versions;

[0086] (3) Deployment point evaluation and optimal model selection (corresponding to step S3):

[0087] - In a narrowband network environment (such as V2X communication), set the bandwidth to dt = 1MB / s;

[0088] - Calculate the feature map size feature_out and upload time feature_out / dt for each pruned model;

[0089] - For the split point forward_layers of each pruned model, call time_predict(forward_layers) to evaluate the edge end time;

[0090] - The total time overhead function Time is represented as:

[0091] Time = feature_out / dt + time_predict(forward_layers)

[0092] - Select the model with minimum Time from the set of models satisfying the accuracy and resource constraints for deployment.

[0093] While the application has been described in terms of several embodiments, it is to be appreciated that those skilled in the art will readily apply the teachings of this application for other embodiments. Accordingly, the teachings of the present application should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. Numerous modifications and adaptations will be apparent to those skilled in the art without departing from the scope and spirit of the application. Accordingly, the scope of the present application is defined only by the appended claims and equivalents thereto.

Claims

1. A model pruning method for heterogeneous cloud-edge-device collaborative systems, characterized by: The following steps are involved: S1. Initialize the model to be called, parse the structure of the model to be called to form a structure set, distribute it to the computing nodes, and generate a cost predictor; S2. Based on the pruning shape control strategy, call the cost predictor and use the pruning evaluation function to obtain the pruning evaluation function; S3. In the set of pruned models at different split points, the optimal deployment model is selected based on the time cost evaluation function and network conditions.

2. A model pruning method for heterogeneous cloud-edge-device collaborative systems according to claim 1, characterized in that: In S1, a set A1 of models to be called is obtained, the structures of the models to be called are parsed to form a structure set A2, and the structures are distributed to the computing nodes. The running speeds of different structures on different computing nodes are measured respectively, and record files are generated. The record files are transmitted back to the cloud, and unified training is performed to generate a cost predictor for each computing node for subsequent model pruning and calling.

3. A model pruning method for heterogeneous cloud-edge-device collaborative systems according to claim 2, characterized in that: In S2, the model to be pruned is obtained, and the segmentation points of the transmitted intermediate feature map are found to obtain a set A3 of segmentation points of the pruned model. For each possible segmentation point position in the set A3 of segmentation points of the pruned model, its size is limited, that is, the channels at the segmentation point positions are preferentially pruned before pruning; The model to be pruned is divided into a set of structures that can be pruned. Each structure of the model to be pruned is scored using a pruning evaluation function. The number of pruning steps N is controlled by the pruning rate. The structure with the lowest score is pruned in reverse order through N cycles to obtain the pruned model. The pruning evaluation function L_total is expressed as: ； Among them, α, β, γ, and δ are the weight coefficients corresponding to the original task loss L_task, the inference time prediction loss L_time, the memory usage prediction loss L_mem, and the device resource utilization loss L_util; The original task loss L_task is expressed as: ； Among them, y_i is the actual label, y_hat_i is the model output, and L is the loss function; The inference time prediction loss is expressed as: L_time = (T(θ) / T_max) ^ 2; Among them, T(θ) is the inference time prediction model, and T_max is the upper limit of inference time; The memory usage prediction loss L_mem is expressed as: L_mem = (M(θ) / M_max) ^ 2; Among them, M(θ) is the memory usage prediction model, and M_max is the set memory upper limit; The device resource utilization loss L_util is expressed as: L_util = ((U(θ) - U_ideal) / U_ideal) ^ 2; Among them, U(θ) is the device resource utilization prediction model, and U_ideal is the target utilization.

4. A model pruning method for heterogeneous cloud-edge-device collaborative systems according to claim 3, characterized in that: In S3, in a narrowband context, based on the communication bandwidth dt, the intermediate feature size transmitted by the pruned model is feature_out, and the communication overhead is feature_out / dt, the structure before the split point of the pruned model is called forward_layers, and the time overhead evaluation function Time is obtained through the time prediction function time_predict; The time cost evaluation function Time is expressed as: Time = feature_out / dt + time_predict(forward_layers); Filter out the pruned model set that meets the accuracy and resource constraints, calculate its time cost function value, and select the model with the smallest time cost function value as the optimal deployment model under the current network conditions.

Images