An end-to-end cloud model compression and deployment method based on split learning

Abstract

The application relates to the technical field of deep learning, and discloses an end-edge-cloud model compression and deployment method based on split learning, which comprises the following steps: a deep learning model is split into a front model, a middle model and a rear model, and the three models are deployed on a client, an edge side and a cloud side; the client performs forward propagation on the front model, and sends the forward propagation result to the edge side; the edge side receives the forward propagation result from at least one client, performs forward propagation on the received result by using the middle model, and sends the forward propagation result to the cloud side; the cloud side receives the forward propagation result from at least one edge side, completes forward propagation by using the rear model, calculates a loss function, performs back propagation, and updates the front model, the middle model and the rear model; the front model, the middle model and the rear model are iteratively pruned, and the pruned models are fine-tuned by using knowledge distillation; and the lightened front model is deployed to the client to perform inference locally.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for compressing and deploying edge-cloud models based on split learning. Background Technology

[0002] In recent years, deep learning technology has achieved significant breakthroughs in various fields and has been widely applied to tasks such as image recognition and natural language processing. However, deploying computationally intensive deep learning models to resource-constrained edge devices (such as smartphones and IoT devices) still faces significant challenges. While traditional cloud-based inference models possess powerful computing resources, they require uploading terminal data to the cloud for processing. This not only introduces significant communication overhead and high inference latency but also potentially raises the risk of user privacy breaches.

[0003] To address these challenges, the edge-cloud collaborative architecture has emerged. This architecture aims to leverage the computing power of the edge to alleviate the computing pressure on the cloud and reduce data transmission between the edge and the cloud, thereby mitigating latency and communication overhead to some extent. However, even under this architecture, directly deploying a complete deep learning model to resource-constrained edge devices remains impractical. Therefore, model compression technology has become crucial for achieving efficient edge inference.

[0004] Currently, mainstream model compression methods include pruning, quantization, and knowledge distillation. Traditional model pruning methods are typically applied directly to the complete model, reducing model size and computational complexity by removing unimportant connections or neurons. However, in an edge-cloud architecture, simply pruning the model in the cloud and deploying it directly to the edge may still fail to meet the resource constraints of edge devices and ignores the distributed nature of the edge-cloud architecture. Furthermore, simple pruning operations can easily lead to a significant decrease in model accuracy.

[0005] Split learning, as an emerging distributed training method, splits deep learning models across different devices for training, effectively protecting data privacy. However, existing split learning methods primarily focus on the distributed training process, with less consideration for model compression and deployment. Directly deploying the split model to the edge may still result in an excessively large model size, failing to fully leverage the advantages of local inference on the edge. Specifically:

[0006] Traditional model compression methods are not well adapted to edge-cloud architectures: Traditional pruning and quantization methods are usually applied to the complete model and do not fully consider the distributed characteristics and communication constraints of edge-cloud architectures. As a result, the compressed model may still be too large or have high communication overhead when deployed on the edge.

[0007] Traditional knowledge distillation methods suffer from insufficient information transfer: Traditional knowledge distillation mainly focuses on knowledge transfer from the output layer, ignoring the rich feature information in the intermediate layers of the model. In edge-cloud architecture, especially when the model is split, this may result in the student model failing to fully learn the knowledge of the teacher model, leading to limited accuracy recovery.

[0008] Lack of collaborative optimization strategies for edge-cloud architecture: Existing technologies rarely consider model splitting, compression and deployment as a whole, lacking collaborative optimization strategies for the characteristics of edge-cloud architecture, and failing to fully utilize the advantages of edge-cloud architecture to achieve low latency and low communication overhead edge inference.

[0009] Due to the aforementioned shortcomings in existing technologies, deploying deep learning models in an edge-cloud architecture still faces challenges such as high communication overhead and high inference latency. Therefore, there is an urgent need for a technical solution that can fully utilize the characteristics of the edge-cloud architecture, efficiently compress models, and ensure accuracy, in order to achieve low-latency, low-communication-overhead local inference on the edge. Summary of the Invention

[0010] This invention provides a method for edge-cloud model compression and deployment based on split learning, which solves the technical problems of insufficient adaptability of traditional model compression methods to edge-cloud architecture, insufficient information transfer in traditional knowledge distillation methods, and lack of collaborative optimization strategies for edge-cloud architecture.

[0011] This invention provides a method for compression and deployment of edge-cloud models based on split learning, comprising the following steps:

[0012] S1: The deep learning model to be compressed is split into a front-end model, a middle-end model, and a back-end model, which are deployed on the client, edge, and cloud respectively.

[0013] S2: The client uses local data to perform forward propagation on the front-end model deployed on the client and sends the forward propagation results to the edge side;

[0014] S3: The edge side receives the forward propagation results from at least one client, uses the middle model deployed on the edge side to propagate the received results forward, and sends the forward propagation results to the cloud;

[0015] S4: The cloud receives the forward propagation results from at least one edge side, completes the forward propagation using the back-end model deployed in the cloud, calculates the loss function, and performs back propagation based on the loss function to update the parameters of the front-end model, middle-end model, and back-end model.

[0016] S5: Iteratively prune the front-end model deployed on the client, the middle-end model deployed on the edge, and the back-end model deployed in the cloud, and fine-tune the pruned model using knowledge distillation;

[0017] S6: Deploy the lightweight front-end model to the client for inference on the client's local machine.

[0018] Preferably, in step S1, the deep learning model is manually split into a front-end model, a middle-end model, and a back-end model based on the resource constraints of the client, the edge, and the cloud.

[0019] I. Edge Cloud Resource Status

[0020] (1) End-side resource status

[0021] Suppose there are m different end-side devices, denoted as clients C1, C2, ..., Cn. m For each client, we can describe its resource status using several metrics.

[0022] 1. Acceptable computing power C for the client comp .use Indicates client C k Acceptable computing power (k = 1, 2, ..., m) is measured in floating-point operations per second (FLOPs).

[0023] 2. Available memory capacity on the client side (C) mem .use Indicates client C k The available memory capacity for (k = 1, 2, ..., m) is expressed in bytes.

[0024] (2) Edge-side resource situation

[0025] Suppose there are n distinct edge devices, denoted as edge devices E1, E2, ..., E... n For each edge side, we can describe its resource situation using several metrics.

[0026] 1. Acceptable computing power at the edge: E comp .use Indicates client E k Acceptable computing power (k = 1, 2, ..., n) is measured in floating-point operations per second (FLOPs).

[0027] 2. Available memory capacity on the edge side E mem .use Indicates client C kThe available memory capacity of (k = 1, 2,..., n), in bytes (Byte).

[0028] (3) Cloud resource situation

[0029] The acceptable computing power of the cloud is represented by Cloud comp while the available memory capacity is represented by Cloud mem to represent.

[0030] II. Determination of dynamic splitting points

[0031] (1) Traverse the combination of splitting points

[0032] Suppose the model M = {M1, M2,..., M L} with L layers is split into three parts (the front part M 1:x , the middle part M x:y , the rear part M y:L ) according to the splitting points x and y, and are respectively deployed to the edge side, the edge side, and the cloud side. Among them, 1 < x < y < L. By traversing the splitting points x and y and calculating the scores in each case, the optimal splitting points x and y are determined. Start traversing from x = 2 and end at x = L - 2. For each x value: start traversing from y = x + 1 and end at y = L - 1. For each combination of x and y, the following evaluation is carried out.

[0033] (2) Evaluation

[0034] For the current splitting points x and y, the performance indicators of each part of the model are as follows:

[0035] The computing power required by the model M comp . The computing power required for each part is expressed as: the front part the middle part the rear part

[0036] The memory capacity required by the model M mem [[ID='48']]. The computing power required for each part is expressed as: the front part the middle part the rear part

[0037] During evaluation, we mainly consider the communication cost and training delay during the training process.

[0038] The communication cost can be regarded as the sum of the communication costs between the edge and the cloud under the premise of training once. The communication between the edge and the cloud is closely related to x. We use C x-cost 1 to represent the communication cost between a certain edge and a certain cloud. Then the total communication cost between m different edge devices and the edge side is C ce-cost = m·C x-costSimilarly, using C... y-cost Let C represent the communication cost between the edge and the cloud. Then, the total communication cost between n different edge devices and the cloud is C. ec-cost =n·C y-cost .

[0039] In summary, the total communication cost is C. cost =C ce-cost +C ec-cost

[0040] The communication cost score is calculated by mapping communication costs to a specific scoring zone, where the minimum communication cost is set to 1. The maximum value is (This can be determined based on practical experience), so the formula for calculating the communication cost score is:

[0041]

[0042] Training latency can be considered as the sum of the inference latency of each part of the model at the edge, device, and cloud, assuming training is performed only once. We use T to represent this latency. client-delay T edge-delay and T cloud-delay Let T represent the training latency at the edge, device, and cloud. The total training latency is then: T delay =T client-delay +T edge-delay +T cloud-delay .

[0043] Training latency score is calculated by mapping training latency to a specific scoring partition, where the minimum training latency is set to 1. The maximum value is (This can be determined based on practical experience), so the formula for calculating the communication cost score is:

[0044]

[0045] Taking all the above factors into account, the total score is calculated using a weighted method. Let the weight of the communication cost score be α, and the weight of the training latency score be β (and α + β = 1, the weight values ​​can be set according to the emphasis on communication cost and inference latency in the actual application scenario). The calculation formula is as follows:

[0046] score=(α·score c +β·score T )·θ

[0047] Where θ is the penalty factor (initialized to 1, the penalty state is a very small number, such as 0.001), and the score is penalized when the following conditions occur.

[0048] End side: or

[0049] Edge side: or

[0050] Cloud: or

[0051] Finally, the best split points x and y that best match the resource environment are selected based on the traversal.

[0052] Preferably, in step S2, the client processes local data and performs forward propagation of the front model until a predefined cutting layer is reached; the output of the client's forward propagation is represented as:

[0053]

[0054] in, It is the i-th front model, x (i) This is the input data from the i-th client device. These are the parameters of the i-th front model.

[0055] Preferably, in step S3, the edge side receives all intermediate results sent by the clients. The data is then aggregated to obtain h1, which is used as the input to the edge side to continue forward propagation. The output of the edge side forward propagation is represented as:

[0056]

[0057] in It is the j-th middle model. These are the parameters of the j-th middle model.

[0058] Preferably, in step S4, the cloud receiving forward propagation results from at least one edge side and completing the forward propagation using a backend model deployed in the cloud includes: the cloud receiving intermediate results sent by the edge side. Then, the models are aggregated to obtain h2, and the final forward propagation is completed to obtain the model output y. pred The output propagated forward from the cloud is represented as:

[0059] y pred =f Cloud (h2,θ Cloud )

[0060] Where f Cloud It is a cloud-based model, θ Cloud These are the parameters of the cloud model.

[0061] Preferably, in step S4, calculating the loss function and performing backpropagation based on the loss function to update the parameters of the front model, the middle model, and the back model includes:

[0062] According to y in the cloud pred The loss function L = (y) is calculated with the true label y. pred The gradient of the loss function with respect to the model parameters is calculated in the cloud and passed to the edge side;

[0063]

[0064] The edge side updates its model portion based on the gradient and computes the gradient passed to the client;

[0065]

[0066] The client updates its model based on the gradient, thus completing the model update.

[0067] Preferably, in step S5, the fine-tuning of the pruned model using knowledge distillation technology includes: taking the unpruned model part as the teacher and the pruned model part as the student, performing fine-tuning using multi-level knowledge distillation, the multi-level distillation fine-tuning includes intermediate feature loss and soft label loss, and simultaneously introducing hard label cross-entropy loss to form the total loss of multi-level distillation fine-tuning, and training the student model.

[0068] Preferably, the intermediate feature loss is expressed as:

[0069]

[0070] in, Represented as a feature map of the student model; This represents the feature map of its corresponding teacher model A; r(·) is a regression variable consisting of a 1×1 convolutional layer and a BN layer; D p It is a measure of the L2 distance between the feature maps of students and teachers;

[0071]

[0072] in, It is the middle feature of the front model on the end side. It is the intermediate feature of the middle model on the edge side;

[0073] The soft label loss is expressed as:

[0074]

[0075] Where x ijX represents the logical output of the student model for the j-th class in the i-th batch of samples; ij and Let T represent the soft output of the student model and teacher model A of the j-th class in the i-th batch of samples, respectively; the temperature parameter T determines the degree of softening of the output.

[0076] The cross-entropy loss with introduced hard labels is expressed as:

[0077]

[0078] in, Y represents the logical output of the j-th class of the i-th batch of samples; ij The hard label representing the j-th class of the i-th batch of samples;

[0079] The total loss of the multi-stage distillation fine-tuning is expressed as:

[0080] L=δl inter +εl output +θl CE

[0081] Where δ, ε, and θ represent the weights of the intermediate feature loss, soft label loss, and hard label loss, respectively, and δ+ε+θ=1.

[0082] Preferably, in step S6, deploying the lightweight front-end model to the client includes:

[0083] Combine the front model with the middle and rear models;

[0084] The combined model is deployed to the client to complete the inference task locally on the client.

[0085] In the local inference process on the client side, there is no need to transmit data to the edge side or the cloud, so as to reduce communication overhead and inference latency.

[0086] Preferably, the combination of the front model with the middle model and the rear model specifically involves:

[0087]

[0088] Among them, f ji It is a combination of the cloud model, the j-th edge model, and the j-th client model.

[0089] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0090] This invention discloses an edge-cloud model compression and deployment method based on split learning. Model splitting and distributed training reduce the computational burden of a single device; the aggregation effect on the edge side reduces the computational burden and communication overhead in the cloud; iterative pruning effectively reduces model size and computational complexity; knowledge distillation technology compensates for the accuracy loss caused by pruning; and edge-side local inference achieves low latency and low communication overhead. Attached Figure Description

[0091] Figure 1 This is a schematic diagram of a method for compressing and deploying edge-cloud models based on split learning, provided by an embodiment of the present invention. Detailed Implementation

[0092] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] like Figure 1 As shown, this application provides a method for edge-cloud model compression and deployment based on split learning, including the following steps:

[0094] S1: The deep learning model to be compressed is split into a front-end model, a middle-end model, and a back-end model, which are deployed on the client, edge, and cloud respectively.

[0095] S2: The client uses local data to perform forward propagation on the front-end model deployed on the client and sends the forward propagation results to the edge side;

[0096] S3: The edge side receives the forward propagation results from at least one client, uses the middle model deployed on the edge side to propagate the received results forward, and sends the forward propagation results to the cloud;

[0097] S4: The cloud receives the forward propagation results from at least one edge side, completes the forward propagation using the back-end model deployed in the cloud, calculates the loss function, and performs back propagation based on the loss function to update the parameters of the front-end model, middle-end model, and back-end model.

[0098] S5: Iteratively prune the front-end model deployed on the client, the middle-end model deployed on the edge, and the back-end model deployed in the cloud, and fine-tune the pruned model using knowledge distillation;

[0099] S6: Deploy the lightweight front-end model to the client for inference on the client's local machine.

[0100] Preferably, in step S1, the deep learning model is manually split into a front-end model, a middle-end model, and a back-end model based on the resource constraints of the client, the edge, and the cloud.

[0101] In the above scheme, model decomposition is a prerequisite for model deployment. Each sub-model needs to be deployed to its corresponding computing node based on its function and data dependencies. The front-end model processes the raw input data, the middle-end model extracts and aggregates intermediate layer features, and the back-end model completes the final prediction or classification task. The deep learning model to be compressed is first analyzed for its network structure, identifying sub-networks suitable for running on different computing nodes. For example, shallow networks with low computational cost and high real-time requirements can be deployed as front-end models on the client side; intermediate-end networks with moderate computational cost and requiring aggregation of data from multiple clients can be deployed as middle-end models on the edge side; and deep networks with high computational cost and relatively low real-time requirements can be deployed as back-end models in the cloud. This decomposition method effectively utilizes the distributed computing capabilities of the edge-cloud architecture. Through model decomposition, a preliminary realization is achieved of breaking down a large model into parts that can run on resource-constrained clients, laying the foundation for subsequent local inference on the edge side and providing the possibility of using edge computing power to alleviate cloud pressure.

[0102] Preferably, in step S2, the client processes local data and performs forward propagation of the front model until a predefined cutting layer is reached; the output of the client's forward propagation is represented as:

[0103]

[0104] in, It is the i-th front model, x (i) This is the input data from the i-th client device. These are the parameters of the i-th front model.

[0105] In the above scheme, the client refers to the edge device that executes the input layer and part of the hidden layer of the model. These devices typically have limited computing and storage resources. The client uses locally collected data as input to drive the front-end model deployed on it to perform calculations and generate intermediate feature representations. The forward propagation result is the key data link connecting the client and the edge device. After collecting local data, the client inputs it into the front-end model for layer-by-layer calculations until the output layer of the front-end model. The output result, usually a high-dimensional feature vector, is serialized and transmitted to the edge device via the network. This process demonstrates the initial data processing capabilities of the edge device and its collaborative working mode with the edge device. Utilizing the client's computing resources for initial data processing reduces the amount of data that needs to be transmitted to the edge device, thus reducing communication overhead.

[0106] Preferably, in step S3, the edge side receives all intermediate results sent by the clients. The data is then aggregated to obtain h1, which is used as the input to the edge side to continue forward propagation. The output of the edge side forward propagation is represented as:

[0107]

[0108] in It is the j-th middle model. These are the parameters of the j-th middle model.

[0109] In the above scheme, the edge side receives forward propagation results from multiple clients, which serve as input to the mid-level model. Acting as a bridge between the clients and the cloud, it possesses stronger computing power than edge devices, enabling it to handle more complex tasks or larger model parts. As an intermediate layer, the edge side is responsible for aggregating local information from multiple clients. The received forward propagation results are input into the mid-level model for further feature extraction and fusion. The output of the mid-level model, typically a higher-level feature representation, is serialized and transmitted to the cloud via the network. The edge side plays a crucial role in this process, both alleviating the computational burden on the clients and reducing the amount of data that the cloud needs to process. Utilizing the edge side's computing power for the aggregation and processing of mid-level features further reduces the computational burden and communication overhead on the cloud.

[0110] Preferably, in step S4, the cloud receiving forward propagation results from at least one edge side and completing the forward propagation using a backend model deployed in the cloud includes: the cloud receiving intermediate results sent by the edge side. Then, the models are aggregated to obtain h2, and the final forward propagation is completed to obtain the model output y. pred The output propagated forward from the cloud is represented as:

[0111] y pred =f Cloud (h2,θ Cloud )

[0112] Where f Cloud It is a cloud-based model, θ Cloud These are the parameters of the cloud model.

[0113] In the above scheme, the cloud has powerful computing and storage resources and is responsible for executing the final part of the model, including the final hidden layer and output layer, and performing the final optimization of the model; after receiving the feature representation from the edge side, the cloud inputs it into the back-end model to complete the final prediction or classification task.

[0114] Preferably, in step S4, calculating the loss function and performing backpropagation based on the loss function to update the parameters of the front model, the middle model, and the back model includes:

[0115] According to y in the cloud pred The loss function L = (y) is calculated with the true label y. pred The gradient of the loss function with respect to the model parameters is calculated in the cloud and passed to the edge side;

[0116]

[0117] The edge side updates its model portion based on the gradient and computes the gradient passed to the client;

[0118]

[0119] The client updates its model based on the gradient, thus completing the model update.

[0120] In the above scheme, the calculation of the loss function depends on the output of the back-end model and the true labels. Backpropagation uses the gradient information of the loss function to update the parameters of the entire distributed model; based on the model's prediction results and the true labels, the loss function is calculated to measure the model's performance. Then, using the backpropagation algorithm, the gradient information of the loss function is passed back layer by layer, and the parameters of the front-end model, middle-end model, and back-end model are updated according to the gradient information. This process realizes distributed training with edge-cloud collaboration. The powerful computing capabilities of the cloud are used to complete the final training and optimization of the model, ensuring the overall performance of the model. By updating the parameters of the distributed model through backpropagation, edge-cloud collaborative training is achieved, overcoming the data silo problem.

[0121] Preferably, in step S5, the fine-tuning of the pruned model using knowledge distillation includes: taking the unpruned model part as the teacher and the pruned model part as the student, performing fine-tuning using multi-level knowledge distillation, the multi-level distillation fine-tuning includes intermediate feature loss and soft label loss, and simultaneously introducing hard label cross-entropy loss to form the total loss of multi-level distillation fine-tuning, and training the student model.

[0122] After the training phase is complete, the original model is obtained, and it will be pruned. Iterative pruning will be performed according to a preset pruning rate.

[0123] For the k-th filter in the convolutional layer, its importance is calculated using the L1 norm:

[0124]

[0125] Among them, C in K is the number of input channels. h ×Kw W is the kernel size. k,i,j This represents the weight value of the k-th filter at the i-th input channel and the j-th spatial position.

[0126] The model has L layers, each with an independently preset pruning rate:

[0127]

[0128] in, Let μ be the initial pruning rate of the l-th layer. (l) is the attenuation coefficient of the l-th layer, which can control the attenuation rate of the pruning rate of each layer.

[0129] The cumulative pruning rate of layer l after t iterations:

[0130]

[0131] Global model cumulative pruning rate:

[0132]

[0133] Params (l) These are the parameters of the l-th layer.

[0134] The iteration terminates when the target pruning rate is reached.

[0135] GlobalPruneRate t ≥MaxGlobalPruneRate

[0136] Preferably, the intermediate feature loss is expressed as:

[0137]

[0138] in, Represented as a feature map of the student model; This represents the feature map of its corresponding teacher model A; r(·) is a regression variable consisting of a 1×1 convolutional layer and a BN layer; D p It is a measure of the L2 distance between the feature maps of students and teachers;

[0139]

[0140] in, It is the middle feature of the front model on the end side. It is the intermediate feature of the middle model on the edge side;

[0141] The above scheme introduces an adaptive layer consisting of dotted convolutions (1×1 convolutional kernels) and batch normalization (BN) layers. This adaptive layer maps the channels of the student model to the corresponding channels of the teacher model, thereby transferring knowledge more efficiently and reducing the difference in feature mapping between the pruned model and the original model. This mapping strategy ensures that structural changes in the student model do not negatively impact the effectiveness of knowledge distillation.

[0142] The soft label loss is expressed as:

[0143]

[0144] Where x ij X represents the logical output of the student model for the j-th class in the i-th batch of samples; ij and Let T represent the soft output of the student model and teacher model A of the j-th class in the i-th batch of samples, respectively; the temperature parameter T determines the degree of softening of the output.

[0145] In the above scheme, the output soft label of the simulated distillation learning is used. In order to learn more from the teacher model, it is also necessary to simulate the softened teacher output. Specifically, the KL divergence loss between the student and teacher outputs is used as the distillation loss of the output simulation. Temperature T softens the output between the student and teacher. This enables the student model to learn the prediction results of the high-performing teacher model more effectively, thereby significantly reducing the classification error rate.

[0146] The cross-entropy loss with introduced hard labels is expressed as:

[0147]

[0148] in, Y represents the logical output of the j-th class of the i-th batch of samples; ij The hard label representing the j-th class of the i-th batch of samples;

[0149] The total loss of the multi-stage distillation fine-tuning is expressed as:

[0150] L=δl inter +εl output +θl CE

[0151] Where δ, ε, and θ represent the weights of the intermediate feature loss, soft label loss, and hard label loss, respectively, and δ+ε+θ=1.

[0152] Preferably, in step S6, deploying the lightweight front-end model to the client includes:

[0153] Combine the front model with the middle and rear models;

[0154] The combined model is deployed to the client to complete the inference task locally on the client.

[0155] In the local inference process on the client side, there is no need to transmit data to the edge side or the cloud, so as to reduce communication overhead and inference latency.

[0156] Preferably, the combination of the front model with the middle model and the rear model specifically involves:

[0157]

[0158] Among them, f ji It is a combination of the cloud model, the j-th edge model, and the j-th client model.

[0159] In the above solution, the carefully compressed and optimized model is deployed to the edge device, realizing the complete integration of the model. This deployment strategy enables subsequent inference tasks to be executed entirely locally on the client, without the need to upload data to the edge or cloud for processing. This significantly reduces the communication overhead caused by data transmission and also reduces the time latency during the inference process.

[0160] By performing inference on the edge, not only is the system's responsiveness improved, but user privacy is also enhanced, as sensitive data no longer needs to leave the user's device. Furthermore, this approach reduces the computational burden on the cloud and edge, allowing them to allocate resources to other tasks, thereby improving the overall system efficiency and scalability.

[0161] In practical applications, this localized inference strategy is particularly suitable for scenarios with high real-time requirements, such as autonomous driving, augmented reality, and instant speech recognition. In these scenarios, rapid decision-making and response are crucial, and any time delay caused by network latency or bandwidth limitations can affect user experience or system performance.

[0162] In summary, deploying the compressed model directly to the edge and completing all inference tasks locally not only optimizes end-to-end system performance but also provides users with a faster and more reliable service experience.

[0163] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for compression and deployment of edge-cloud models based on split learning, characterized in that, Includes the following steps: S1: The deep learning model to be compressed is split into a front-end model, a middle-end model, and a back-end model, which are deployed on the client, edge, and cloud respectively. S2: The client uses local data to perform forward propagation on the front-end model deployed on the client and sends the forward propagation results to the edge side; S3: The edge side receives the forward propagation results from at least one client, uses the middle model deployed on the edge side to propagate the received results forward, and sends the forward propagation results to the cloud; S4: The cloud receives the forward propagation results from at least one edge side, completes the forward propagation using the back-end model deployed in the cloud, calculates the loss function, and performs back propagation based on the loss function to update the parameters of the front-end model, middle-end model, and back-end model. S5: Iteratively prune the front-end model deployed on the client, the middle-end model deployed on the edge, and the back-end model deployed in the cloud, and fine-tune the pruned model using knowledge distillation; S6: Deploy the lightweight front-end model to the client for inference on the client's local machine; In step S1, the deep learning model is manually split into a front-end model, a middle-end model, and a back-end model based on the resource constraints of the client, the edge, and the cloud. In step S2, the client processes local data and performs forward propagation of the front model until a predefined cut layer is reached; the output of the client's forward propagation is represented as: ； in, It is the i-th front model. This is the input data from the i-th client device. These are the parameters of the i-th front model; In step S3, the edge side receives the intermediate results sent by all clients. And aggregate them to obtain And this is used as the input to the edge side to continue the forward propagation. The output of the forward propagation at the edge side is represented as: in It is the j-th middle model. These are the parameters of the j-th middle model; In step S4, the cloud receives forward propagation results from at least one edge side, and completes the forward propagation using a backend model deployed in the cloud, including: the cloud receiving intermediate results sent by the edge side. And aggregate them to obtain And complete the final forward propagation to obtain the model output. The output propagated forward from the cloud is represented as: ； in It is a cloud-based model. These are the parameters of the cloud-based model; In step S4, calculating the loss function and performing backpropagation based on the loss function to update the parameters of the front model, the middle model, and the back model includes: cloud-based Calculate the loss function with the true label y. The gradient of the loss function with respect to the model parameters is calculated in the cloud and passed to the edge side; The edge side updates its model portion based on the gradient and computes the gradient passed to the client; The client updates its model based on the gradient, thus completing the model update.

2. The edge-cloud model compression and deployment method based on split learning according to claim 1, characterized in that, In step S5, the fine-tuning of the pruned model using knowledge distillation technology includes: taking the unpruned model part as the teacher and the pruned model part as the student, and using multi-level knowledge distillation for fine-tuning. The multi-level distillation fine-tuning includes intermediate feature loss and soft label loss, and simultaneously introduces hard label cross-entropy loss to form the total loss of multi-level distillation fine-tuning, and trains the student model.

3. The edge-cloud model compression and deployment method based on split learning according to claim 2, characterized in that, The intermediate feature loss is expressed as: in, Represented as a feature map of the student model; Let r be the feature map of its corresponding teacher model A; r(·) is a regression variable consisting of a 1×1 convolutional layer and a BN layer; It is a measure of the L2 distance between the feature maps of students and teachers; in, It is the middle feature of the front model on the end side. It is the intermediate feature of the middle model on the edge side; The soft label loss is expressed as: in This represents the logical output of the student model for the j-th class in the i-th batch of samples. and Let T represent the soft output of the student model and teacher model A of the j-th class in the i-th batch of samples, respectively; the temperature parameter T determines the degree of softening of the output. The cross-entropy loss with introduced hard labels is expressed as: in, This represents the logical output of the j-th class of the i-th batch of samples; The hard label representing the j-th class of the i-th batch of samples; The total loss of the multi-stage distillation fine-tuning is expressed as: in, , and , respectively, represent the weight values ​​of the intermediate feature loss, soft label loss, and hard label loss, and .

4. The edge-cloud model compression and deployment method based on split learning according to claim 3, characterized in that, In step S6, deploying the lightweight front-end model to the client includes: Combine the front model with the middle and rear models; The combined model is deployed to the client to complete the inference task locally on the client. In the local inference process on the client side, there is no need to transmit data to the edge side or the cloud, so as to reduce communication overhead and inference latency.

5. The edge-cloud model compression and deployment method based on split learning according to claim 4, characterized in that, The specific steps of combining the front model with the middle and rear models are as follows: + ； in, It is a combination of the cloud model, the j-th edge model, and the j-th client model.

Images