Large model cloud edge-end cooperative training framework and method based on split learning

Abstract

The invention relates to the technical field of large model training, and provides a training framework of a cloud side end, which improves the ability of providing personalized services for users under the condition of ensuring that generalization of pre-training LLM is not affected, and adopts a split learning mode to be well integrated with distributed deployment of a large model. According to the framework, an LLM integrated with a fine adjustment module is deployed on a cloud server, and meanwhile, a deep neural network with a relatively small scale is deployed on each terminal device and a corresponding edge server in a split learning mode. The deep neural network and the LLM are subjected to cooperative training and reasoning in a serial mode; a mode of optimizing a split layer decision and resource scheduling is adopted, a split layer decision optimization algorithm is used on a large time scale, a joint calculation and spectrum resource allocation optimization algorithm is used on a small time scale, and the training delay of cloud side end cooperative training is reduced from two dimensions.

Description

Technical Field

[0001] The present invention relates to the technical field of large model training, and particularly relates to a large model cloud-edge-end collaborative training method based on split learning. Background Art

[0002] Currently, large language models (LLMs) are considered the core technology for realizing an intelligence-embedded network in future 6G. Due to the huge number of parameters of LLMs, the computational complexity and memory overhead for training LLMs are extremely high, and the pre-training process can only be carried out on high-performance cloud servers located in data centers.

[0003] Generally, pre-trained LLMs obtained through centralized training on public datasets have excellent generalization capabilities but lack personalization capabilities. However, this centralized training mode cannot effectively utilize the large amount of valuable private data and computing power distributed on a huge number of terminal devices, enabling large models to continuously evolve. Classic distributed learning frameworks, such as federated learning, are not applicable to training LLMs because the computing resources and communication resources of terminal devices are limited. Due to the extremely large number of parameters of LLMs, their transmission, storage, and computing overheads are unacceptable to terminal devices.

[0004] Parameter-efficient fine-tuning (PEFT) is widely used for personalizing pre-trained LLMs. The basic idea of PEFT is to update a small number of parameters during the training process while freezing other parameters. PEFT is essentially a centralized training paradigm, that is, the fine-tuning process is only executed on one device. Before executing PEFT, the device must save all the parameters of the LLM. And LLMs usually have a large number of parameters, so it is infeasible for devices with limited storage resources to save the complete LLM. In addition, PEFT is still unaffordable for resource-constrained terminal devices. Summary of the Invention

[0005] In view of the above problems, the present invention provides a large model cloud-edge-end collaborative training framework and method based on split learning.

[0006] The technical solution adopted is a large model cloud-edge-end collaborative training framework based on split learning, including a cloud server, an edge server, and a terminal device;

[0007] A large language model integrated with a fine-tuning adapter module is deployed on the cloud server;

[0008] A deep neural network is deployed on the edge server;

[0009] The terminal device is located within the coverage area of the edge server, and a deep neural network is deployed on the terminal device;

[0010] The deep neural network is deployed on the edge server and the terminal device in a split learning manner, and the split position is determined based on the available resources of the terminal device.

[0011] Furthermore, the cloud-edge-terminal collaborative training framework for large models based on split learning includes forward propagation and backward propagation;

[0012] In the forward propagation, the terminal device inputs a batch of sampled training data into the device-side deep neural network model. After forward propagation, the device-side deep neural network model outputs shredded data. The terminal device transmits the shredded data together with the corresponding labels to the edge server. The edge server inputs the shredded data into the server-side deep neural network model. The edge server obtains the output intermediate data and uploads the intermediate data to the cloud server. The cloud server performs forward propagation on the large model to obtain the predicted labels. The edge server calculates the loss value based on the prediction and the true labels. In the forward propagation, the deep neural network deployed on the terminal device and the edge server in the form of split learning completes the initial feature extraction and / or data processing, and the large language model deployed on the cloud server completes the remaining calculation steps;

[0013] In the backward propagation, the cloud server first performs backward propagation to update the parameters of the fine-tuning module of the large model. The cloud server then transmits the loss function value to the edge server. The edge server performs backward propagation to update the server-side neural network model. Then the edge server sends the gradients of the split layer to the terminal device. The terminal device continues backward propagation to update the device-side deep neural network model. In the backward propagation, the large language model and the deep neural network calculate gradients and update parameters in parallel.

[0014] Furthermore, the available computing resources and memory resources of the edge server are denoted as C e and M e ;

[0015] The set of terminal devices is denoted as U = {u1, u2, …, u N}, N = |U| is the number of terminal devices, and the available computing power and memory of the terminal device are respectively represented as C n and M n ;

[0016] The communication link between the cloud server and the edge server is a wired optical fiber link, and the communication link between the edge server and all terminal devices it serves is a wireless link. The total available bandwidth is denoted as B e , the channel between the terminal device and the edge server is a frequency division multiple access channel. The spectrum with a bandwidth of B e is evenly divided into K subcarriers, and the bandwidth of each subcarrier is B s = Be / K. The subcarrier allocation is dynamically scheduled by the edge server;

[0017] Denote the number of splittable layers of the deep neural network as L, and the set of feasible split layers is denoted as L = {1, 2, …, L}. And for any l ∈ L, the parameters of layer l are denoted as w l , the input dimension of layer l is I l , and the output dimension is O l , w l is represented in size as z l , in bits, the size of w l is defined from the perspective of parameter transmission;

[0018] The w l The forward propagation computational cost on a data sample is denoted as h l , in CPU cycles, the size of w l is defined from the perspective of computation;

[0019] The hierarchical decision variable of the terminal device u n is denoted as s n , where s n ∈ L, and s n = l means splitting the deep neural network model of layer l into two parts, and the parameter matrix of the first l layers is deployed on the terminal device u n , and the last L - l layers are deployed on the edge server;

[0020] The parameter matrix sets saved on the terminal device u n and the edge server are respectively denoted as and The parameter matrix set The parameter size is denoted as There are l1, l2 ∈ L and l1 ≤ l2;

[0021] In and The forward propagation computational costs are respectively denoted as and

[0022] For The terminal device u n Needs to occupy the memory and computing resources of the edge server to save and train The proportion of computing resources allocated by the edge server to the terminal device u n is α n , and α n ∈ (0, 1). At the same time, in order to transmit fragmented data and gradient data between u n and the edge server, the edge server will β nSubcarriers are allocated to u n , where β n ∈{0, 1, 2, …, K}.

[0023] This application also provides a split - learning - based large - model cloud - edge - terminal collaborative training method implemented by a split - learning - based large - model cloud - edge - terminal collaborative training framework. The training method includes T rounds, each round is divided into Q generations, and each generation contains J iterations;

[0024] At the beginning of each round, the edge server determines the split layer set s = {s1, s2, …, s N} and, after the split layer is updated, the edge server or the terminal device transmits the parameter matrix using the incremental transmission method;

[0025] The terminal device u n The split layer in the t - th round is When , the terminal device u n uploads the parameter matrix w3 to the edge server, and when , the edge server needs to send the parameter matrix w4 to the terminal device u n ;

[0026] At the beginning of each iteration, the edge server updates the joint allocation decision of computing resources and communication resources, that is, α = {α1, α2, …, α N}, β = {β1, β2, …, β N}.

[0027] Furthermore, in the j - th iteration of the q - th generation in the t - th training round, the forward propagation includes the following steps:

[0028] The terminal device u n samples a batch of training data from the local dataset D n , and the batch size is denoted as b;

[0029] The sampled training data and the corresponding labels are respectively denoted as x n and y n ;

[0030] The terminal device u n inputs the training data into the device - side deep neural network deployed locally and obtains the shredded data v through the following formula n :

[0031]

[0032] where f(x; w) represents the mapping function between the input x and the given parameter matrix w;

[0033] The terminal device u n through the allocated βn The subcarriers will crush the data v n , the training data x n and the corresponding label y n and upload them to the edge server;

[0034] The edge server uses the received crushed data to perform forward propagation on the remaining layer server-side deep neural network to obtain the intermediate data m through the following formula n :

[0035]

[0036] The edge server uploads all the intermediate data {m1, m2, …, m N}, the original data {x1, x2, …, x N} and the corresponding labels {y1, y2, …, y N} to the cloud server;

[0037] For the cloud server inputs the intermediate data m n into the large language model to obtain the prediction result through the following formula

[0038]

[0039] where w LLM (t, j) is the parameter matrix of the large language model with an adapter layer in the j-th iteration of the q-th generation in the t-th training round.

[0040] Furthermore, in the j-th iteration of the q-th generation in the t-th training round, the backpropagation includes the following steps:

[0041] The cloud server calculates the loss value based on the prediction result and the true label as:

[0042]

[0043] where f loss is the loss function;

[0044] The cloud server performs backpropagation on the large language model to update the parameter matrix of the adapter layer through the following formula

[0045]

[0046] where η c is the learning rate for parameter-efficient fine-tuning on the large language model;

[0047] The cloud server uses the large language model to calculate the inference result based on the original data as:

[0048]

[0049] And calculate the loss function as:

[0050]

[0051] The cloud server transmits Loss n = L' n - L n to the edge server, and the edge server uses this value as the loss value for training the deep neural network to update the parameters of the deep neural network:

[0052]

[0053] Perform backpropagation on the locally deployed L-s n layer neural network, where η e is the learning rate of the deep neural network deployed on the edge server and the terminal device;

[0054] The edge server transmits the gradient vector of the s n th layer to u n , and performs backpropagation on the local s n th layer deep neural network of the terminal device u n :

[0055]

[0056] After the backpropagation process, the cloud server updates the parameter matrix of the adapter layer embedded in the large language model, and the edge server and the terminal device update the parameters of the deep neural network.

[0057] Furthermore, the latency in one iteration includes the computing latency on the terminal device, the transmission latency between the terminal device and the edge server, the computing latency on the edge server, the transmission latency between the edge server and the cloud server, and the computing latency on the cloud server;

[0058] Regarding the computing latency on the terminal device, On the terminal device u n the total computing latency is composed of the forward propagation and the backward propagation on the set of parameter matrices , and the local computing latency of the terminal device u n is:

[0059]

[0060] where ò represents the ratio of the computational complexity of the backward propagation to the forward propagation on the same parameter matrix, C n(t, q) is the computing resource of the terminal device in the q-th generation of the t-th round. It is assumed that the computing resource of the terminal device remains unchanged within one generation;

[0061] For the transmission delay between the terminal device and the edge server, terminal device u n The transmission delay between it and the edge server consists of two parts. In the forward propagation, terminal device u n needs to transmit the shredded data to the edge server. In the backward propagation, the edge server needs to transmit the gradient vector to terminal device u n ;

[0062] The transmission rate from terminal device u n to the edge server is:

[0063]

[0064] where G n is the channel gain, P d is the transmission power of the terminal device. It is stipulated that the transmission powers of all devices are the same, and N0 is the power spectral density of Gaussian white noise;

[0065] The transmission rate from the edge server to terminal device u n is:

[0066]

[0067] where P e is the transmission power of the edge server;

[0068] The transmission delay between terminal device u n and the edge server is:

[0069]

[0070] where is the size of the shredded data, training data, and the corresponding labels, is the size of the gradient vector;

[0071] For the computing delay on the edge server, in the j-th iteration of the q-th generation of the t-th round, the computing resource allocated by the edge server to terminal device u n is The computing delay on the edge server is

[0072]

[0073] For the transmission delay between the edge server and the cloud server, in the forward propagation, after the edge server completes the forward calculation of N server-side deep neural networks, the edge server transmits the intermediate data, training data, and corresponding labels to the cloud server;

[0074] In the backward propagation, the cloud server transmits N gradient vectors to the edge server;

[0075] The transmission rate from the edge server to the cloud server is R c,e , so the transmission delay between the edge server and the cloud server is:

[0076]

[0077] where is the size of N groups of intermediate data, training data, and corresponding labels, is the size of N gradient vectors;

[0078] For the computing delay on the cloud server, since the cloud server has extremely strong computing power, this part of the delay can be ignored.

[0079] The delay of the j-th iteration of the q-th generation in the t-th round is D(t, q, j), and there is:

[0080]

[0081] Therefore, the training delay of the q-th generation in the t-th round is:

[0082]

[0083] At the beginning of the training round, the terminal device or the edge server may need to transmit part of the parameter matrix, and the delay of parameter matrix transmission is:

[0084]

[0085] The overall delay of the t-th round is:

[0086]

[0087] Furthermore, the split layer decision set of the terminal device is denoted as S = {s1, s2,..., s N};

[0088] The set of computing resource allocation variables is denoted as α = {α1, α2,..., α N};

[0089] The set of spectrum resource allocations is denoted as β = {β1, β2,..., β N};

[0090] In one round, the splitting layer decision s is optimized;

[0091] In one iteration, the allocation of computing resources α and spectrum resources β is optimized.

[0092] Furthermore, in optimizing the splitting layer decision s, there is:

[0093]

[0094] where E C,G [D(s, α, β)] represents the mathematical expectation of the training delay per round, and C and G respectively represent the computing resources of N devices and the channel gains between the edge servers, both of which are random vectors.

[0095] Furthermore, in optimizing the allocation of computing resources α and spectrum resources β, there is:

[0096]

[0097] where V is an auxiliary variable representing the maximum delay among N devices, and there is

[0098] The beneficial effects of the present invention are as follows;

[0099] 1. A training framework for the cloud-edge-terminal is provided, which improves its ability to provide personalized services to users while ensuring that the generalization of the pre-trained LLM is not affected.

[0100] 2. By adopting the split learning method, it can be well integrated with the distributed deployment of large models. The framework deploys the LLM with an integrated fine-tuning module on the cloud server, and at the same time deploys relatively small deep neural networks on each terminal device and its corresponding edge server in the form of split learning. The deep neural networks and the LLM cooperate in training and inference in a serial manner;

[0101] 3. By adopting the method of optimizing the split layer decision and resource scheduling, the split layer decision optimization algorithm is used on a large time scale, and the joint computing and spectrum resource allocation optimization algorithm is used on a small time scale, reducing the training delay of cloud-edge-terminal collaborative training from two dimensions. Description of the Drawings

[0102] Figure 1 It is a framework diagram for cloud-edge-terminal collaborative training / inference;

[0103] Figure 2 It is a timing diagram for split decision and resource allocation;

[0104] Figure 3 It is a diagram for split point update and incremental transmission parameter matrix;

[0105] Figure 4 It is a diagram of the collaborative training mechanism;

[0106] Figure 5 It is a diagram of the algorithm convergence;

[0107] Figure 6 It is a diagram of the average latency for each iteration;

[0108] Figure 7 It is a diagram of the accuracy. Specific implementation manners

[0109] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0110] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0111] Embodiment 1

[0112] In this embodiment, as Figure 1 shown, a large model cloud-edge-end collaborative training framework based on split learning is provided, including a cloud server, an edge server, and a terminal device;

[0113] A large language model integrated with a fine-tuning adapter module is deployed on the cloud server;

[0114] A deep neural network is deployed on the edge server;

[0115] The terminal device is located within the coverage area of the edge server, and a deep neural network is deployed on the terminal device;

[0116] The deep neural network is deployed on the edge server and the terminal device in a split learning manner, and the split position is determined based on the available resources of the terminal device.

[0117] It should be noted that in actual implementation, an LLM is deployed on a cloud server, while a relatively small-scale deep neural network (DNN) model is deployed on edge servers and terminal devices in a split learning manner. The position of the split layer is determined according to the available resources of the terminal device.

[0118] The available computing and memory resources of the cloud server are sufficient to fine-tune the LLM because the computing resources required by PEFT are much smaller than those required for pre-training. The available computing resources and memory resources of the edge server are denoted as C e and M e . The set of terminal devices is denoted as U = {u1, u2, …, u N}, and N = |U| is the number of terminal devices. For its available computing power and memory are respectively expressed as C n and M n . The communication link between the cloud server and the edge server is a wired fiber optic link, and the communication links between the edge server and all terminal devices of the service are wireless links. The total available bandwidth is denoted as B e . It is assumed that the channel between the terminal device and the edge server is a frequency division multiple access (FDMA) channel. The spectrum with a bandwidth of B e is evenly divided into K subcarriers, and the bandwidth of each subcarrier is B s = B e / K. The allocation of subcarriers is dynamically scheduled by the edge server.

[0119] The number of split layers of the DNN is denoted as L, and the set of feasible split layers is denoted as L = {1, 2, …, L}. For any l ∈ L, the parameters of layer l are denoted as w l , the input dimension of layer l is I l , and the output dimension is O l . The size of w l is denoted as z l , in bits, which defines the size of w l from the perspective of parameter transmission. The forward propagation (FP) computational overhead of w l on a data sample is denoted as h l , in CPU cycles, which defines the size of w l from the perspective of computation. The layer splitting decision variable of terminal device u n is denoted as s n , where s n ∈ L. For example, s n = l means splitting the DNN model of layer l into two parts, with the parameter matrix of the first l layers deployed on terminal device u n , and the last L - l layers deployed on the edge server. Saved on u nThe parameter matrix sets on the edge server are respectively denoted as and parameter matrix set The parameter size of is denoted as There are l1, l2 ∈ L and l1 ≤ l2. In and The computational overheads of forward propagation on are respectively denoted as and

[0120] For u n It is necessary to occupy a certain proportion of the memory and computing resources of the edge server to save and train The computing resources allocated by the edge server to u n The proportion is α n , there is α n ∈(0,1). At the same time, in order to transmit shredded data and gradient data between u n and the edge server, it is assumed that the edge server allocates β n subcarriers to u n , where β n ∈{0,1,2,…,K}.

[0121] At the same time, in this three-layer network, the LLM is deployed on the cloud server, and the DNN is deployed on the edge server and the terminal device in the split learning mode. As Figure 1 shown, the LLM and the DNN cooperate to execute the training process in a serial manner. Specifically, in the forward propagation process, the DNN completes initial feature extraction / data processing and other tasks, and the remaining computing steps are completed by the LLM. The calculation order of the backpropagation (BP) process is opposite to that of the forward propagation, and the processes of the DNN calculating gradients and updating parameters are parallel.

[0122] During the collaborative training process, the LLM performs parameter-efficient fine-tuning by inserting a serial adapter layer. It should be noted that in order to enhance the personalized ability of the LLM without affecting the generalization of the pre-trained LLM, the cloud server maintains a set of adapter layers for each task it serves. That is, when an edge server requests the LLM service corresponding to a certain task from the cloud server, the cloud server inserts the set of adapter layers corresponding to the task into the LLM and provides the service using the LLM with the embedded adapter layer. Different from the PEFT used when the LLM deployed on the cloud server is trained, the DNN model deployed on the edge server and the terminal device adopts the full-parameter update training mode. This is because the number of parameters in the DNN is much smaller than that in the LLM, and the computing power of the edge server and the edge device can fully afford the training of the DNN.

[0123] The entire training process consists of T rounds, each round is divided into Q generations, and each generation contains J iterations. As Figure 2 shown, on the time scale, one round is relatively large, while one iteration is relatively small. Considering the dynamics of the network, to improve the training efficiency, it is necessary to periodically update the split layer and resource allocation decisions. To reduce the communication overhead, the split layer decision is updated within a larger time range (one round) and the resource allocation decision is updated within a smaller time range (one iteration). At the beginning of each round, the edge server needs to determine the split layer set s = {s1, s2, …, s N}. To reduce the communication overhead, after the split layer is updated, the edge server or the terminal device uses the incremental transmission method to transmit the parameter matrix. Taking Figure 3 as an example, u n 's split layer in the t-th round is If u n needs to upload the parameter matrix w3 to the edge server, if the edge server needs to send the parameter matrix w4 to the device u n . At the beginning of each iteration, the edge server needs to update the joint allocation decision of computing resources and communication resources, that is, α = {α1, α2, …, α N} and β = {β1, β2, …, β N}.

[0124] As Figure 4 shown, in the j-th iteration of the t-th training round, the process of cloud-edge collaborative training can be divided into two parts: forward propagation and backward propagation, as follows:

[0125] Forward propagation: For u n first samples a batch of training data from its local dataset D n , and the batch size is denoted as b. The sampled training data and the corresponding labels are denoted as x n and y n . Then u n inputs the training data into the device-side deep neural network deployed locally, and through

[0126]

[0127] obtains the shredded data v n , where f(x; w) represents the mapping function between the input x and the given parameter matrix w. Then, u n transmits the shredded data v n , the training data x n and the corresponding label y n through the allocated β nUpload to the edge server. The edge server uses the received shredded data to perform forward propagation on the remaining layer server-side neural network, and obtains intermediate data m through

[0128]

[0129] Then, the edge server uploads all intermediate data {m1, m2, …, m n}, original data {x1, x2, …, x N}, and corresponding labels {y1, y2, …, y N} to the cloud server. Finally, for N The cloud server inputs the intermediate data m into the LLM, and obtains the prediction result through n where w

[0130]

[0131] is the parameter matrix of the large language model with an adapter layer in the j-th iteration of the q-th generation in the t-th training round. Backward propagation: The cloud server calculates the loss value based on the prediction result and the true label as LLM where f

[0132]

[0133]

[0134] Figure 4 is the loss function. As loss shown, the cloud server performs backpropagation on the LLM, and updates the parameter matrix of the adapter layer through

[0135]

[0136] where η

[0137] c is the learning rate of PEFT on the LLM.

[0138] Then the cloud server calculates the inference result based on the original data for the LLM as

[0139]

[0140] and calculates the loss function as

[0141] The cloud server sets Loss n = L′ n - L n ​​Transmitted to the edge server, and the edge server updates the parameters of the DNN using this value as the loss value for DNN training:

[0142]

[0143] Perform backpropagation on the locally deployed L-s n layer neural network, where η e is the learning rate of the DNN. Then the edge server transmits the gradient vector of the s n th layer to u n , and perform backpropagation on the local s n th layer DNN of u n :

[0144]

[0145] After the backpropagation process, the cloud server updates the parameter matrix of the adapter layer embedded in the LLM, and the edge server and the terminal device update the parameters of the DNN.

[0146] The latency in one iteration includes the computing latency on the terminal device, the transmission latency between the terminal device and the edge server, the computing latency on the edge server, the transmission latency between the edge server and the cloud server, and the computing latency on the cloud server;

[0147] For the computing latency on the terminal device, the total computing latency on the terminal device u n consists of the forward propagation and backward propagation on the set of parameter matrices , and the local computing latency of the terminal device u n is:

[0148]

[0149] where ò represents the ratio of the computational complexity of backpropagation to that of forward propagation on the same parameter matrix, and C n (t, q) is the computing resource of the terminal device in the qth generation of the tth round. It is assumed that the computing resource of the terminal device remains unchanged within one generation;

[0150] For the transmission latency between the terminal device and the edge server, the transmission latency between the terminal device u n and the edge server consists of two parts. In the forward propagation, the terminal device u n needs to transmit the shredded data to the edge server, and in the backward propagation, the edge server needs to transmit the gradient vector to the terminal device u n ;

[0151] From the terminal device u nThe transmission rate to the edge server is:

[0152]

[0153] where G n is the channel gain, P d is the transmission power of the terminal device, and it is stipulated that the transmission powers of all devices are the same. N0 is the power spectral density of Gaussian white noise;

[0154] The transmission rate from the edge server to the terminal device u n is:

[0155]

[0156] where P e is the transmission power of the edge server;

[0157] The transmission delay between the terminal device u n and the edge server is:

[0158]

[0159] where is the size of the shredded data, training data, and corresponding labels, is the size of the gradient vector;

[0160] Regarding the computing delay on the edge server,

[0161] In the j-th iteration of the q-th generation in the t-th round, the computing resources allocated by the edge server to the terminal device u n are The computing delay on the edge server is

[0162]

[0163] Regarding the transmission delay between the edge server and the cloud server, in the forward propagation, after the edge server completes the forward calculation of N server-side deep neural networks, the edge server transmits the intermediate data, training data, and corresponding labels to the cloud server;

[0164] In the backpropagation, the cloud server transmits N gradient vectors to the edge server;

[0165] The transmission rate from the edge server to the cloud server is R c,e , so the transmission delay between the edge server and the cloud server is:

[0166]

[0167] where is the size of N groups of intermediate data, training data, and corresponding labels, is the size of N gradient vectors;

[0168] Regarding the computing latency on the cloud server, since the cloud server has extremely strong computing power, this part of the latency can be ignored.

[0169] Therefore, the training latency of the q-th generation in the t-th round is:

[0170]

[0171] At the beginning of the training round, the terminal device or the edge server may need to transmit part of the parameter matrix, and the latency of parameter matrix transmission is:

[0172]

[0173] The overall latency of the t-th round is:

[0174]

[0175] To reduce the training latency of cloud-edge device collaborative training, it is necessary to optimize the split layer decision and resource scheduling. The set of split layer decisions for all devices is denoted as S = {s1, s2,..., s N}. The set of computing resource allocation variables is denoted as α = {α1, α2,..., α N}. The set of spectrum resource allocations is denoted as β = {β1, β2,..., β N}. To improve the training efficiency, the following optimization problem is modeled:

[0176]

[0177] Problem P1 is a mixed-integer non-linear programming problem. The optimization objective is to minimize the training latency. Constraints 1 and 2 represent the computing resource limitations of the edge server. Constraints 3 and 4 solve the spectrum resource constraints between the edge server and the terminal device. Constraint 5 defines the feasible split layer constraint. Constraint 6 involves the memory constraint of the terminal device, and Constraint 7 considers the memory constraint of the edge server. Considering the dynamics of the available computing resources of the devices and the wireless channels, the split layer decision s, the computing resource allocation α, and the spectrum resource allocation β need to be updated regularly. P1 is actually a two-time-scale optimization problem because s is updated at the beginning of each round of training, while α and β are updated at the beginning of each generation. Therefore, a two-time-scale optimization algorithm is designed in the next section.

[0178] For the optimization problem P1, a two - time - scale algorithm is designed. On the large time scale, i.e., in one round, the split - layer decision s is optimized. On the small time scale, i.e., in one generation, the allocations of computing resources α and spectrum resources β are jointly optimized.

[0179] In the large - scale scenario, since the update of the split layer involves transmitting the model parameter matrix, this will result in a large communication overhead. The available resources of the terminal device and the edge server do not fluctuate significantly in the short term. Therefore, it is reasonable and feasible to perform large - scale updates on the split - layer selection. The split - layer selection sub - problem can be expressed as:

[0180]

[0181] The optimization objective of this sub - problem can be approximated as:

[0182]

[0183] where, E C,G [D(s,α,β)] represents the mathematical expectation of the training delay per round. C and G represent the computing resources of N devices and the channel gain between the edge server respectively, both of which are random vectors (due to the dynamics of the network).

[0184] According to the law of large numbers, the mathematical expectation can be approximated by the sample average. Therefore, a sampling - based average approximation method is adopted to extract a batch of computing resources and channel state data of edge devices from the historical records. According to these data, the corresponding resource allocation decisions are calculated. For a specified split - layer selection scheme, the approximate average delay can be derived:

[0185]

[0186] For this complex combinatorial optimization problem, heuristic algorithms provide a solution with low complexity and stable performance. The Adaptive Large Neighborhood Search (ALNS) algorithm can be used to solve this problem. Algorithm 1 shows the split - layer selection algorithm.

[0187]

[0188]

[0189] [[ID=�5]]

[0190] In the small - scale scenario, during each training iteration, the allocations of computing resources and spectrum resources need to be jointly adjusted according to network changes. For this joint optimization problem, it is further decomposed into a spectrum resource allocation sub - problem and a computing resource allocation sub - problem. Then, by iteratively solving these two sub - problems, the optimal joint resource allocation scheme is obtained.

[0191] Under a given computing resource allocation scheme, the sub-problem of spectrum resource allocation can be written in the following form:

[0192]

[0193] Since the optimization variable β is an integer, this problem is a simple combinatorial optimization problem and can be solved using a low-complexity greedy algorithm. The idea is to first set the number of subcarriers allocated to N devices to 1, and then allocate the remaining K - N subcarriers one by one according to the greedy idea. After a given spectrum resource allocation scheme, the sub-problem of computing resource allocation can be written in the following form:

[0194]

[0195] To simplify this min-max problem, an auxiliary variable V is introduced to represent the maximum latency among N devices. Therefore, it can be known that Then the above sub-problem can be transformed into the following form:

[0196] [[ID=2,0]]

[0197] The Lagrangian function can be written using the method of Lagrange multipliers as follows:

[0198]

[0199] From this, the KKT conditions can be written, and then the Newton method is used to calculate the search / iteration direction, and the feasible solution is iterated multiple times. Finally, the optimal solution is obtained.

[0200] The joint optimization algorithm for jointly computing resources and spectrum resource allocation is as follows:

[0201]

[0202]

[0203] Embodiment 2

[0204] In this embodiment, the cloud-edge-end collaborative training framework is composed of BART-Large (400MB of memory required for INT8 precision, 800MB of video memory required for half-precision training) and Llama2-70B with the PEFT module inserted (70GB of memory required for INT8 precision, 140GB of video memory required for half-precision training) in series. BART-Large is mainly responsible for enriching / retouching the input text according to the user's personalized characteristics, and Llama2-70B then performs reasoning based on the retouched text to generate a reply.

[0205] The dataset used in the simulation is MultiWOZ 2.2, which is a multi-domain multi-turn dialogue dataset suitable for personalized training.

[0206]

[0207] In this experiment, the following four algorithms are compared. Algorithms 1, 2, and 3 are all based on the cloud-edge-end three-layer deployment framework proposed in this paper, but they differ in the split point decision and resource allocation. Algorithm 1 is the dual-scale optimization algorithm proposed in this paper. Algorithm 2 is based on Algorithm 1 and sets the resource allocation scheme to equal distribution. Algorithm 3 is based on Algorithm 1 and sets the split point decision scheme to unified configuration, that is, the split points of all edge devices are the same, and the splitting is performed according to the minimum value of the maximum feasible split points of all devices. Algorithm 4 is the most common large model partial architecture currently, that is, it is only deployed on the cloud server.

[0208] Algorithm 1 (SCCT-T): Dual-scale optimization algorithm, optimizing the split point at a large time scale and optimizing the computing power and communication resource configuration at a small time scale;

[0209] Algorithm 2 (SCCT-S): Dynamic optimization of split points + equal distribution of resources;

[0210] Algorithm 3 (SCCT-R): Unified configuration of split points + dynamic allocation of resources;

[0211] Algorithm 4 (Cloud-O): Only fine-tune Llama on the cloud, that is, the cloud service mode.

[0212] In this embodiment, taking perplexity (PPL) as an index, the convergence speeds of the four algorithms under the same learning rate and dataset are compared, and the numerical results are as Figure 5 shown. According to Figure 5 , it can be found that the proposed SCCT-T algorithm has the fastest convergence speed, followed by SCCT-S, then SCCT-R, and Cloud-O is the slowest. This is because the SCCT architecture has an additional BART module for preprocessing compared to the architecture that only deploys the large model on the cloud, effectively enhancing the personalization ability. And the dual-scale optimization algorithm in SCCT-T optimizes both the split point and resource allocation compared to SCCT-S and SCCT-R, enabling more edge devices to participate in the fine-tuning process, improving the fine-tuning efficiency, and accelerating the algorithm convergence.

[0213] In this embodiment, the latency of the four algorithms in each iteration under different numbers of edge devices is studied, and the numerical results are as Figure 6 shown. According to Figure 6, it can be found that the more edge devices there are, the higher the average latency per iteration. This is because the more devices there are, the fewer resources allocated to each device, resulting in an increase in latency. In each group of bar charts, it can be found that Cloud-O has the lowest latency, because this algorithm has no edge devices and edge servers participating in the calculation, and only the cloud server participates in the fine-tuning process. The latency of SCCT-T is the second, lower than that of SCCT-S and SCCT-R, because SCCT-T has the highest resource utilization efficiency.

[0214] In this embodiment, taking EM (Exact Match) as an indicator, the changes in the test accuracy of the four algorithms during the training process are studied, and the numerical results are as Figure 7 shown. According to Figure 7 , it can be found that as the number of training rounds increases, the test accuracy gradually increases. The SCCT-T algorithm has the highest accuracy, followed by SCCT-S, then SCCT-R, and Cloud-O is the lowest. This is because the SCCT architecture has advantages over the cloud deployment architecture, and the dual-scale optimization algorithm of SCCT-T has better flexibility and higher resource utilization efficiency than SCCT-S and SCCT-R.

[0215] In this way, by adopting the split learning method, it can be well integrated with the distributed deployment of large models. It divides the large neural network model into multiple parts at specific split points, and then deploys different parts on different devices to cooperate in completing training and inference; by optimizing the split layer decision-making and resource scheduling, it uses the split layer decision-making algorithm for large-scale scenarios and joint calculation and spectrum resource allocation for small-scale scenarios, reducing the training latency of cloud-edge device collaborative training from two dimensions.

[0216] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large model cloud-edge-terminal collaborative training framework based on split learning, characterized in that, It includes a cloud server, an edge server, and a terminal device; A large language model integrated with a fine-tuning adapter module is deployed on the cloud server; A deep neural network is deployed on the edge server; The terminal device is located within the coverage area of the edge server, and a deep neural network is deployed on the terminal device; The deep neural network is deployed on the edge server and the terminal device in a split learning manner, and the splitting location is determined based on the available resources of the terminal device.

2. The collaborative training framework for large models at the cloud, edge, and terminal based on split learning according to claim 1, wherein The large model cloud-edge-end collaborative training framework based on split learning includes forward propagation and backward propagation; In the forward propagation, the terminal device inputs a batch of sampled training data into the device-side deep neural network model. After forward propagation, the device-side deep neural network model outputs shredded data. The terminal device transmits the shredded data together with the corresponding labels to the edge server. The edge server inputs the shredded data into the server-side deep neural network model. The edge server obtains the output intermediate data and uploads the intermediate data to the cloud server. The cloud server performs forward propagation on the large model to obtain predicted labels. The edge server calculates the loss value based on the prediction and the true labels. In the forward propagation, the deep neural network deployed on the terminal device and the edge server in the form of split learning completes initial feature extraction and / or data processing, and the large language model deployed on the cloud server completes the remaining calculation steps; In the backward propagation, the cloud server first performs backward propagation to update the parameters of the fine-tuning module of the large model. The cloud server then transmits the loss function value to the edge server. The edge server performs backward propagation to update the server-side neural network model. Then the edge server sends the gradients of the split layer to the terminal device. The terminal device continues backward propagation to update the device-side deep neural network model. In the backward propagation, the large language model and the deep neural network calculate gradients and update parameters in parallel.

3. The collaborative training framework for large models at the cloud, edge, and terminal based on split learning according to claim 2, wherein The available computing resources and memory resources of the edge server are denoted as C e and M e ; The set of the terminal devices is denoted as U = {u1, u2, …, u N}, N = |U| is the number of the terminal devices, and the available computing power and memory of the terminal devices are respectively represented as C n and M n ; The communication link between the cloud server and the edge server is a wired optical fiber link, and the communication link between the edge server and all terminal devices of the service is a wireless link. The total available bandwidth is denoted as B e , the channel between the terminal device and the edge server is a frequency division multiple access channel, and the spectrum with bandwidth B e is evenly divided into K subcarriers, and the bandwidth of each subcarrier is B s = B e / K. The allocation of subcarriers is dynamically scheduled by the edge server; Denote the number of detachable layers of the deep neural network as \(L\), and denote the set of feasible detachable layers as \(L=\{1,2,\cdots,L\}\). For any \(l\in L\), denote the parameters of layer \(l\) as \(w\). l , the input dimension of layer \(l\) is \(I\). l , the output dimension is \(O\). l , \(w\). l The size of is denoted as \(z\). l , from the perspective of parameter transmission, the size of \(w\) is defined in bits; l ​ The described w l The forward propagation calculation overhead on a data sample is denoted as h l , in CPU cycles, and the size of w is defined from the perspective of calculation overhead l ; The hierarchical decision variable of the terminal device u n is denoted as s n , where s n ∈L, and s n =l means splitting the depth neural network model of layer l into two parts, with the parameter matrix of the first l layers deployed on the terminal device u n and the last L-l layers deployed on the edge server; Stored in the terminal device u n The sets of parameter matrices stored on the edge server are respectively denoted as and The set of parameter matrices The parameter size of is denoted as There are l1, l2 ∈ L and l1 ≤ l2; Before and The computational overheads of forward propagation are respectively denoted as and For terminal device u n it is necessary to occupy the memory and computing resources of the edge server to save and train the computing resources allocated by the edge server to terminal device u n is α n , and there is α n ∈(0,1). At the same time, in order to transmit the shredded data and gradient data between u n and the edge server, the edge server allocates β n sub-carriers to u n , where β n ∈{0,1,2,…,K}.

4. A large model cloud-edge-terminal collaborative training method based on split learning, which is implemented based on the large model cloud-edge-terminal collaborative training framework described in claim 3, and is characterized in that The training method includes T rounds, each round is divided into Q generations, and each generation contains J iterations; At the beginning of each round, the edge server determines the split layer set s = {s1, s2, …, s N}. After the split layer is updated, the edge server or the terminal device uses the incremental transmission method to transmit the parameter matrix; Terminal device u n The split layer in the t-th round is When the terminal device u n uploads the parameter matrix w3 to the edge server. When the edge server needs to send the parameter matrix w4 to the terminal device u n ; At the beginning of each generation, the edge server updates the joint allocation decision of computing resources and communication resources, i.e., α = {α1, α2, …, α N}, β = {β1, β2, …, β N}.

5. A large model cloud-edge-end collaborative training method based on split learning according to claim 4, characterized in that, In the jth iteration of the qth generation of the tth training round, the forward propagation includes the following steps: Terminal device u n Sample a batch of training data from the local dataset D n with the batch size denoted as b; The sampled training data and the corresponding labels are denoted as x n and y n ; Terminal device u n Input the training data into the device-side deep neural network deployed locally, and obtain the shredded data v through the following formula n : where f(x; w) represents the mapping function between the input x and the given parameter matrix w; Terminal device u n Through the allocated β n sub - carriers will crush data v n , training data x n and the corresponding label y n and upload them to the edge server; The edge server uses the received shredded data to perform forward propagation on the remaining layer server-side deep neural network, and obtains the intermediate data m through the following formula n : The edge server then uploads all the intermediate data {m1, m2, …, m N}, the original data {x1, x2, …, x N}, and the corresponding labels {y1, y2, …, y N} to the cloud server; For The cloud server inputs the intermediate data m n into the large language model and obtains the prediction result through the following formula where w LLM (t, q, j) is the parameter matrix of the large language model with an adapter layer in the j-th iteration of the q-th generation in the t-th training round.

6. A large model cloud-edge-end collaborative training method based on split learning according to claim 4, characterized in that, In the jth iteration of the qth generation of the tth training round, the backward propagation includes the following steps: The cloud server calculates the loss value based on the prediction result and the true label as: where f loss is the loss function; The cloud server performs backpropagation on the large language model and updates the parameter matrix of the adapter layer through the following formula where η c is the learning rate for parameter-efficient fine-tuning on large language models; The cloud server calculates the inference result based on the original data using the large language model as: And calculates the loss function as: The cloud server transmits Loss n = L n ′ - L n to the edge server, and the edge server uses this value as the loss value for deep neural network training to update the parameters of the deep neural network: Perform backpropagation on the locally deployed L-s n layer neural network, where η e is the learning rate of the deep neural network deployed on the edge server and the terminal device; The edge server transmits the gradient vector of the s n -th layer to u n , and performs backpropagation on the local s n -th layer deep neural network of the terminal device u n : After the backward propagation process, the cloud server updates the parameter matrix of the adapter layer embedded in the large language model, and the edge server and the terminal device update the parameters of the deep neural network.

7. A large model cloud-edge-end collaborative training method based on split learning according to claim 5 or 6, characterized in that The latency in one iteration includes the computing latency on the terminal device, the transmission latency between the terminal device and the edge server, the computing latency on the edge server, the transmission latency between the edge server and the cloud server, and the computing latency on the cloud server; Regarding the computing latency on the terminal device, On the terminal device u n The total computing latency on it Consists of forward propagation and backward propagation on the parameter matrix set. The local computing latency of the terminal device u n Is: where ò represents the ratio of the computational complexity of backpropagation to that of forward propagation on the same parameter matrix, C n (t, q) is the computing resource of the terminal device in the q-th generation of the t-th round. It is assumed that the computing resource of the terminal device remains unchanged within one generation; Regarding the transmission delay between the terminal device and the edge server, the terminal device u n The transmission delay between it and the edge server consists of two parts. In the forward propagation, the terminal device u n needs to transmit the shredded data to the edge server. In the backward propagation, the edge server needs to transmit the gradient vector to the terminal device u n ; The transmission rate from the terminal device u n to the edge server is: where G n is the channel gain of the terminal device u n and P d is the transmission power of the terminal device. It is stipulated that the transmission powers of all devices are the same, and N0 is the power spectral density of Gaussian white noise; Transmission rate from the edge server to the terminal device u n is as follows: Among which P e is the transmission power of the edge server; Terminal device u n The transmission delay between it and the edge server is as follows: Among them is the size of the shredded data, training data, and corresponding labels, is the size of the gradient vector; For the computing latency on the edge server, in the j-th iteration of the q-th generation in the t-th round, the computing resources allocated by the edge server to the terminal device u n are The computing latency on the edge server is For the transmission delay between the edge server and the cloud server, in the forward propagation, after the edge server completes the forward computation of N server-side deep neural networks, the edge server transmits the intermediate data, training data, and corresponding labels to the cloud server; In the backward propagation, the cloud server transmits N gradient vectors to the edge server; The transmission rate from the edge server to the cloud server is R c,e , so the transmission delay between the edge server and the cloud server is: wherein is the size of N groups of intermediate data, training data, and corresponding labels, is the size of N gradient vectors; For the computation delay on the cloud server, since the cloud server has extremely strong computing power, this part of the delay can be ignored. The delay of the j-th iteration of the q-th generation in the t-th round is D(t,q,j), and there is: Therefore, the training delay of the q-th generation in the t-th round is: At the beginning of the training round, the terminal device or the edge server may need to transmit a partial parameter matrix, and the delay of parameter matrix transmission is as follows: The overall delay of the t-th round is:

8. A large model cloud-edge-end collaborative training method based on split learning according to claim 7, characterized in that, The disassembly layer decision set of the terminal device is denoted as S = {s1, s2, …, s N}; The set of computing resource allocation variables is denoted as α = {α1, α2, …, α N}; The set of spectrum resource allocations is denoted as β = {β1, β2, …, β N}; In one round, the splitting layer decision s is optimized; In one iteration, the allocation of computing resources α and spectrum resources β is optimized.

9. A large model cloud-edge-end collaborative training method based on split learning according to claim 8, characterized in that, In the optimization of the splitting layer decision s, there is: Among them, E C,G [D(s,α,β)] represents the mathematical expectation of the delay per round of training. C and G respectively represent the computing resources of N devices and the channel gains between the devices and the edge server, both of which are random vectors.

10. A large model cloud-edge-end collaborative training method based on split learning according to claim 8, characterized in that, In the optimization of the allocation of computing resources α and spectrum resources β, there is: Where V is an auxiliary variable representing the maximum latency among N devices, and there is

Images