Distributed AI training service-oriented 6G computing power network adaptive splitting federated learning method

By adopting adaptive split federated learning method in 6G computing power network, the model segmentation method and resource allocation are optimized, the resource allocation problem during intelligent terminal model aggregation in multi-base station scenarios is solved, training efficiency and model accuracy are improved, and model accuracy is simplified. Resource management and alleviate network congestion.

CN119031415AActive Publication Date: 2024-11-26NANJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202411249269.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-26
Estimated Expiration
2044-09-06

AI Technical Summary

Technical Problem

In multi-base station scenarios, it is difficult for the prior art to effectively optimize the model segmentation method and communication and computing multi-dimensional resource allocation in the 6G computing power network, resulting in insufficient network congestion and resource allocation during the aggregation of smart terminal models, which cannot meet the actual application needs.

Method used

A federated learning method for 6G computing power network adaptive splitting of federated learning for distributed AI training services is proposed. By establishing a 6G computing power network model, defining a wireless communication model, analyzing the delay of smart terminals in the model training and aggregation stage, and using it The shortest path search algorithm optimizes the model segmentation method, collaborative node selection and multi-domain resource scheduling to form joint optimization problems to reduce the total business delay.

Benefits of technology

It effectively alleviates the network congestion problem caused by the aggregation of smart terminal models under the single-base station learning framework, improves model training efficiency and generalization capabilities, simplifies resource allocation, shortens optimization time, and improves model accuracy and scalability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119031415A_ABST
    Figure CN119031415A_ABST
Patent Text Reader

Abstract

The invention discloses a distributed AI training service-oriented 6G computing power network adaptive splitting federated learning method, and belongs to the technical field of communication networks. The method comprises the following steps: establishing a 6G computing power network model, and constructing a distributed AI training service-oriented adaptive splitting federated learning model; analyzing the time delay of each intelligent terminal in the model training stage and the model aggregation stage; constructing a service total time delay minimization optimization problem; and carrying out joint optimization on the model segmentation mode, cooperative node selection and a multi-domain resource scheduling mode in the model training stage and the model aggregation stage by utilizing a shortest path search algorithm. Through joint optimization of a model segmentation mode, node selection and multi-dimensional resource allocation in a multi-base-station scene, the problem of network congestion caused by the fact that a single-base-station learning framework processes a large number of intelligent terminal model aggregation is relieved, the method adapts to heterogeneous intelligent terminal computing power and differentiated channel quality in an actual network, and the method is suitable for large-scale popularization and application. And the efficiency and generalization ability of model training are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of communication network technology, and specifically relates to a 6G computing power network adaptive splitting federated learning method for distributed AI training services. Background Art

[0002] With the standardization and widespread deployment of 5G networks, 6G for 2030 and beyond will continue to expand on the basis of 5G's three typical scenarios: enhanced mobile broadband, low-latency and highly reliable connections, and massive machine-type communications. As one of the new typical scenarios of 6G, inclusive intelligent services rely on the network to provide integrated communications and AI computing power for intelligent services that require efficient distributed intelligent learning or reasoning. It will not only serve specific applications, but will also serve the entire communication system in the future, improving the overall performance and efficiency of the network. In this scenario, a variety of devices in the network will be allowed to participate in the development and use of AI, so that fast and flexible intelligent services can cover all areas of society.

[0003] However, any single network element will face challenges such as limited computing power, intelligence and power at any time and anywhere. In view of the extremely high communication requirements and huge computing power consumption brought by intelligent services such as AI model training and reasoning, holographic communication, and immersive cloud XR (eXtended Reality), it is necessary for the computing power, algorithms, data and other resources and functions in the ubiquitously distributed network elements to work efficiently and collaboratively. In other words, in order to achieve intelligent inclusiveness and connected intelligence, and to ensure the quality of computing services such as AI in future networks, 6G networks need to transform from traditional information transmission infrastructure to a new type of infrastructure that provides both connection and computing services, namely computing power networks. This network can allocate and flexibly schedule computing resources, storage resources and network resources on demand between the cloud, edge and end according to business needs, realize deep integration of general computing, and give full play to the cluster advantages of computing power by connecting ubiquitous computing power, so as to build efficient, energy-saving, secure and flexible AI services and open ecosystems.

[0004] When developing AI services in the 6G computing network, it is mainly achieved through distributed learning technologies such as Federated Learning (FL) and Split Learning (SL). The advantage of FL is that it adopts parallel training between devices and fully considers the protection of data privacy. However, its training process will bring large computing and communication overheads, because each device needs to train a complete model and transmit it. In contrast, SL divides the model and uses multiple nodes for collaborative training, which can reduce the computing overhead of smart terminals compared to FL. However, the communication overhead of SL is still large. For this reason, some studies combine FL and SL, using parallel training while also performing model segmentation and collaborative training, thereby effectively improving training efficiency and accuracy to a certain extent. However, most of the current studies only focus on the FL or SL framework, and improve the accuracy or efficiency of model training by optimizing different variables. While considering fewer optimization variables, resource allocation usually only considers optimizing communication resource allocation or computing resource allocation. Among the few studies that combine FL and SL, few studies consider multi-base station scenarios, and the joint optimization of model segmentation methods and communication and computing multi-dimensional resource allocation in multi-base station scenarios, which cannot fully meet the actual application needs.

[0005] Therefore, how to solve the problem of reasonably allocating communications and calculating multi-dimensional resources is the technical problem that the present invention aims to solve. Summary of the invention

[0006] The purpose of the present invention is to provide a 6G computing power network adaptive splitting federated learning method for distributed AI training business, so as to solve the problems raised in the above background technology and the shortcomings of the prior art.

[0007] The object of the present invention is achieved by: a 6G computing network adaptive splitting federated learning method for distributed AI training business, characterized in that: the method comprises the following steps:

[0008] Step S1: Establish a 6G computing network model and build an adaptive split federated learning model for distributed AI training services;

[0009] Step S2: define a wireless communication model and analyze the delay of each smart terminal in the model training stage and the model aggregation stage;

[0010] Step S2-1: Analyze the delay of each smart terminal during the model training phase;

[0011] Step S2-2: Analyze the delay of each smart terminal in the model aggregation stage;

[0012] Step S3: construct an optimization problem for minimizing the total service delay; convert the optimization problem for minimizing the total service delay into two sub-problems, the two sub-problems include sub-problem 1 and sub-problem 2, sub-problem 1 is to optimize the training delay problem in the model training phase in each round of training iteration, and sub-problem 2 is to optimize the transmission delay problem in the model aggregation phase;

[0013] Step S4: Use the shortest path search algorithm to jointly optimize the model segmentation method, collaborative node selection, and multi-domain resource scheduling method in the model training phase and the model aggregation phase.

[0014] Preferably, the 6G computing network model includes a central cloud server, N base stations and K intelligent terminals, and the base stations are used for collection and Indicates that each base station n is equipped with a mobile edge computing server, which has computing resources and storage resources. The computing resources are represented by F n Indicates that storage resources are represented by M n express;

[0015] The intelligent terminal uses a collection Indicates that the computing resources and storage resources owned by each intelligent terminal k are represented by F k and M k express;

[0016] Own local datasets Indicates that D k represents the number of samples, and Represent the corresponding sample input features and output labels, Q represents the dimension of the sample input features, and let Indicates the total data set size of all smart terminals;

[0017] The relationship between the computing resources and storage resources of the intelligent terminal and the computing resources and storage resources of the base station is:

[0018] Preferably, in step S2-1, the delay of each smart terminal in the model training phase is analyzed, specifically:

[0019] Step S2-1-1: Determine the local loss function of the intelligent terminal k and the segmentation layer set of the DNN model in the model training stage;

[0020] The local loss function of each intelligent terminal k Among them, L(w; x i ,y i ) represents the sample loss function, x i Input features for the sample, y i is the sample output label; w is the model parameter;

[0021] use Represents the set of selectable segmentation layers in the DNN model, using Represents the model training split layer selection for terminal k;

[0022] Step S2-1-2: Divide the model training delay of each terminal k Divide the training modes and determine the update method of each training mode;

[0023] The training modes include training mode 1, training mode 2 and training mode 3, wherein the training mode for performing model training completely locally is training mode 1, the training mode for performing model training through edge-end collaboration is training mode 2, and the training mode through edge-end collaboration and edge-edge collaboration is training mode 3;

[0024] The subset of intelligent terminals in training mode 1 is The local model parameters of smart terminal k in the tth iteration are expressed as Each intelligent terminal k updates the local model parameters by stochastic gradient descent method:

[0025]

[0026] Among them, η l Represents the learning rate of the model in local training, represents the local model gradient calculated by smart terminal k; is the partial derivative vector of the local loss function of the intelligent terminal k with respect to the model parameters;

[0027] Training mode 1 only includes the local model training delay. The local model training delay

[0028] Among them, l FB Represents the computational effort of the forward and backward propagation processes to process a data sample.

[0029] Preferably, the intelligent terminal subset of the training mode 2 is The local model parameters of smart terminal k in the tth iteration Expressed as in and They represent the training mode 2 terminal side model and the training mode 2 base station side model respectively;

[0030] In training mode 2, the terminal side model of training mode 2 needs to be trained. The terminal side model delay of training mode 2 in, Indicates the computational effort of the forward propagation process of the training mode 2 terminal-side model to process one data sample;

[0031] Training mode 2: Segmentation layer output data of the terminal side model Obtained by the following formula:

[0032]

[0033] Where f(x;w) represents the mapping function between input x and output s under given model parameters w;

[0034] Will and i Transmission delay to base station n Among them, the binary variable Used to indicate whether terminal k selects base station n for access, if yes, it is 1, otherwise 0; represents the access base station subset, Indicates the data size of the segmentation layer of a data sample;

[0035] The terminal side model of training mode 2 uses SGD to update the terminal side model of training mode 2. The update formula is:

[0036]

[0037] Latency

[0038] in, represents the computational amount of the back propagation process of the training mode 2 terminal side model for processing a data sample; the intelligent terminal k updates the training mode 2 terminal side model and training mode 2 base station side model Stitch them together to get a new local model

[0039] The training mode 2 base station side model needs to be trained. The training mode 2 base station side model training and update delay in represents the computational effort of the forward and backward propagation processes of the training mode 2 base station side model for processing one data sample, Indicates the computing resources allocated by base station n to smart terminal k;

[0040] Segmentation layer gradient and updated training mode 2 base station side model parameters are transmitted back to the smart terminal k together, and the delay of this part in Indicates the size of the segmentation layer gradient data of a data sample;

[0041] Prediction results on the base station side Obtained by the following formula:

[0042]

[0043] According to the prediction results and the true value label y i , calculate the average gradient of the loss function The base station side model updates parameters through SGD, and the update formula is:

[0044]

[0045] Among them, η e Represents the learning rate of the model trained on the base station side; according to the chain rule of gradient calculation, the model parameters will be updated layer by layer from the output layer to the segmentation layer, and the segmentation layer gradient will be obtained

[0046] Preferably, the intelligent terminal subset of the training mode 3 is The local model parameters of smart terminal k in the tth iteration Expressed as in, and Respectively represent the terminal side model, the access base station side model and the cooperative base station side model;

[0047] The terminal side model of training mode 3 is trained with a delay of in Represents the amount of computation required for the forward propagation of a terminal-side model to process a data sample; the calculated output data of the terminal-side model segmentation layer is and i Transmitted to access base station n, its delay in Represents a segmentation layer for a data sample Data size;

[0048] Train the access base station side model of training mode 3, delay in, It represents the computational amount of the forward propagation process of the access base station side model to process one data sample;

[0049] The split layer output result of the access base station side model Obtained by the following formula:

[0050]

[0051] Will and i The transmission delay to the cooperative base station n′ The binary variable Used to indicate whether terminal k selects base station n′ to collaborate with access base station n to train the model. If yes, it is 1, otherwise it is 0; represents the subset of cooperative base stations, Represents a segmentation layer for a data sample Data size;

[0052] Train and update the cooperative base station side model of training mode 3, delay in, represents the computational effort of the forward and backward propagation processes of the cooperative base station side model to process a data sample, represents the computing resources allocated by the cooperative base station n′ to the terminal k;

[0053] Segmentation layer gradient and updated cooperative base station side model parameters are transmitted back to the access base station n together, and the delay of this part in, Represents a segmentation layer for a data sample The amount of gradient data;

[0054] Prediction results on the collaborative base station side Obtained by the following formula:

[0055]

[0056] According to the received segmentation layer gradient The access base station side model uses SGD to update parameters until the segmentation layer. The delay of this part in It represents the computational amount of the back propagation process of the access base station side model to process one data sample;

[0057] Access base station n will split the layer gradient Updated access base station side model And the collaborative base station side model transmitted to the corresponding terminal k together, with a delay of in, Represents a segmentation layer for a data sample The amount of gradient data;

[0058] The terminal side model is also updated using SGD, with a delay of in represents the computational amount of the back propagation process of the terminal side model to process a data sample. Finally, terminal k updates the terminal side model Access base station side model and collaborative base station side model Stitch them together to get a new local model

[0059] Preferably, in step S2-2, the delay of each smart terminal in the model aggregation stage is analyzed, specifically:

[0060] In the model aggregation stage, each terminal k directly transmits the local model to the local network through D2D link relay or cell link. Transmission delay of uploading to the selected base station n for aggregation It is expressed as:

[0061]

[0062] Among them, the binary variable Used to indicate whether terminal k selects base station n for model upload, if yes, it is 1, otherwise 0; binary variable It is used to indicate whether terminal k selects terminal k' as a relay to access base station n. If yes, it is 1, otherwise it is 0; terminal subset represents all directly connected terminals within the coverage area of ​​base station n, M w Indicates the amount of data for the model parameters.

[0063] Preferably, the service total delay minimization optimization problem is:

[0064]

[0065] Among them, the decision variables include the model segmentation method Collaboration Node Selection Multi-domain resource allocation method in the model training phase / model aggregation phase

[0066] C1 indicates that the total computing resources allocated by any base station to the terminals that select training modes 2 and 3 cannot exceed the upper limit of the base station; C2 indicates that the bandwidth resources allocated by any base station to the cell link cannot exceed the corresponding upper limit, and C3 and C4 respectively indicate the corresponding collaborative link and D2D link bandwidth resource constraints; C5 indicates that the sum of the transmission power of the terminals accessing any base station and the sum of the transmission power of all base stations have an upper limit, and the transmission power of any terminal or base station cannot exceed its maximum transmission power; C6 is a constraint on the model partitioning method of each terminal, and C7 indicates that any terminal can only select one base station to access; C8 indicates that in the model training stage, for any terminal, in addition to accessing the base station, only one collaborative base station can be selected; C9 indicates that in the model aggregation stage, for any terminal, if it needs to access the base station through a relay, only one terminal within the coverage of the base station can be selected as a relay node;

[0067] The optimization model for subproblem 1 is:

[0068]

[0069] C1, C3, C6, C8;

[0070] Among them, C2′ indicates that the bandwidth resources allocated by any base station to the cell link during the model training stage cannot exceed the corresponding upper limit; C5′ is the sub-constraint of the sum of the terminal transmission power accessing any base station during the model training stage, the total base station transmission power, and the transmission power range of each terminal; C7′ indicates that during the model training stage, any terminal can only choose to access one of the accessible base stations;

[0071] The optimization model for subproblem 2 is:

[0072]

[0073] C4, C9;

[0074] Among them, C2″ means that the bandwidth resources allocated by any base station to the cell link during the model aggregation stage cannot exceed the corresponding upper limit; C5″ is the sum of the terminal transmission power accessing any base station during the model aggregation stage and the sub-constraints of the transmission power range of each terminal; C7″ means that during the model aggregation stage, any terminal can only select one base station for uploading.

[0075] Preferably, in step S4, the shortest path search algorithm is used to jointly optimize the model segmentation mode, collaborative node selection and multi-domain resource scheduling mode in the model training stage and the model aggregation stage, specifically:

[0076] Step S4-1: allocating initial values ​​of the multi-domain resource allocation method for the model training phase and the model aggregation phase;

[0077] Assign the same initial value of the multi-domain resource allocation method in the model training phase to each terminal k to eliminate the impact of differences in resource allocation;

[0078] Assign the same initial value of the multi-domain resource allocation method in the model aggregation phase to each terminal k

[0079] Step S4-2: Select the shortest path search algorithm to optimize the model segmentation method selection and the collaborative node selection in sub-problem 1;

[0080] Specifically:

[0081] Construct a weighted directed graph G for each terminal k in subproblem 1 train =(O train ,E train );

[0082] Among them, O train is a node set, is the first-layer node, representing the first layer to the v0th layer of the training model of terminal k; is a layer 2 node, representing base station n x Train the model from layer v1 to layer v2; is a layer 3 node, representing base station n y Train the model from layer v3 to layer v; e train is the edge between nodes, e train ∈E train It represents the selection method of collaborative nodes, and its weight represents the transmission and computing delay;

[0083] Add a pair of virtual start / end vertices k ,e k As the source and destination nodes in the graph, the weight of the edge connecting to other nodes in the graph is 0, and the optimization goal becomes searching for the node from s k To e k The shortest path, all vertices on this path form the corresponding optimal model segmentation method and collaborative node selection;

[0084] That is in They represent the optimal access base station selection, cooperative base station selection and model segmentation mode of terminal k respectively;

[0085] The operation flow of the shortest path search algorithm for subproblem 1 is:

[0086] S4-2-1: Input terminal k transmits the output data of the model layer v to each base station n x The transmission delay Each base station n x Transmit the output data of the model layer v to other base stations n y The transmission delay Processing delay of layer v of terminal k computing model And each base station n x Calculate the processing delay of layer v of the model

[0087] S4-2-2: Calculate each node in the third layer The segmentation cost And let Indicates a layer 3 relay node The segmentation cost is 0;

[0088] S4-2-3: Layer 2 nodes According to the starting segmentation layer v1 and base station n x Divide into multiple subsets, and calculate the segmentation cost within each subset

[0089] S4-2-4: Get the optimal partitioning cost in each subset Corresponding to the termination segmentation layer and corresponding to the third layer base station And let Represents a layer 2 relay node The segmentation cost is 0, and it selects any relay node in layer 3 connect;

[0090] S4-2-5: Calculate each node in the first layer The segmentation cost

[0091] S4-2-6: Select the termination segmentation layer that minimizes the segmentation cost And get the corresponding second-layer base station Corresponding to the second layer termination split layer And corresponding to the third layer base station

[0092] S4-2-7: Output the shortest path to terminal k

[0093] Based on the optimal model segmentation method and collaborative node selection method Optimize multi-domain resource allocation during model training Transform subproblem 1 into a convex optimization problem P2′ as follows:

[0094]

[0095] stC1,C2′,C3,C5′;

[0096] in, and The optimizations between them do not affect each other;

[0097] Step S4-3: Select the shortest path search algorithm to optimize the model segmentation method selection and collaborative node selection in sub-problem 2.

[0098] Preferably, in step S4-3, the model segmentation method selection and the collaborative node selection in the shortest path search algorithm optimization sub-problem 2 are selected, specifically:

[0099] Construct a weighted directed graph G for each terminal k aggr,tran =(O aggr,tran ,E aggr,tran ), node collection Among them k is the only layer 1 node, representing terminal k; is the second-layer node, representing all terminals; is the layer 3 node, representing all base stations;

[0100] The edge between nodes aggr,tran ∈E aggr,tran Indicates the selection method of the collaborative node, and its weight represents the transmission delay; add a pair of virtual start / end vertices s k ,e k As the source node and destination node in the graph, the weight of the edge connecting to other nodes in the graph is 0; k To e k All vertices on the shortest path of form the corresponding optimal collaborative node selection method, that is, like but in, They represent the optimal terminal selection and base station selection of terminal k respectively;

[0101] The operation flow of selecting the shortest path search algorithm for sub-problem 1 is:

[0102] Step S4-3-1: Input terminal k transmits model parameters to each terminal k x The transmission delay Each terminal k x Transmit the model parameters to each base station n y The transmission delay

[0103] Step S4-3-2: Calculate each node in the third layer The transmission overhead

[0104] Step S4-3-3: Calculate each node in the second layer The transmission overhead And let That is, the delay of the terminal transmitting the model parameters to itself is 0;

[0105] Step S4-3-4: Layer 1 node o k Select the layer 2 node that minimizes the transmission overhead And further get the corresponding third-layer base station

[0106] Step S4-3-5: Output the shortest path to terminal k

[0107] Based on the optimal cooperation node selection method Optimize the multi-domain resource allocation method in the model aggregation phase Transform subproblem 2 into a convex optimization problem P3′ as follows:

[0108]

[0109] stC2″,C4,C5″;

[0110] in, and The optimizations do not affect each other.

[0111] Compared with the prior art, the present invention has the following improvements and advantages:

[0112] 1. By considering the joint optimization of model segmentation, node selection and multi-dimensional resource allocation in multi-base station scenarios, the network congestion problem that may be caused by the aggregation of a large number of smart terminal models in a single base station learning framework can be effectively alleviated; at the same time, multi-dimensional resource allocation and flexible model segmentation and node selection can better adapt to the heterogeneous smart terminal computing power and differentiated channel quality in the actual network, and improve the efficiency and generalization ability of model training.

[0113] 2. Convert the optimization problem of minimizing the total service latency into two 2-sub-problems, so that the latency of each round of training iteration can be minimized, simplifying the complexity of obtaining the optimal solution, shortening the optimization time, and improving the accuracy and scalability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0115] Figure 2 This is the training flow chart of the smart terminal training in training mode 2.

[0116] Figure 3 This is the training flow chart of the smart terminal training in training mode 3.

[0117] Figure 4 This is a schematic diagram of the ResNet18 neural network model structure.

[0118] Figure 5 It is the optimal transmission topology in the model training phase.

[0119] Figure 6 This is the flow chart of the optimal transmission topology training in the model aggregation stage.

[0120] Figure 7 This is a graph showing how the model accuracy changes with the number of iterations in the method of the present invention. DETAILED DESCRIPTION

[0121] The present invention is further summarized below with reference to the accompanying drawings.

[0122] like Figure 1 As shown, a 6G computing network adaptive splitting federated learning method for distributed AI training business, the method includes the following steps:

[0123] Step S1: Establish a 6G computing network model and build an adaptive split federated learning model for distributed AI training services; the adaptive split federated learning model is applicable to DNN models, such as Figure 4 As shown, for example, LeNet, AlexNet, VGG16, and ResNet18, etc.;

[0124] The 6G computing network model includes a central cloud server, N base stations and K smart terminals. Indicates that each base station n is equipped with a mobile edge computing server, which has computing resources and storage resources. The computing resources are represented by F n Indicates that storage resources are represented by M n It indicates that the physical distance between the central cloud server and the intelligent terminal is far, and the computing resources and storage resources are sufficient, which are only used for edge model aggregation and global model delivery in the model aggregation stage.

[0125] Smart terminal collection Indicates that the computing resources and storage resources owned by each intelligent terminal k are represented by F k and M k express;

[0126] Own local datasets Indicates that D k represents the number of samples, and Represent the corresponding sample input features and output labels, Q represents the dimension of the sample input features, and let Indicates the total data set size of all smart terminals;

[0127] The relationship between the computing resources and storage resources of the intelligent terminal and the computing resources and storage resources of the base station is:

[0128] The adaptive split federated learning model adopts the DNN model structure; any DNN model structure is acceptable; the parameters of the DNN model such as input, output, number of hidden layers and number of neurons are known, the model parameters are represented by w, and the total number of model layers is represented by V.

[0129] S101. The central cloud server initializes the model parameter w0 and sends it to all smart terminals; initializes the training iteration round t=1 and the maximum training iteration round T; the maximum training iteration round T is related to the learning accuracy requirements of the distributed AI training service. After each iteration, the verification data set can be used to calculate the model accuracy θ(t) of each smart terminal, thereby obtaining the average model accuracy And T needs to satisfy where θ thIndicates the model accuracy threshold of the distributed AI training service;

[0130] S102, if t>T, then enter step S109; if t≤T, then enter the model training stage, and solve the training mode and resource allocation mode of each intelligent terminal k according to the proposed algorithm; wherein, the training mode is divided into three types, a part of the intelligent terminals with more computing resources and storage resources can use the local data set to independently complete the model training (as shown in training mode 1), another part of the intelligent terminals with limited computing resources and storage resources need to complete the model training through edge-end collaboration (as shown in training mode 2), and the remaining intelligent terminals with less computing resources and storage resources need to complete the model through edge-end collaboration and edge-edge collaboration (as shown in training mode 3);

[0131] S103, each intelligent terminal k independently completes or collaborates with the base station to complete the local model according to the obtained training method and resource allocation method Update calculation of ;

[0132] S104, entering the model aggregation phase, solving the transmission mode and resource allocation mode of each smart terminal k according to the proposed algorithm; wherein, there are two transmission modes, each smart terminal transmits the updated local model to the neighboring smart terminal through the D2D link based on its own channel quality status, and then relays and uploads it to the selected base station (as shown in transmission mode 1), or directly uploads it to the selected base station through the cell link (as shown in transmission mode 2);

[0133] Each smart terminal k directly transmits the local model through the D2D link relay The transmission mode for uploading to the selected base station n for aggregation is transmission mode 1. Each intelligent terminal k directly transmits the local model through the cell link The transmission mode for uploading to the selected base station n for aggregation is transmission mode 2.

[0134] S105, each intelligent terminal k updates the local model according to the obtained transmission mode and resource allocation mode Transmit to selected base station;

[0135] S106. Each base station n aggregates the local model through the FedAvg algorithm to obtain the edge model And sent to all intelligent terminals that access this base station through the cell link or D2D link relay; represents the set of intelligent terminals connected to base station n, Indicates the total data set size of smart terminals that access base station n through cell links or D2D link relays;

[0136] S107, if t%T_GLOBAL (global aggregation round constant)==0, each base station n will edge model Upload to the central cloud server, the central cloud server aggregates the edge model through the FedAvg algorithm to obtain the global model And send it to all smart terminals; otherwise, directly go to step S108;

[0137] S108, let t=t+1, and return to step S102;

[0138] S109, output the final model parameter w T .

[0139] Step S2: define a wireless communication model and analyze the delay of each smart terminal in the model training stage and the model aggregation stage;

[0140] Define the wireless communication model, specifically:

[0141] The total bandwidth used by the cell link, cooperative link and D2D link is represented as B Cellular ,B B2B and B D2D ,The frequency bands used by cell links, cooperative links and D2D links are different;

[0142] The orthogonal channel resources allocated to each base station n are expressed as b n Express, satisfy

[0143] The signal-to-noise ratio threshold for maintaining communication between intelligent terminal k and base station n is expressed as When the uplink signal-to-noise ratio γ between the intelligent terminal k and the base station n k,n Greater than or equal to threshold When , it is considered that the intelligent terminal k is within the coverage of the base station n, and the intelligent terminal k communicates directly with the base station n;

[0144] Introducing binary variables Indicator, uplink signal-to-noise ratio

[0145] Among them, p k represents the transmission power of intelligent terminal k, and its maximum value is h k,n represents the cell link channel gain from intelligent terminal k to base station n, and N0 represents the noise power;

[0146] Introducing binary variables Used to indicate whether base station n can communicate directly with another base station n′, where: Indicates the signal-to-noise ratio threshold for maintaining communication between base stations. The signal-to-noise ratio between base station n and another base station n′ p n represents the transmission power of base station n, and its maximum value is hn,n′ represents the cooperative link channel gain from base station n to another base station n′;

[0147] Downlink signal-to-noise ratio between base station n and smart terminal k The signal-to-noise ratio between intelligent terminal k and another intelligent terminal k′ where h k,k′ represents the D2D link channel gain from smart terminal k to another smart terminal k′;

[0148] According to Shannon's formula, the uplink rate of each intelligent terminal k in the cell link is R k,n , the downlink rate R of each base station n n,k , the achievable rate R of the cooperative link of each base station n n,n′ and the D2D link achievable rate R of each smart terminal k k,k′ ;

[0149] R k,n =b k,n log2(1+γ k,n );

[0150] R n,k =b k,n log2(1+γ n,k );

[0151] R n,n′ =b n,n′ log2(1+γ n,n′ );

[0152] R k,k′ =b k,k′ log2(1+γ k,k′ );

[0153] Among them, b k,n represents the bandwidth resources allocated to the intelligent terminal k for communication with the base station n, b n,n′ represents the bandwidth resources allocated to base station n for communication with another base station n′, b k,k′ Represents the bandwidth resources allocated to the intelligent terminal k for communication with another intelligent terminal k′.

[0154] All base stations are connected to the central cloud server through different high-bandwidth backhaul links. However, due to factors such as multi-user scheduling and complex routing algorithms, the transmission delay is difficult to model. Therefore, it is assumed that the backhaul link resource allocation strategy and routing algorithm of each base station are known, so that the achievable rate of its backhaul link is fixed.

[0155] Step S2-1: Analyze the delay of each smart terminal during the model training phase, specifically:

[0156] Step S2-1-1: Determine the local loss function of the intelligent terminal k and the segmentation layer set of the DNN model in the model training stage;

[0157] The local loss function of each intelligent terminal k Among them, L(w; x i ,y i ) represents the sample loss function, x i Input features for the sample, y i is the sample output label; w is the model parameter;

[0158] use Represents the set of selectable segmentation layers in the DNN model, using represents the model training segmentation layer selection of smart terminal k;

[0159] Step S2-1-2: Divide the model training delay of each smart terminal k Divide the training modes and determine the update method for each training mode;

[0160] The training modes include training mode 1, training mode 2 and training mode 3. The training mode for model training completely locally is training mode 1, the training mode for model training through edge-end collaboration is training mode 2, and the training mode through edge-end collaboration and edge-edge collaboration is training mode 3.

[0161] The subset of intelligent terminals in training mode 1 is The local model parameters of smart terminal k in the tth iteration are expressed as Each intelligent terminal k updates the local model parameters by stochastic gradient descent method:

[0162]

[0163] Among them, η l Represents the learning rate of the model in local training, represents the local model gradient calculated by smart terminal k; is the partial derivative vector of the local loss function of the intelligent terminal k with respect to the model parameters;

[0164] Training mode 1 only includes the local model training delay. The local model training delay

[0165] Among them, l FB Represents the computational effort of the forward and backward propagation processes to process a data sample.

[0166] The subset of intelligent terminals in training mode 2 is The local model parameters of smart terminal k in the tth iteration Expressed as in and They represent the training mode 2 terminal side model and the training mode 2 base station side model respectively;

[0167] In training mode 2, the terminal side model of training mode 2 needs to be trained, and the delay in, Indicates the computational effort of the forward propagation process of the training mode 2 terminal-side model to process one data sample;

[0168] Training mode 2: Segmentation layer output data of the terminal side model Obtained by the following formula:

[0169]

[0170] Where f(x;w) represents the mapping function between input x and output s under given model parameters w;

[0171] Will and i Transmission delay to base station n Among them, the binary variable Used to indicate whether terminal k selects base station n for access, if yes, it is 1, otherwise 0; represents the access base station subset, Indicates the data size of the segmentation layer of a data sample;

[0172] The terminal side model of training mode 2 uses SGD to update the terminal side model of training mode 2. The update formula is:

[0173]

[0174] Latency

[0175] in, represents the computational amount of the back propagation process of the training mode 2 terminal side model for processing a data sample; the intelligent terminal k updates the training mode 2 terminal side model and training mode 2 base station side model Stitch them together to get a new local model

[0176] The training mode 2 base station side model needs to be trained. The training mode 2 base station side model training and update delay in represents the computational effort of the forward and backward propagation processes of the training mode 2 base station side model for processing one data sample, Indicates the computing resources allocated by base station n to smart terminal k;

[0177] Segmentation layer gradient and updated training mode 2 base station side model parameters are transmitted back to the smart terminal k together, and the delay of this part in Indicates the size of the segmentation layer gradient data of a data sample;

[0178] Prediction results on the base station side Obtained by the following formula:

[0179]

[0180] According to the prediction results and the true value label y i , calculate the average gradient of the loss function The base station side model updates parameters through SGD, and the update formula is:

[0181]

[0182] Among them, η e Represents the learning rate of the model trained on the base station side; according to the chain rule of gradient calculation, the model parameters will be updated layer by layer from the output layer to the segmentation layer, and the segmentation layer gradient will be obtained

[0183] The subset of intelligent terminals in training mode 3 is The local model parameters of smart terminal k in the tth iteration Expressed as in, and Respectively represent the terminal side model, the access base station side model and the cooperative base station side model;

[0184] Train the terminal side model of training mode 3, delay in Represents the amount of computation required for the forward propagation of a terminal-side model to process a data sample; the calculated output data of the terminal-side model segmentation layer is and i Transmitted to access base station n, its delay in Represents a segmentation layer for a data sample Data size;

[0185] Train the access base station side model of training mode 3, delay in, It represents the computational amount of the forward propagation process of the access base station side model to process one data sample;

[0186] The split layer output result of the access base station side model Obtained by the following formula:

[0187]

[0188] Will and i The transmission delay to the cooperative base station n′ The binary variable Used to indicate whether terminal k selects base station n′ to collaborate with access base station n to train the model. If yes, it is 1, otherwise it is 0; represents the subset of cooperative base stations, Represents a segmentation layer for a data sample Data size;

[0189] Train and update the cooperative base station side model of training mode 3, delay in, represents the computational effort of the forward and backward propagation processes of the cooperative base station side model to process a data sample, represents the computing resources allocated by the cooperative base station n′ to the terminal k;

[0190] Segmentation layer gradient and updated cooperative base station side model parameters are transmitted back to the access base station n together, and the delay of this part in, Represents a segmentation layer for a data sample The amount of gradient data;

[0191] Prediction results on the collaborative base station side Obtained by the following formula:

[0192]

[0193] According to the received segmentation layer gradient The access base station side model uses SGD to update parameters until the segmentation layer. The delay of this part in It represents the computational amount of the back propagation process of the access base station side model to process one data sample;

[0194] Access base station n will split the layer gradient Updated access base station side model And the collaborative base station side model transmitted to the corresponding terminal k together, with a delay of in, Represents a segmentation layer for a data sample The amount of gradient data;

[0195] The terminal side model is also updated using SGD, with a delay of in represents the computational amount of the back propagation process of the terminal side model to process a data sample. Finally, terminal k updates the terminal side model Access base station side model and collaborative base station side model Stitch them together to get a new local model

[0196] In step S2-2, the delay of each smart terminal in the model aggregation stage is analyzed, specifically:

[0197] In the model aggregation stage, each smart terminal k directly transmits the local model to the local network through D2D link relay or cell link. Transmission delay of uploading to the selected base station n for aggregation It is expressed as:

[0198]

[0199] Among them, the binary variable Used to indicate whether the intelligent terminal k selects base station n for model upload, if yes, it is 1, otherwise 0; binary variable Used to indicate whether smart terminal k selects smart terminal k' as a relay to access base station n. If yes, it is 1, otherwise it is 0; smart terminal subset represents all directly connected smart terminals within the coverage area of ​​base station n, M w Indicates the amount of data for the model parameters.

[0200] Step S3: construct the optimization problem of minimizing the total service delay;

[0201] The optimization problem of minimizing the total service delay is transformed into two sub-problems, including sub-problem 1 and sub-problem 2. Sub-problem 1 is to optimize the training delay problem of the model training phase in each round of training iteration, and sub-problem 2 is to optimize the transmission delay problem of the model aggregation phase.

[0202] The optimization problem of minimizing the total service delay is:

[0203]

[0204]

[0205]

[0206]

[0207] Among them, the decision variables include the model segmentation method Collaboration Node Selection Multi-domain resource allocation method in the model training phase / model aggregation phase

[0208] C1 indicates that the total computing resources allocated by any base station to the smart terminals that select training modes 2 and 3 cannot exceed the upper limit of the base station; C2 indicates that the bandwidth resources allocated by any base station to the cell link cannot exceed the corresponding upper limit, and C3 and C4 respectively indicate the corresponding collaborative link and D2D link bandwidth resource constraints; C5 indicates that the sum of the transmission power of smart terminals accessing any base station and the sum of the transmission power of all base stations have an upper limit, and the transmission power of any smart terminal or base station cannot exceed its maximum transmission power; C6 is a constraint on the model segmentation method of each smart terminal, and C7 indicates that any smart terminal can only select one base station to access; C8 indicates that in the model training stage, for any smart terminal, in addition to accessing the base station, only one collaborative base station can be selected; C9 indicates that in the model aggregation stage, for any smart terminal, if it needs to access the base station through a relay, only one smart terminal within the coverage of the base station can be selected as a relay node;

[0209] The optimization model for subproblem 1 is:

[0210]

[0211]

[0212] C1, C3, C6, C8;

[0213] Among them, C2′ indicates that the bandwidth resources allocated by any base station to the cell link during the model training stage cannot exceed the corresponding upper limit; C5′ is the sub-constraint of the sum of the terminal transmission power accessing any base station during the model training stage, the total base station transmission power, and the transmission power range of each terminal; C7′ indicates that during the model training stage, any terminal can only choose to access one of the accessible base stations;

[0214] The optimization model for subproblem 2 is:

[0215]

[0216] C4, C9;

[0217] Among them, C2″ means that the bandwidth resources allocated by any base station to the cell link during the model aggregation stage cannot exceed the corresponding upper limit; C5″ is the sum of the terminal transmission power accessing any base station during the model aggregation stage and the sub-constraints of the transmission power range of each terminal; C7″ means that during the model aggregation stage, any terminal can only select one base station for uploading.

[0218] Step S4: Use the shortest path search algorithm to jointly optimize the model segmentation method, collaborative node selection, and multi-domain resource scheduling method in the model training phase and the model aggregation phase, specifically:

[0219] Step S4-1: allocating initial values ​​of the multi-domain resource allocation method for the model training phase and the model aggregation phase;

[0220] Assign the same initial value of the multi-domain resource allocation method in the model training phase to each smart terminal k to eliminate the impact of differences in resource allocation;

[0221] Assign the same initial value of the multi-domain resource allocation method in the model aggregation phase to each smart terminal k

[0222] Step S4-2: Select the shortest path search algorithm to optimize the model segmentation method selection and the collaborative node selection in sub-problem 1;

[0223] Specifically:

[0224] Construct a weighted directed graph G for each intelligent terminal k in subproblem 1 train =(O train ,E train );

[0225] Among them, O train is a node set, It is the first-layer node, representing the first layer to the v0 layer of the intelligent terminal k training model; is a layer 2 node, representing base station n x Train the model from layer v1 to layer v2; is a layer 3 node, representing base station n y Train the model from layer v3 to layer v; e train is the edge between nodes, e train ∈E train It represents the selection method of collaborative nodes, and its weight represents the transmission and computing delay;

[0226] Add a pair of virtual start / end vertices k ,e k As the source and destination nodes in the graph, the weight of the edge connecting to other nodes in the graph is 0, and the optimization goal becomes searching for the node from s k To e k The shortest path, all vertices on this path form the corresponding optimal model segmentation method and collaborative node selection;

[0227] That is in They represent the optimal access base station selection, cooperative base station selection and model segmentation method of intelligent terminal k respectively;

[0228] like Figure 2 As shown, the operation flow of selecting the short path search algorithm for sub-problem 1 is:

[0229] S4-2-1: Input intelligent terminal k transmits the output data of the model layer v to each base station n x The transmission delay Each base station n x Transmit the output data of the model layer v to other base stations n y The transmission delay Processing delay of layer v of intelligent terminal k computing model And each base station n x Calculate the processing delay of layer v of the model

[0230] S4-2-2: Calculate each node in the third layer The segmentation cost And let

[0231] Indicates a layer 3 relay node The segmentation cost is 0;

[0232] S4-2-3: Layer 2 nodes According to the starting segmentation layer v1 and base station n x Divide into multiple subsets, and calculate the segmentation cost within each subset

[0233] S4-2-4: Get the optimal partitioning cost in each subset Corresponding to the termination segmentation layer and corresponding to the third layer base station And let Represents a layer 2 relay node The segmentation cost is 0, and it selects any relay node in layer 3 connect;

[0234] S4-2-5: Calculate each node in the first layer The segmentation cost

[0235] S4-2-6: Select the termination segmentation layer that minimizes the segmentation cost And get the corresponding second-layer base station

[0236] Corresponding to the second layer termination split layer And corresponding to the third layer base station

[0237] S4-2-7: Output the shortest path of intelligent terminal k

[0238] Based on the optimal model segmentation method and collaborative node selection method Optimize multi-domain resource allocation during model training Transform subproblem 1 into a convex optimization problem P2′ as follows:

[0239]

[0240] stC1,C2′,C3,C5′;

[0241] in, and The optimizations between them do not affect each other;

[0242] Step S4-3: Select the shortest path search algorithm to optimize the model segmentation method selection and collaborative node selection in sub-problem 2, specifically:

[0243] Construct a weighted directed graph G for each intelligent terminal k aggr,tran =(O aggr,tran ,E aggr,tran ), node collection Among them k is the only first-layer node, representing smart terminal k; It is the second layer node, representing all intelligent terminals; is the layer 3 node, representing all base stations;

[0244] The edge between nodes aggr,tran ∈E aggr,tran Indicates the selection method of the collaborative node, and its weight represents the transmission delay; add a pair of virtual start / end vertices s k ,e k As the source node and destination node in the graph, the weight of the edge connecting to other nodes in the graph is 0; k To e k All vertices on the shortest path of form the corresponding optimal collaborative node selection method, that is, like but in, They represent the optimal intelligent terminal selection and base station selection of intelligent terminal k respectively;

[0245] The operation flow of selecting the shortest path search algorithm for sub-problem 1 is:

[0246] like Figure 3As shown, step S4-3-1: input intelligent terminal k to transmit the model parameters to each intelligent terminal k x The transmission delay Each intelligent terminal k x Transmit the model parameters to each base station n y The transmission delay

[0247] Step S4-3-2: Calculate each node in the third layer The transmission overhead

[0248] Step S4-3-3: Calculate each node in the second layer The transmission overhead And let That is, the delay of the intelligent terminal transmitting the model parameters to itself is 0;

[0249] Step S4-3-4: Layer 1 node o k Select the layer 2 node that minimizes the transmission overhead And further get the corresponding third-layer base station

[0250] Step S4-3-5: Output the shortest path of smart terminal k

[0251] Based on the optimal cooperation node selection method Optimize the multi-domain resource allocation method in the model aggregation phase Transform subproblem 2 into a convex optimization problem P3′ as follows:

[0252] stC2″,C4,C5″;

[0253] in, and The optimizations do not affect each other.

[0254] In order to verify the effectiveness of the adaptive split federated learning model of the present invention, the following verification is carried out. According to the setting of relevant simulation parameters, in the model training stage and the model aggregation stage, in order to more prominently and clearly display the collaboration between the edge and the smart terminal; Figure 5As shown in the figure, during the model training phase, smart terminals 2 and 4 chose to perform only local training due to their sufficient computing power. Smart terminals 1, 3, 5, and 6 chose to collaborate with dual base stations, and since smart terminal 1 has relatively high computing power, the segmentation layer of its smart terminal side model is selected as layer 4, while the segmentation layer of smart terminal side model of smart terminals 3, 5, and 6 is selected as layer 1. Taking into account their own computing power and the channel conditions between the base stations that can communicate directly, smart terminals 7, 8, and 9 chose to collaborate with a single base station. Figure 6 As shown, in the model aggregation stage, in order to speed up the transmission of model parameters, smart terminals 1, 3 and 9 select other smart terminals as relays for transmission.

[0255] Furthermore, the accuracy of the proposed learning framework is verified based on the optimal model segmentation method for each intelligent terminal, the selection of collaborative nodes, and the multi-domain resource allocation method in the model training phase and the aggregation phase, and it is compared with some benchmark algorithms. The simulation results are as follows: Figure 7 As shown in the figure, the algorithms involved in the comparison include: centralized learning, which uses only one base station to train the entire ResNet18 neural network using the HAM10000 dataset; split learning (SL), which uses one base station and nine smart terminals for serial collaborative training, where the split layer of the smart terminal side model and the service side model is fixed to the second layer; federated learning (FL), which uses one base station and nine smart terminals for parallel collaborative training, where each smart terminal trains the entire neural network on its local dataset, and the base station aggregates and updates the model; and split federated learning (SFL), which combines the above two, uses one base station and nine smart terminals for parallel collaborative training, where each smart terminal is only trained to the second layer of the model, and the rest is trained in parallel through the base station, and the smart terminal side model and the base station side model are aggregated and updated after each round of iteration. In addition, the test accuracy is the average of the model accuracy calculated by all smart terminals on their respective test sets, and the batch size used in the training process is 256.

[0256] from Figure 7It can be seen that the algorithm proposed in the present invention gradually tends to converge during 100 rounds of iterations, and the final test accuracy is stable at about 75%, and the convergence trend is roughly the same as that of several other algorithms. It is worth noting that in the initial stage (0 to 20 rounds of iterations), the test accuracy of the algorithm proposed in the present invention experienced a process of first decreasing and then increasing. This phenomenon is mainly attributed to the differences in model segmentation methods, selected model aggregation nodes, and data distribution adopted by different smart terminals. Therefore, after completing edge model aggregation and updating model parameters at the beginning of the iteration, these insufficiently adjusted parameters cannot adapt well to the data set characteristics of some smart terminals, resulting in an initial decrease in test accuracy. As the iteration proceeds, through continuous optimization and adjustment, the algorithm can gradually overcome these initial challenges, so that the test accuracy can be recovered and stabilized. In contrast, centralized learning shows a higher test accuracy at the beginning of the iteration because this method is trained only through a single node, thereby avoiding the problems caused by data set differences and model aggregation. The reason why SL can obtain a high accuracy close to centralized learning in the early stage is that it trains one by one according to smart terminals, which also avoids the above problems. However, both approaches come at the expense of flexibility and result in longer training latency.

[0257] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A 6G computing network adaptive splitting federated learning method for distributed AI training services, characterized by: The method comprises the following steps: Step S1: Establish a 6G computing network model and build an adaptive split federated learning model for distributed AI training services; Step S2: define a wireless communication model and analyze the delay of each smart terminal in the model training stage and the model aggregation stage; Step S2-1: Analyze the delay of each smart terminal during the model training phase; Step S2-2: Analyze the delay of each smart terminal in the model aggregation stage; Step S3: construct an optimization problem for minimizing the total service delay; convert the optimization problem for minimizing the total service delay into two sub-problems, the two sub-problems include sub-problem 1 and sub-problem 2, sub-problem 1 is to optimize the training delay problem in the model training phase in each round of training iteration, and sub-problem 2 is to optimize the transmission delay problem in the model aggregation phase; Step S4: Use the shortest path search algorithm to jointly optimize the model segmentation method, collaborative node selection, and multi-domain resource scheduling method in the model training phase and the model aggregation phase.

2. According to claim 1, a 6G computing network adaptive splitting federated learning method for distributed AI training services is characterized by: The 6G computing network model includes a central cloud server, N base stations and K smart terminals. Indicates that each base station n is equipped with a mobile edge computing server, which has computing resources and storage resources. The computing resources are represented by F n Indicates that storage resources are represented by M n express; The intelligent terminal uses a collection Indicates that the computing resources and storage resources owned by each intelligent terminal k are represented by F k and M k express; Own local datasets Indicates that D k represents the number of samples, and Represent the corresponding sample input features and output labels, Q represents the dimension of the sample input features, and let Indicates the total data set size of all smart terminals; The relationship between the computing resources and storage resources of the intelligent terminal and the computing resources and storage resources of the base station is:

3. According to claim 1, a 6G computing network adaptive splitting federated learning method for distributed AI training business is characterized by: In step S2-1, the delay of each smart terminal in the model training phase is analyzed, specifically: Step S2-1-1: Determine the local loss function of the intelligent terminal k and the segmentation layer set of the DNN model in the model training stage; The local loss function of each intelligent terminal k Among them, L(w; x i ,y i ) represents the sample loss function, x i Input features for the sample, y i is the sample output label; w is the model parameter; use Represents the set of selectable segmentation layers in the DNN model, using represents the model training split layer selection for terminal k; Step S2-1-2: Divide the model training delay of each terminal k Divide the training modes and determine the update method of each training mode; The training modes include training mode 1, training mode 2 and training mode 3, wherein the training mode for performing model training completely locally is training mode 1, the training mode for performing model training through edge-end collaboration is training mode 2, and the training mode through edge-end collaboration and edge-edge collaboration is training mode 3; The subset of intelligent terminals in training mode 1 is The local model parameters of smart terminal k in the tth iteration are expressed as Each intelligent terminal k updates the local model parameters by stochastic gradient descent method: Among them, η l Represents the learning rate of the model in local training, represents the local model gradient calculated by smart terminal k; ▽L k is the partial derivative vector of the local loss function of the intelligent terminal k with respect to the model parameters; Training mode 1 only includes the local model training delay. The local model training delay Among them, Represents the computational effort of the forward and backward propagation processes to process a data sample.

4. According to claim 3, a 6G computing network adaptive splitting federated learning method for distributed AI training services is characterized by: The intelligent terminal subset of the training mode 2 is: The local model parameters of smart terminal k in the tth iteration Expressed as in and They represent the training mode 2 terminal side model and the training mode 2 base station side model respectively; In training mode 2, the terminal side model of training mode 2 needs to be trained. The terminal side model delay of training mode 2 in, Indicates the computational effort of the forward propagation process of the training mode 2 terminal-side model to process one data sample; Training mode 2: Segmentation layer output data of the terminal side model It is obtained by the following formula: Where f(x;w) represents the mapping function between input x and output s under given model parameters w; Will and i Transmission delay to base station n Among them, the binary variable Used to indicate whether terminal k selects base station n for access, if yes, it is 1, otherwise 0; represents the access base station subset, Indicates the data size of the segmentation layer of a data sample; The terminal side model of training mode 2 uses SGD to update the terminal side model of training mode 2. The update formula is: Latency in, represents the computational amount of the back propagation process of the training mode 2 terminal side model for processing a data sample; the intelligent terminal k updates the training mode 2 terminal side model and training mode 2 base station side model Stitch them together to get a new local model The training mode 2 base station side model needs to be trained. The training mode 2 base station side model training and update delay in represents the computational effort of the forward and backward propagation processes of the training mode 2 base station side model for processing one data sample, Indicates the computing resources allocated by base station n to smart terminal k; Segmentation layer gradient and updated training mode 2 base station side model parameters are transmitted back to the smart terminal k together, and the delay of this part in Indicates the size of the segmentation layer gradient data of a data sample; Prediction results on the base station side It is obtained by the following formula: According to the prediction results and the true value label y i , calculate the average gradient of the loss function The base station side model updates parameters through SGD, and the update formula is: Among them, η e Represents the learning rate of the model trained on the base station side; according to the chain rule of gradient calculation, the model parameters will be updated layer by layer from the output layer to the segmentation layer, and the segmentation layer gradient will be obtained 5. According to claim 3, a 6G computing network adaptive splitting federated learning method for distributed AI training services is characterized by: The intelligent terminal subset of the training mode 3 is: The local model parameters of smart terminal k in the tth iteration Expressed as in, and Respectively represent the terminal side model, the access base station side model and the cooperative base station side model; The terminal side model of training mode 3 is trained with a delay of in Represents the amount of computation required for the forward propagation of a terminal-side model to process a data sample; the calculated output data of the terminal-side model segmentation layer is and i Transmitted to access base station n, its delay in Represents a segmentation layer for a data sample Data size; Train the access base station side model of training mode 3, delay in, It represents the computational amount of the forward propagation process of the access base station side model to process one data sample; The split layer output result of the access base station side model It is obtained by the following formula: Will and i The transmission delay to the cooperative base station n′ The binary variable Used to indicate whether terminal k selects base station n′ to collaborate with access base station n to train the model. If yes, it is 1, otherwise it is 0; represents the subset of cooperative base stations, Represents a segmentation layer for a data sample Data size; Train and update the cooperative base station side model of training mode 3, delay in, represents the computational effort of the forward and backward propagation processes of the cooperative base station side model to process a data sample, represents the computing resources allocated by the cooperative base station n′ to the terminal k; Segmentation layer gradient and updated cooperative base station side model parameters are transmitted back to the access base station n together, and the delay of this part in, Represents a segmentation layer for a data sample The amount of gradient data; Prediction results on the collaborative base station side It is obtained by the following formula: According to the received segmentation layer gradient The access base station side model uses SGD to update parameters until the segmentation layer. The delay of this part in It represents the computational amount of the back propagation process of the access base station side model to process a data sample; Access base station n will split the layer gradient Updated access base station side model And the collaborative base station side model transmitted to the corresponding terminal k together, with a delay of in, Represents a segmentation layer for a data sample The amount of gradient data; The terminal side model is also updated using SGD, with a delay of in represents the computational amount of the back propagation process of the terminal side model to process a data sample. Finally, terminal k updates the terminal side model Access base station side model and collaborative base station side model Stitch them together to get a new local model 6. According to claim 1, a 6G computing network adaptive splitting federated learning method for distributed AI training services is characterized by: In step S2-2, the delay of each smart terminal in the model aggregation stage is analyzed, specifically: In the model aggregation stage, each terminal k directly transmits the local model to the local network through D2D link relay or cell link. Transmission delay of uploading to the selected base station n for aggregation It is expressed as: Among them, the binary variable Used to indicate whether terminal k selects base station n for model upload, if yes, it is 1, otherwise 0; binary variable It is used to indicate whether terminal k selects terminal k' as a relay to access base station n. If yes, it is 1, otherwise it is 0; terminal subset represents all directly connected terminals within the coverage area of ​​base station n, M w Indicates the amount of data for the model parameters.

7. According to claim 1, a 6G computing network adaptive splitting federated learning method for distributed AI training services is characterized by: The optimization problem of minimizing the total service delay is: Among them, the decision variables include the model segmentation method Collaboration Node Selection Multi-domain resource allocation method in the model training phase / model aggregation phase C1 indicates that the total computing resources allocated by any base station to the terminals that select training modes 2 and 3 cannot exceed the upper limit of the base station; C2 indicates that the bandwidth resources allocated by any base station to the cell link cannot exceed the corresponding upper limit, and C3 and C4 respectively indicate the corresponding collaborative link and D2D link bandwidth resource constraints; C5 indicates that the sum of the transmission power of the terminals accessing any base station and the sum of the transmission power of all base stations have an upper limit, and the transmission power of any terminal or base station cannot exceed its maximum transmission power; C6 is a constraint on the model partitioning method of each terminal, and C7 indicates that any terminal can only select one base station to access; C8 indicates that in the model training stage, for any terminal, in addition to accessing the base station, only one collaborative base station can be selected; C9 indicates that in the model aggregation stage, for any terminal, if it needs to access the base station through a relay, only one terminal within the coverage of the base station can be selected as a relay node; The optimization model for subproblem 1 is: C1, C3, C6, C8; Among them, C2′ indicates that the bandwidth resources allocated by any base station to the cell link during the model training stage cannot exceed the corresponding upper limit; C5′ is the sub-constraint of the sum of the terminal transmission power accessing any base station during the model training stage, the total base station transmission power, and the transmission power range of each terminal; C7′ indicates that during the model training stage, any terminal can only choose to access one of the accessible base stations; The optimization model for subproblem 2 is: C4, C9; Among them, C2″ means that the bandwidth resources allocated by any base station to the cell link during the model aggregation stage cannot exceed the corresponding upper limit; C5″ is the sum of the terminal transmission power accessing any base station during the model aggregation stage and the sub-constraints of the transmission power range of each terminal; C7″ means that during the model aggregation stage, any terminal can only select one base station for uploading.

8. The 6G computing network adaptive splitting federated learning method for distributed AI training services according to claim 1 is characterized by: In step S4, the shortest path search algorithm is used to jointly optimize the model segmentation method, collaborative node selection and multi-domain resource scheduling method in the model training stage and the model aggregation stage, specifically: Step S4-1: allocating initial values ​​of the multi-domain resource allocation method for the model training phase and the model aggregation phase; Assign the same initial value of the multi-domain resource allocation method in the model training phase to each terminal k to eliminate the impact of differences in resource allocation; Assign the same initial value of the multi-domain resource allocation method in the model aggregation phase to each terminal k Step S4-2: Select the shortest path search algorithm to optimize the model segmentation method selection and the collaborative node selection in sub-problem 1; Specifically: Construct a weighted directed graph G for each terminal k in subproblem 1 train =(O train ,E train ); Among them, O train is a node set, is the first-layer node, representing the first layer to the v0th layer of the training model of terminal k; is a layer 2 node, representing base station n x Train the model from layer v1 to layer v2; is a layer 3 node, representing base station n y Train the model from layer v3 to layer v; train is the edge between nodes, e train ∈E train It represents the selection method of collaborative nodes, and its weight represents the transmission and computing delay; Add a pair of virtual start / end vertices k ,e k As the source and destination nodes in the graph, the weight of the edge connecting to other nodes in the graph is 0, and the optimization goal becomes searching for the node from s k To e k The shortest path, all vertices on this path form the corresponding optimal model segmentation method and collaborative node selection; That is in They represent the optimal access base station selection, cooperative base station selection and model segmentation mode of terminal k respectively; The operation flow of the shortest path search algorithm for subproblem 1 is: S4-2-1: Input terminal k transmits the output data of the model layer v to each base station n x The transmission delay Each base station n x Transmit the output data of the model layer v to other base stations n y The transmission delay Processing delay of layer v of terminal k computing model And each base station n x Calculate the processing delay of layer v of the model S4-2-2: Calculate each node in the third layer The segmentation cost And let Indicates a layer 3 relay node The segmentation cost is 0; S4-2-3: Layer 2 nodes According to the starting segmentation layer v1 and base station n x Divide into multiple subsets, and calculate the segmentation cost within each subset S4-2-4: Get the optimal partitioning cost in each subset Corresponding to the termination segmentation layer and corresponding to the third layer base station And let Represents a layer 2 relay node The segmentation cost is 0, and it selects any relay node in layer 3 connect; S4-2-5: Calculate each node in the first layer The segmentation cost S4-2-6: Select the termination segmentation layer that minimizes the segmentation cost And get the corresponding second-layer base station Corresponding to the second layer termination split layer And corresponding to the third layer base station S4-2-7: Output the shortest path to terminal k Based on the optimal model segmentation method and collaborative node selection method Optimize the multi-domain resource allocation method during the model training phase Transform subproblem 1 into a convex optimization problem P2′ as follows: stC1,C2′,C3,C5′; in, and The optimizations between them do not affect each other; Step S4-3: Select the shortest path search algorithm to optimize the model segmentation method selection and collaborative node selection in sub-problem 2.

9. The 6G computing network adaptive splitting federated learning method for distributed AI training services according to claim 8, characterized in that: In step S4-3, the model segmentation method selection and the collaborative node selection in the shortest path search algorithm optimization sub-problem 2 are selected as follows: Construct a weighted directed graph G for each terminal k aggr,tran =(O aggr,tran ,E aggr,tran ), node collection Among them k is the only layer 1 node, representing terminal k; is the second-layer node, representing all terminals; is the layer 3 node, representing all base stations; The edge between nodes aggr,tran ∈E aggr,tran Indicates the selection method of the collaborative node, and its weight represents the transmission delay; add a pair of virtual start / end vertices s k ,e k As the source node and destination node in the graph, the weight of the edge connecting to other nodes in the graph is 0; k To e k All vertices on the shortest path of form the corresponding optimal collaborative node selection method, that is, like but in, They represent the optimal terminal selection and base station selection of terminal k respectively; The operation flow of selecting the shortest path search algorithm for sub-problem 1 is: Step S4-3-1: Input terminal k transmits model parameters to each terminal k x The transmission delay Each terminal k x Transmit the model parameters to each base station n y The transmission delay Step S4-3-2: Calculate each node o in the third layer ny The transmission overhead Step S4-3-3: Calculate each node in the second layer The transmission overhead And let That is, the delay of the terminal transmitting the model parameters to itself is 0; Step S4-3-4: Layer 1 node o k Select the layer 2 node that minimizes the transmission overhead And further get the corresponding third-layer base station Step S4-3-5: Output the shortest path to terminal k Based on the optimal cooperation node selection method Optimize the multi-domain resource allocation method in the model aggregation phase Transform subproblem 2 into a convex optimization problem P3′ as follows: stC2″,C4,C5″; in, and The optimizations do not affect each other.

Citation Information

Patent Citations

  • Method for accelerating convergence of global federated learning model and federated learning system

    CN116416508A

  • Hierarchical node selection method for federated learning in mobile edge environment

    CN117336824A

  • Cost optimization method for splitting federated learning

    CN117521778A

  • Global model optimization method and system based on split federated learning

    CN118540739A

  • Federated learning lower-side cooperative channel adaptive gradient compression method

    WO2024050659A1

Cited By

  • End-side cloud model compression and deployment method based on split learning

    CN119940455A

  • Adaptive split federated learning method for distributed ai training service in 6g computing power network

    WO2025156825A1