Adaptive split federated learning method for distributed ai training service in 6g computing power network

Abstract

The present invention belongs to the technical field of communication networks. Disclosed is an adaptive split federated learning method for a distributed AI training service in a 6G computing power network. The method comprises: establishing a 6G computing power network model, and constructing an adaptive split federated learning model for a distributed AI training service; analyzing the delays of each intelligent terminal in a model training stage and a model aggregation stage; constructing a total service delay minimization optimization problem; and using a shortest-path search algorithm to perform joint optimization on model segmentation modes, cooperative node selection modes and multi-domain resource scheduling modes in the model training stage and the model aggregation stage. In the present invention, by means of joint optimization of a model segmentation mode, node selection and multi-dimensional resource allocation in a multi-base-station scenario, the problem of network congestion caused by a single-base-station learning framework processing the aggregation of a large number of intelligent terminal models is alleviated, and the present invention adapts to the heterogeneous computing power of intelligent terminals and the differentiated channel quality in an actual network, thereby improving the efficiency and generalization capability of model training.

Description

A 6G computing network adaptive splitting federated learning method for distributed AI training business Technical Field

[0001] The present invention belongs to the field of communication network technology, and specifically relates to a 6G computing power network adaptive splitting federated learning method for distributed AI training services. Background Art

[0002] With the standardization and widespread deployment of 5G networks, 6G, targeting 2030 and beyond, will continue to expand upon 5G's three key scenarios: enhanced mobile broadband, low-latency, highly reliable connectivity, and massive machine-type communications. As one of the new key 6G scenarios, inclusive intelligent services leverage the network to provide integrated communications and AI computing power for intelligent services requiring efficient distributed intelligent learning or reasoning. This will serve not only specific applications but also the entire future communications system, improving overall network performance and efficiency. In this scenario, a wide variety of devices within the network will be enabled to participate in the development and use of AI, enabling fast and flexible intelligent services to reach all areas of society.

[0003] However, any single network element, regardless of location, faces challenges such as limited computing power, intelligence, and power consumption. To meet the extremely high communication demands and massive computing power consumption brought about by intelligent services such as AI model training and inference, holographic communications, and immersive cloud XR (eXtended Reality), the computing power, algorithms, data, and other resources and functions in ubiquitously distributed network elements must operate efficiently and collaboratively. In other words, to achieve universal intelligence and connected intelligence, and to ensure the quality of computing services like AI in future networks, 6G networks must transform from traditional information transmission infrastructure to a new type of infrastructure that provides both connectivity and computing services—a computing network. This network can allocate and flexibly schedule computing, storage, and network resources across the cloud, edge, and end based on business needs, achieving deep convergence of computing. By connecting ubiquitous computing power and leveraging the advantages of computing clusters, it can build efficient, energy-efficient, secure, and flexible AI services and an open ecosystem.

[0004] When developing AI services in 6G computing networks, they are primarily implemented through distributed learning technologies such as federated learning (FL) and split learning (SL). FL's advantage lies in its parallel training approach across devices, fully considering data privacy. However, its training process incurs significant computational and communication overhead, as each device must train and transmit a complete model. In contrast, SL partitions the model and utilizes multiple nodes for collaborative training, which can reduce the computational overhead of smart terminals compared to FL. However, SL still has significant communication overhead. Therefore, some research has combined FL and SL, employing both parallel training and model partitioning and collaborative training, effectively improving training efficiency and accuracy to a certain extent. However, most current research focuses solely on the FL or SL framework, optimizing different variables to improve model training accuracy or efficiency. These optimization variables are limited, and resource allocation typically focuses solely on optimizing communication or computing resources. Of the few studies that have combined FL and SL, few consider multi-base station scenarios or the joint optimization of model partitioning with multi-dimensional communication and computing resource allocation in these scenarios, failing to fully meet practical application requirements.

[0005] Therefore, how to solve the problem of reasonably allocating communication and calculating multi-dimensional resources is the technical problem that the present invention aims to solve. Summary of the Invention

[0006] The purpose of the present invention is to provide a 6G computing power network adaptive splitting federated learning method for distributed AI training business, so as to solve the problems raised in the above background technology and the shortcomings of the existing technology.

[0007] The object of the present invention is to achieve this by providing a 6G computing network adaptive splitting federated learning method for distributed AI training services, characterized in that the method comprises the following steps:

[0008] Step S1: Establish a 6G computing network model and build an adaptive split federated learning model for distributed AI training services;

[0009] Step S2: Define a wireless communication model and analyze the delay of each smart terminal in the model training phase and the model aggregation phase;

[0010] Step S2-1: Analyze the latency of each smart terminal during the model training phase;

[0011] Step S2-2: Analyze the time delay of each smart terminal during the model aggregation phase;

[0012] Step S3: Construct an optimization problem for minimizing the total service delay. This problem is converted into two subproblems: Subproblem 1 and Subproblem 2. Subproblem 1 is to optimize the training delay of the model training phase in each training iteration, and Subproblem 2 is to optimize the transmission delay of the model aggregation phase.

[0013] Step S4: Use the shortest path search algorithm to jointly optimize the model segmentation method, collaborative node selection, and multi-domain resource scheduling method in the model training phase and the model aggregation phase.

[0014] Compared with the prior art, the present invention has the following improvements and advantages:

[0015] 1. By considering the joint optimization of model segmentation, node selection, and multi-dimensional resource allocation in multi-base station scenarios, the network congestion problem that may be caused by the aggregation of a large number of smart terminal models in a single base station learning framework can be effectively alleviated. At the same time, multi-dimensional resource allocation and flexible model segmentation and node selection can better adapt to the heterogeneous computing power of smart terminals and differentiated channel quality in actual networks, improving the efficiency and generalization ability of model training.

[0016] 2. Convert the optimization problem of minimizing the total service latency into two 2-subproblems, so that the latency of each training iteration can be minimized, simplifying the complexity of obtaining the optimal solution, shortening the optimization time, and improving the accuracy and scalability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG1 is an overall flow chart of the method of the present invention.

[0018] Figure 2 is a flow chart of training mode 2 smart terminal training.

[0019] Figure 3 is a flow chart of training mode 3 smart terminal training.

[0020] Figure 4 is a schematic diagram of the ResNet18 neural network model structure.

[0021] Figure 5 shows the optimal transmission topology during the model training phase.

[0022] Figure 6 is a flowchart of the optimal transmission topology training in the model aggregation stage.

[0023] FIG7 is a graph showing the variation of the model accuracy with the number of iterations in the method of the present invention. DETAILED DESCRIPTION

[0024] The present invention is further summarized below with reference to the accompanying drawings.

[0025] As shown in Figure 1, a 6G computing network adaptive split federated learning method for distributed AI training services includes the following steps:

[0026] Step S1: Establish a 6G computing network model and build an adaptive split federated learning model for distributed AI training services. The adaptive split federated learning model is applicable to DNN models, such as LeNet, AlexNet, VGG16, and ResNet18, as shown in Figure 4.

[0027] The 6G computing network model includes a central cloud server, N base stations and K smart terminals. The base stations are represented by N = {1, 2, ..., N}. Each base station n is equipped with a mobile edge computing server. The mobile edge computing server has computing resources and storage resources. The computing resources are represented by F n Indicates that storage resources are represented by M n Indicates that the physical distance between the central cloud server and the smart terminal is relatively far, and the computing and storage resources are sufficient, which are only used for edge model aggregation and global model distribution in the model aggregation stage.

[0028] The intelligent terminals are represented by a set K = {1, 2, ..., K}, and the computing resources and storage resources owned by each intelligent terminal k are represented by F k and M k express.

[0029] The local dataset you own is represented by D k ={(x i ,y i )|i=1,...,D k} indicates that D k represents the number of samples, and Represent the corresponding sample input features and output labels, Q represents the dimension of the sample input features, and let Indicates the total dataset size of all smart terminals.

[0030] The relationship between the computing resources and storage resources of the intelligent terminal and the computing resources and storage resources of the base station is F k < <F n ,M k < <M n ,

[0031] The adaptive split federated learning model adopts a DNN model structure; any DNN model structure is acceptable; the parameters of the DNN model, such as input, output, number of hidden layers, and number of neurons, are known. The model parameters are represented as w, and the total number of model layers is represented as V.

[0032] S101. The central cloud server initializes the model parameter w0 and sends it to all smart terminals; initializes the training iteration round t=1 and the maximum training iteration round T; the maximum training iteration round T is related to the learning accuracy requirements of the distributed AI training business. After each iteration, the model accuracy θ(t) of each smart terminal can be calculated using the verification data set to obtain the average model accuracy θ(t), and T must meet where θ th Indicates the model accuracy threshold for distributed AI training services.

[0033] S102. If t>T, then enter step S109; if t≤T, then enter the model training stage, and solve the training method and resource allocation method of each smart terminal k according to the proposed algorithm; among them, the training methods are divided into three types. A part of the smart terminals with more computing resources and storage resources can use local data sets to independently complete model training (as shown in training mode 1), and another part of the smart terminals with limited computing resources and storage resources need to complete model training through edge-end collaboration (as shown in training mode 2), and the remaining smart terminals with fewer computing resources and storage resources need to complete model training through edge-end collaboration and edge-edge collaboration (as shown in training mode 3).

[0034] S103, each intelligent terminal k completes the local model independently or in collaboration with the base station according to the obtained training method and resource allocation method Update calculation of .

[0035] S104. Enter the model aggregation phase and solve the transmission mode and resource allocation mode of each smart terminal k according to the proposed algorithm; among them, there are two transmission modes. Each smart terminal transmits the updated local model to the neighboring smart terminal through the D2D link based on its own channel quality status, and then relays it to the selected base station (as shown in transmission mode 1), or directly uploads it to the selected base station through the cell link (as shown in transmission training mode 2).

[0036] Each smart terminal k directly transmits the local model through the D2D link relay The transmission mode for uploading to the selected base station n for aggregation is transmission mode 1. Each smart terminal k directly transmits the local model through the cell link. The transmission mode for uploading to the selected base station n for aggregation is transmission training mode 2.

[0037] S105, each intelligent terminal k updates the local model according to the obtained transmission mode and resource allocation mode Transmit to selected base station;

[0038] S106. Each base station n aggregates the local model through the FedAvg algorithm to obtain the edge model And send it to all smart terminals that access this base station through the cell link or D2D link relay; among them, K n represents the set of intelligent terminals connected to base station n, Indicates the total data set size of smart terminals that access base station n through cell links or D2D link relays;

[0039] S107, if t% T_GLOBAL (global aggregation round constant) == 0, then each base station n will edge model Upload to the central cloud server, the central cloud server aggregates the edge model through the FedAvg algorithm to obtain the global model And send it to all smart terminals; otherwise, go directly to step S108;

[0040] S108, set t=t+1, and return to step S102;

[0041] S109, output the final model parameter w T .

[0042] Step S2: Define a wireless communication model and analyze the delay of each smart terminal in the model training stage and the model aggregation stage.

[0043] Define the wireless communication model, specifically:

[0044] The total bandwidth used by the cell link, cooperative link and D2D link is represented as B Cellular ,B B2B and B D2D ,The frequency bands used by cell links, cooperative links and D2D links are different;

[0045] The orthogonal channel resources allocated to each base station n are expressed as b n express, satisfy

[0046] The signal-to-noise ratio threshold for maintaining communication between intelligent terminal k and base station n is expressed as When the uplink signal-to-noise ratio γ between the intelligent terminal k and the base station n k,n Greater than or equal to the threshold When , it is considered that the intelligent terminal k is within the coverage of the base station n, and the intelligent terminal k communicates directly with the base station n.

[0047] Introducing binary variables Indicator, uplink signal-to-noise ratio

[0048] Among them, p k represents the transmission power of intelligent terminal k, and its maximum value is h k,nrepresents the cell link channel gain from intelligent terminal k to base station n, and N0 represents the noise power.

[0049] Introducing binary variables Used to indicate whether base station n can communicate directly with another base station n′, Indicates the signal-to-noise ratio threshold for maintaining communication between base stations, the signal-to-noise ratio between base station n and another base station n′ p n represents the transmission power of base station n, and its maximum value is h n,n′ represents the cooperative link channel gain from base station n to another base station n′.

[0050] Downlink signal-to-noise ratio between base station n and smart terminal k The signal-to-noise ratio between intelligent terminal k and another intelligent terminal k′ where h k,k′ represents the D2D link channel gain from smart terminal k to another smart terminal k′;.

[0051] According to Shannon's formula, the uplink rate of each intelligent terminal k in the cell link is R k,n , the cell link downlink rate R of each base station n n,k , the achievable rate R of the cooperative link of each base station n n,n′ and the D2D link achievable rate R of each smart terminal k k,k′ ; R k,n =b k,n log2(1+γ k,n ); R n,k =b k,n log2(1+γ n,k ); R n,n′ =b n,n′ log2(1+γ n,n′ ); R k,k′ =b k,k′ log2(1+γ k,k′ );

[0052] Among them, b k,n represents the bandwidth resources allocated to the communication between the intelligent terminal k and the base station n, b n,n′ represents the bandwidth resources allocated to base station n for communication with another base station n′, b k,k′ It represents the bandwidth resources allocated to the intelligent terminal k for communication with another intelligent terminal k′.

[0053] All base stations are connected to the central cloud server via different high-bandwidth backhaul links. However, due to factors such as multi-user scheduling and complex routing algorithms, their transmission delay is difficult to model. Therefore, it is assumed that the backhaul link resource allocation strategy and routing algorithm of each base station are known, so that the achievable rate of its backhaul link is fixed.

[0054] Step S2-1: Analyze the latency of each smart terminal during the model training phase, specifically:

[0055] Step S2-1-1: Determine the local loss function of the intelligent terminal k and the segmentation layer set of the DNN model during the model training phase.

[0056] The local loss function of each intelligent terminal k Among them, L(w;x i ,y i ) represents the sample loss function, x i Input features for the sample, y i Output label for the sample; w is the model parameter.

[0057] Use V = {2, 3, ..., V} to represent the set of selectable segmentation layers in the DNN model, and use Indicates the model training segmentation layer selection for smart terminal k.

[0058] Step S2-1-2: Divide the model training delay of each smart terminal k Divide the training modes and determine the update method for each training mode.

[0059] The training modes include training mode 1, training mode 2, and training mode 3. The training mode that performs model training completely locally is training mode 1, the training mode that performs model training through edge-end collaboration is training mode 2, and the training mode that performs model training through edge-end collaboration and edge-edge collaboration is training mode 3.

[0060] The subset of intelligent terminals in training mode 1 is The local model parameters of smart terminal k in the tth iteration are expressed as Each intelligent terminal k updates the local model parameters by the stochastic gradient descent method:

[0061] Among them, η l Indicates the learning rate of the model in local training, represents the local model gradient calculated by intelligent terminal k; is the partial derivative vector of the local loss function of smart terminal k with respect to the model parameters;

[0062] Training mode 1 only includes local model training delay, local model training delay

[0063] in, Represents the computational effort of the forward and backward propagation processes for processing a data sample.

[0064] The subset of intelligent terminals in training mode 2 is Local model parameters of smart terminal k in the tth iteration Expressed as in and They represent the training mode 2 terminal side model and the training mode 2 base station side model respectively.

[0065] In training mode 2, the terminal side model of training mode 2 needs to be trained, and the delay in, Indicates the computational effort of the forward propagation process of the training mode 2 terminal-side model for processing one data sample.

[0066] Training mode 2: Segmentation layer output data of the terminal side model It is obtained by the following formula:

[0067] Where f(x;w) represents the mapping function between input x and output s under given model parameters w;

[0068] Will and y i Transmission delay to base station n Among them, the binary variable Used to indicate whether terminal k selects base station n for access, if yes, it is 1, otherwise 0; Indicates the access base station subset, Indicates the split layer data size of one data sample.

[0069] The terminal side model of training mode 2 uses SGD to update the terminal side model of training mode 2. The update formula is:

[0070] Latency

[0071] in, Indicates the computational amount of the back propagation process of the training mode 2 terminal side model for processing a data sample; the intelligent terminal k will update the training mode 2 terminal side model and training mode 2 base station side model Stitch them together to get a new local model

[0072] The training mode 2 base station side model needs to be trained, and the training and update delay of the training mode 2 base station side model in It represents the computational amount of the forward propagation and back propagation process of the training mode 2 base station side model for processing one data sample, Represents the computing resources allocated by base station n to smart terminal k.

[0073] Segmentation layer gradient and updated training mode 2 base station side model parameters are transmitted back to the smart terminal k together, and the delay of this part in Indicates the size of the segmentation layer gradient data for a data sample.

[0074] Prediction results on the base station side It is obtained by the following formula:

[0075] According to the prediction results and the true value label y i , calculate the average gradient of the loss function The base station side model updates parameters through SGD, and the update formula is:

[0076] Among them, η e Represents the learning rate of the model trained on the base station side; according to the chain rule of gradient calculation, the model parameters will be updated layer by layer from the output layer to the segmentation layer, and the segmentation layer gradient will be obtained

[0077] The subset of intelligent terminals in training mode 3 is Local model parameters of smart terminal k in the tth iteration Expressed as in, and Represent the terminal side model, access base station side model and cooperative base station side model respectively;

[0078] Train the terminal side model of training mode 3, delay in Represents the computational amount of the forward propagation process of the terminal side model to process a data sample; the calculated terminal side model segmentation layer output data and y i Transmitted to access base station n, its delay in Represents a segmentation layer for a data sample Data size.

[0079] Train the access base station side model of training mode 3, delay in, It represents the computational effort of the forward propagation process of the access base station side model to process one data sample.

[0080] The split layer output results of the access base station side model It is obtained by the following formula:

[0081] Will and y i Transmission delay to cooperative base station n′ The binary variables Used to indicate whether terminal k selects base station n′ to collaborate with access base station n to train the model. If yes, it is 1, otherwise 0; represents the subset of cooperative base stations, Represents a segmentation layer for a data sample Data size.

[0082] Training and updating of the cooperative base station side model of training mode 3, delay in, represents the computational effort of the forward and backward propagation processes of the collaborative base station side model to process a data sample, It represents the computing resources allocated by the cooperative base station n′ to the terminal k.

[0083] Segmentation layer gradient and updated cooperative base station side model parameters are transmitted back to the access base station n together, and the delay of this part in, Represents a segmentation layer for a data sample The amount of gradient data.

[0084] Prediction results on the collaborative base station side It is obtained by the following formula:

[0085] According to the received segmentation layer gradient The access base station side model uses SGD to update parameters until the segmentation layer. The delay of this part in Indicates the computational effort of the back-propagation process of the access base station side model to process one data sample.

[0086] Access base station n will split the layer gradient Updated access base station side model And the collaborative base station side model Transmitted together to the corresponding terminal k, the delay in, Represents a segmentation layer for a data sample The amount of gradient data.

[0087] The terminal side model is also updated using SGD, and its delay in Indicates the computational cost of the back propagation process of the terminal side model to process a data sample. Finally, terminal k updates the terminal side model Access base station side model and collaborative base station side model Stitch them together to get a new local model

[0088] In step S2-2, the delay of each smart terminal in the model aggregation stage is analyzed, specifically:

[0089] In the model aggregation stage, each smart terminal k directly transmits the local model to the network through the D2D link relay or cell link. Transmission delay of uploading to selected base station n for aggregation Expressed as:

[0090] Among them, the binary variable Used to indicate whether the intelligent terminal k selects base station n for model upload, if yes, it is 1, otherwise 0; binary variable Used to indicate whether smart terminal k selects smart terminal k' as a relay to access base station n. If yes, it is 1, otherwise it is 0; smart terminal subset represents all directly connected smart terminals within the coverage area of ​​base station n, M w Indicates the amount of data for the model parameters.

[0091] Step S3: Construct the optimization problem of minimizing the total service delay;

[0092] The optimization problem of minimizing the total service latency is transformed into two sub-problems: Sub-problem 1 and Sub-problem 2. Sub-problem 1 is to optimize the training latency of the model training phase in each training iteration, and Sub-problem 2 is to optimize the transmission latency of the model aggregation phase.

[0093] The optimization problem of minimizing the total service delay is:

[0094] Among them, the decision variables include the model split method Collaborative node selection Multi-domain resource allocation method in the model training phase / model aggregation phase

[0095] C1 indicates that the total computing resources allocated by any base station to smart terminals that select training modes 1 and 3 cannot exceed the base station upper limit; C2 indicates that the bandwidth resources allocated by any base station to the cell link cannot exceed the corresponding upper limit, and C3 and C4 represent the corresponding collaborative link and D2D link bandwidth resource constraints respectively; C5 indicates that the sum of the transmission power of smart terminals accessing any base station and the sum of the transmission power of all base stations have an upper limit, and the transmission power of any smart terminal or base station cannot exceed its maximum transmission power; C6 is a constraint on the model segmentation method of each smart terminal, C7 indicates that any smart terminal can only choose one base station to access; C8 indicates that in the model training stage, for any smart terminal, in addition to accessing the base station, only one collaborative base station can be selected; C9 indicates that in the model aggregation stage, for any smart terminal, if it needs to access the base station through a relay, only one smart terminal within the coverage area of ​​the base station can be selected as a relay node.

[0096] The optimization model for subproblem 1 is:

[0097] C1, C3, C6, C8;

[0098] Among them, C2′ indicates that the bandwidth resources allocated by any base station to the cell link during the model training phase cannot exceed the corresponding upper limit; C5′ is the sub-constraint of the sum of the terminal transmit power accessing any base station during the model training phase, the total base station transmit power, and the transmit power range of each terminal; C7′ indicates that during the model training phase, any terminal can only choose to access one of the accessible base stations;

[0099] The optimization model for subproblem 2 is:

[0100] C4, C9;

[0101] Among them, C2″ indicates that the bandwidth resources allocated by any base station to the cell link during the model aggregation stage cannot exceed the corresponding upper limit; C5″ is the sum of the terminal transmission power accessing any base station during the model aggregation stage and the sub-constraint of the transmission power range of each terminal; C7″ indicates that during the model aggregation stage, any terminal can only select one base station for uploading.

[0102] Step S4: Use the shortest path search algorithm to jointly optimize the model segmentation method, collaborative node selection, and multi-domain resource scheduling method in the model training phase and the model aggregation phase. Specifically:

[0103] Step S4-1: allocating initial values ​​of the multi-domain resource allocation method for the model training phase and the model aggregation phase;

[0104] Assign the same initial value of the multi-domain resource allocation method in the model training phase to each smart terminal k To eliminate the impact of differences in resource allocation.

[0105] Assign the same initial value of the multi-domain resource allocation method in the model aggregation phase to each smart terminal k

[0106] Step S4-2: Select the shortest path search algorithm to optimize the model segmentation method selection and collaborative node selection in sub-problem 1;

[0107] Specifically:

[0108] Construct a weighted directed graph G for each intelligent terminal k in subproblem 1 train =(O train ,E train );

[0109] Among them, O train is a node set, It is the first-layer node, representing the first to v0 layers of the intelligent terminal k training model; is a layer 2 node, representing base station n x Train the model from layer v1 to layer v2; It is a layer 3 node, representing base station n y Training model layers v3 to v; e train is the edge between nodes, e train ∈E train It represents the selection method of collaborative nodes, and its weight represents the transmission and computing delay.

[0110] Add a pair of virtual start / end vertices s k ,e k As the source and destination nodes in the graph, the weight of the edges connecting to other nodes in the graph is 0, and the optimization goal becomes searching for the nodes from s k to e k The shortest path, all vertices on this path form the corresponding optimal model segmentation method and collaborative node selection.

[0111] That is in They represent the optimal access base station selection, cooperative base station selection and model segmentation method for intelligent terminal k respectively.

[0112] As shown in Figure 2, the operation flow of selecting the short path search algorithm for sub-problem 1 is as follows:

[0113] S4-2-1: Input intelligent terminal k transmits the output data of the model layer v to each base station n x Transmission delay Each base station n x Transmit the output data of the model layer v to other base stations n y Transmission delay Processing delay of layer v of the computing model of intelligent terminal k And each base station n x Calculate the processing delay of layer v of the model

[0114] S4-2-2: Calculate each node in the third layer Segmentation overhead And let Indicates a layer 3 relay node The segmentation cost is 0.

[0115] S4-2-3: Layer 2 nodes According to the starting segmentation layer v1 and base station n x Divide into multiple subsets, and calculate the segmentation cost within each subset first

[0116] S4-2-4: Get the optimal partitioning cost in each subset Corresponding termination segmentation layer and corresponding Layer 3 base station And let Represents a layer 2 relay node The segmentation cost is 0, which selects any relay node in the third layer connect.

[0117] S4-2-5: Calculate each node in the first layer Segmentation overhead

[0118] S4-2-6: Select the termination segmentation layer that minimizes the segmentation overhead And get the corresponding second-layer base station Corresponding to the second layer termination split layer And the corresponding 3rd layer base station

[0119] S4-2-7: Output the shortest path of smart terminal k

[0120] Based on the optimal model segmentation method and collaborative node selection method Optimize multi-domain resource allocation during model training Transform subproblem 1 into a convex optimization problem P2′ as follows:

[0121] stC1,C2′,C3,C5′;

[0122] in, and The optimizations do not affect each other.

[0123] Step S4-3: Select the shortest path search algorithm to optimize the model segmentation method and collaborative node selection in sub-problem 2, specifically:

[0124] Construct a weighted directed graph G for each intelligent terminal k aggr,tran =(O aggr,tran ,E aggr,tran ), node collection Among them k is the only first-layer node, representing smart terminal k; It is the second layer node, representing all intelligent terminals; It is a layer 3 node, representing all base stations.

[0125] The edge between nodes aggr,tran ∈E aggr,tran Indicates the selection method of the collaborative node, and its weight represents the transmission delay; add a pair of virtual start / end vertices s k ,e k As the source node and destination node in the graph, the weight of the edge connecting to other nodes in the graph is 0; k to e k All vertices on the shortest path form the corresponding optimal collaborative node selection method, that is, like but in, They represent the optimal intelligent terminal selection and base station selection for intelligent terminal k respectively.

[0126] The operation flow of selecting the shortest path search algorithm for sub-problem 1 is:

[0127] As shown in Figure 3, step S4-3-1: input smart terminal k to transmit model parameters to each smart terminal k x Transmission delay Each intelligent terminal k x Transmit the model parameters to each base station n y Transmission delay

[0128] Step S4-3-2: Calculate each node in the third layer Transmission overhead

[0129] Step S4-3-3: Calculate each node in the second layer Transmission overhead And let That is, the delay of the intelligent terminal transmitting the model parameters to itself is 0.

[0130] Step S4-3-4: Layer 1 node o k Select the layer 2 node that minimizes the transmission overhead And further get the corresponding third-layer base station

[0131] Step S4-3-5: Output the shortest path s of smart terminal k k ,o k , e k .

[0132] Based on the optimal collaborative node selection method Optimize the multi-domain resource allocation method during the model aggregation phase Transform subproblem 2 into a convex optimization problem P3′ as follows:

[0133] stC2″,C4,C5″;

[0134] in, and The optimizations do not affect each other.

[0135] In order to verify the effectiveness of the adaptive split federated learning model of the present invention, a verification was conducted. According to the settings of relevant simulation parameters, in the model training stage and the model aggregation stage, in order to more prominently and clearly demonstrate the collaboration between the edge and the smart terminal; as shown in Figure 5, in the model training stage, smart terminals 2 and 4 chose to perform only local training due to their sufficient computing power. Smart terminals 1, 3, 5, and 6 chose to collaborate with dual base stations, and because the computing power of smart terminal 1 is relatively high, the segmentation layer of its smart terminal side model is selected as the 4th layer, while the segmentation layer of the smart terminal side model of smart terminals 3, 5, and 6 is selected as the 1st layer. Taking into account their own computing power and the channel conditions between the base stations that can communicate directly, smart terminals 7, 8, and 9 chose a single base station collaboration method. As shown in Figure 6, in the model aggregation stage, in order to speed up the transmission speed of the model parameters, smart terminals 1, 3, and 9 selected other smart terminals as relays for transmission.

[0136] Furthermore, the present invention verifies the accuracy of the proposed learning framework based on the optimal model segmentation method for each intelligent terminal, the selection of collaborative nodes, and the multi-domain resource allocation method in the model training stage and the aggregation stage, and compares it with some benchmark algorithms. The simulation results are shown in Figure 7. The algorithms involved in the comparison include: centralized learning in which the entire ResNet18 neural network is trained using the HAM10000 dataset through only one base station; split learning (SL) in which serial collaborative training is performed through one base station and nine intelligent terminals, in which the segmentation layer of the intelligent terminal side model and the service side model is fixed to the second layer; federated learning (FL) in which one base station and nine intelligent terminals perform parallel collaborative training, in which each intelligent terminal trains the entire neural network on its local dataset, and the base station aggregates and updates the model; and split federated learning (FL) in which the above two methods are combined, in which one base station and nine intelligent terminals perform parallel collaborative training, in which each intelligent terminal is only trained to the second layer of the model, and the remaining part is trained in parallel through the base station, and the intelligent terminal side model and the base station side model are aggregated and updated respectively after each round of iteration. In addition, the test accuracy is the average of the model accuracy calculated by all smart terminals on their respective test sets, and the batch size used in the training process is 256.

[0137] As shown in Figure 7, the proposed algorithm gradually converged over 100 iterations, achieving a final test accuracy rate of approximately 75%, similar to the convergence trends of several other algorithms. Notably, in the initial phase (iterations 0 to 20), the test accuracy of the proposed algorithm experienced a period of decline followed by an increase. This phenomenon is primarily attributed to differences in the model segmentation methods, model aggregation nodes, and data distribution used by different smart terminals. Therefore, after completing edge model aggregation and updating model parameters in the early stages of the iteration, these inadequately adjusted parameters were unable to adapt well to the dataset characteristics of certain smart terminals, resulting in an initial decrease in test accuracy. As iterations progressed, through continuous optimization and adjustment, the algorithm gradually overcame these initial challenges, allowing the test accuracy rate to recover and stabilize. In contrast, centralized learning exhibited higher test accuracy in the early stages of the iterations because it trained only on a single node, thus avoiding the issues associated with dataset variability and model aggregation. SL achieved similarly high initial accuracy rates to centralized learning because it trained on a smart terminal-by-device basis, similarly avoiding these issues. However, both approaches come at the expense of flexibility and result in longer training latency.

[0138] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. An adaptive split federated learning method for 6G computing power network in the context of distributed AI training services, characterized in that: The method includes the following steps: Step S1: Establish a 6G computing power network model and construct an adaptive split federated learning model for distributed AI training services; Step S2: Define a wireless communication model and analyze the latency of each intelligent terminal in the model training stage and the model aggregation stage; Step S2-1: Analyze the latency of each intelligent terminal in the model training stage; Step S2-2: Analyze the latency of each intelligent terminal in the model aggregation stage; Step S3: Construct an optimization problem for minimizing the total service latency; transform the optimization problem for minimizing the total service latency into two sub-problems, where the two sub-problems include sub-problem 1 and sub-problem 2. Sub-problem 1 is the problem of optimizing the training latency in the model training stage of each round of training iteration, and sub-problem 2 is the problem of optimizing the transmission latency in the model aggregation stage; Step S4: Use the shortest path search algorithm to jointly optimize the model splitting method, cooperative node selection, and multi-domain resource scheduling method in the model training stage and the model aggregation stage.

2. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 1, characterized in that: The 6G computing power network model includes a central cloud server, N base stations, and K intelligent terminals. The set of base stations is represented by N = {1, 2,..., N}. Each base station n is equipped with a mobile edge computing server, and the mobile edge computing server has computing resources and storage resources. The computing resources are represented by F n and the storage resources are represented by M n respectively. The set of intelligent terminals is represented by the set \(K = \{1, 2, \ldots, K\}\), and the computing resources and storage resources owned by each intelligent terminal \(k\) are represented by \(F\) k and \(M\) k respectively. The owned local dataset is denoted by D k ={(x i ,y i )|i = 1,...,D k}, where D k denotes the number of samples, and respectively represent the corresponding sample input features and output labels, Q represents the dimension of the sample input features, and let Denote the total dataset size of all intelligent terminals; The size relationship between the computing resources and storage resources of the intelligent terminal and those of the base station is 3. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 1, characterized in that: In step S2-1, the analysis of the latency of each intelligent terminal in the model training stage is specifically as follows: Step S2-1-1: Determine the local loss function of intelligent terminal k and the set of split layers of the DNN model in the model training stage; Local loss function of each intelligent terminal k Among them, L(w; x i , y i ) represents the sample loss function, x i is the sample input feature, y i is the sample output label; w is the model parameter; Let \(V = \{2, 3,\cdots, V\}\) represent the set of selectable split layers in the DNN model, and use Denote the selection of the model training split layer of terminal k; Step S2-1-2: Divide the model training latency of each terminal k Divide the training mode and determine the update method for each training mode; The training modes include training mode 1, training mode 2, and training mode 3. The training mode where the model training is completely performed locally is training mode 1, the training mode where the model training is performed through edge-terminal cooperation is training mode 2, and the training mode where the model training is performed through edge-terminal cooperation and edge-edge cooperation is training mode 3; The subset of intelligent terminals in training mode 1 is The local model parameters of the intelligent terminal k in the t-th round of iteration are denoted as Each intelligent terminal k updates its local model parameters by the stochastic gradient descent method: Among them, η l represents the learning rate of the model during local training, Represents the local model gradient calculated by the intelligent terminal k; Is the partial derivative vector of the local loss function of intelligent terminal k with respect to the model parameters; Training mode 1 only includes the local model training latency, and the local model training latency Among them, Denote the amount of computation for the forward and backward propagation processes of processing one data sample.

4. The adaptive split federated learning method for a 6G computing power network oriented to distributed AI training services according to claim 3, wherein: The subset of intelligent terminals in the training mode 2 is The local model parameters of intelligent terminal k in the t-th round of iteration are denoted as where and respectively represent the terminal-side model and the base-station-side model of the training mode 2; In training mode 2, it is necessary to train the terminal-side model of training mode 2, and the latency of the terminal-side model of training mode 2 Among them, Denote the amount of computation for the forward propagation process of the terminal-side model in training mode 2 for processing one data sample; Output data of the segmentation layer of the terminal-side model in training mode 2 Obtained by the following formula: Among them, f(x; w) represents the mapping function between the input x and the output s under the given model parameters w; Will and y i The time delay transmitted to base station n Among them, the binary variable Used to indicate whether the terminal k selects the base station n for access, 1 if yes, otherwise 0; Indicates a subset of access base stations, Denote the data size of the split layer of one data sample; In training mode 2, the model on the terminal side uses SGD to update the model on the terminal side in training mode 2, and the update formula is as follows: Time delay Among them, Represents the computational amount of the backpropagation process of the terminal-side model in training mode 2 for processing a data sample; The intelligent terminal k will update the terminal-side model of training mode 2 And the base station side model in the training mode 2 Stitch them together to obtain a new local model It is necessary to train the base station side model in training mode 2, and the time delay of training and updating the base station side model in training mode 2 Among them Indicates the computational volume of the forward and backward propagation processes of the base station side model in training mode 2 for processing a data sample. Denote the computing power resources allocated by base station n to intelligent terminal k for use; Partition layer gradient and updated training mode 2 base station side model parameters is transmitted back to the smart terminal k together, and the latency of this part Among them Denote the size of the split layer gradient data volume of one data sample; Prediction results on the base station side Obtained by the following formula: According to the prediction results and the true value label y i , calculate the average gradient of the loss function The base station side model updates parameters through SGD, and the update formula is: Among them, η e represents the learning rate at which the model is trained on the base station side; according to the chain rule of gradient calculation, the model parameters will be updated layer by layer in reverse from the output layer until the segmentation layer, and the segmentation layer gradient will be obtained 5. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 3, characterized in that: The subset of intelligent terminals in the training mode 3 is The local model parameters of intelligent terminal k in the t-th round of iteration are denoted as where and represent the terminal-side model, the access base station-side model, and the cooperative base station-side model respectively; Train the terminal-side model in training mode 3, and its latency Among them Represents the computational complexity of the forward propagation process of the terminal-side model for processing a data sample; the output data of the segmentation layer of the terminal-side model obtained by calculation and y i is transmitted to access base station n, and its delay Among them Split layer representing a data sample Data size; Train the access base station side model for training mode 3, latency Among them, Denote the amount of computation for the forward propagation process of the access base station-side model for processing one data sample; The output result of the segmentation layer accessing the base station side model Obtained by the following formula: Will and y i Delay in transmitting to the cooperating base station n′ Among them, the binary variable Used to indicate whether terminal k selects base station n' to cooperate with access base station n to train the model. If so, it is 1; otherwise, it is 0; Denote a subset of cooperative base stations, A split layer representing a data sample Data size; Train and update the collaborative base station side model for training mode 3, latency Among them, Indicates the computational amount of the forward and backward propagation processes of the collaborative base station side model for processing a data sample, Denote the computing power resources allocated by cooperative base station n′ to terminal k for use; Partition layer gradient and the updated cooperative base station side model parameters is transmitted back to access base station n together, and the delay of this part Among them, A segmentation layer representing a data sample Gradient data volume size; Prediction results on the collaborative base station side Obtained by the following formula: Based on the received splitting layer gradient The access base station side model uses SGD to update parameters until the segmentation layer, and the latency of this part Among them Denote the amount of computation for the backward propagation process of the access base station-side model for processing one data sample; Access base station n will divide the layer gradient Updated access base station side model and the collaborative base station side model Transmitted to the corresponding terminal k together, with its delay Among them, A split layer representing a data sample Gradient data volume size; The model on the terminal side is also updated using SGD, and its latency Among them Indicates the computational amount of the backpropagation process of the terminal-side model for processing a data sample. Finally, terminal k will update the terminal-side model Access base station side model And the collaborative base station side model Stitch them together to obtain a new local model 6. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 1, wherein: In step S2-2, the analysis of the latency of each intelligent terminal in the model aggregation stage is specifically as follows: In the model aggregation stage, each terminal k relays the local model via the D2D link or directly through the cell link Transmission delay for uploading to the selected base station n for aggregation Expressed as: Among them, the binary variable Used to indicate whether terminal k selects base station n for model upload, which is 1 if so, otherwise 0; binary variable Used to indicate whether terminal k selects terminal k′ as a relay to access base station n, if so it is 1, otherwise it is 0; the subset of terminals Denote all directly connected terminals within the coverage of base station n, M w Denote the data volume size of model parameters.

7. An adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 1, characterized in that: The optimization problem of minimizing the total service delay is as follows: Among them, the decision variables include the model segmentation method Collaborative Node Selection Multi-domain resource allocation methods in the model training phase / model aggregation phase C1 represents that the total computing resources allocated by any base station to the terminals selecting training modes 2 and 3 cannot exceed the upper limit of the base station; C2 represents that the bandwidth resources allocated by any base station to the cell link cannot exceed the corresponding upper limit, and C3 and C4 respectively represent the bandwidth resource constraints of the corresponding cooperative link and D2D link; C5 represents that there is an upper limit on the sum of the transmission powers of the terminals accessing any base station and the sum of the transmission powers of all base stations, and the transmission power of any terminal or base station cannot exceed its maximum transmission power; C6 is a constraint on the model splitting method for each terminal, C7 represents that any terminal can only select one base station to access; C8 represents that during the model training stage, for any terminal, in addition to accessing the base station, it can only select one cooperative base station; C9 represents that during the model aggregation stage, for any terminal, if it needs to access the base station through a relay, it can only select one terminal within the coverage of the base station as the relay node; The optimized model for sub-problem 1 is as follows: C1, C3, C6, C8; Among them, C2' represents that the bandwidth resources allocated by any base station to the cell link during the model training stage cannot exceed the corresponding upper limit; C5' is a sub-constraint on the sum of the transmission powers of the terminals accessing any base station, the total transmission power of the base stations, and the transmission power range of each terminal during the model training stage; C7' represents that any terminal can only select one of the accessible base stations to access during the model training stage; The optimized model for sub-problem 2 is as follows: C4, C9; Among them, C2″ represents that the bandwidth resources allocated by any base station to the cell link during the model aggregation stage cannot exceed the corresponding upper limit; C5″ is a sub-constraint on the sum of the transmission powers of the terminals accessing any base station and the transmission power range of each terminal during the model aggregation stage; C7″ represents that any terminal can only select one base station for uploading during the model aggregation stage.

8. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 1, characterized in that: In step S4, the shortest path search algorithm is used to jointly optimize the model splitting method, cooperative node selection, and multi-domain resource scheduling method during the model training stage and the model aggregation stage. Specifically: Step S4-1: Assign initial values to the multi-domain resource allocation methods during the model training stage and the model aggregation stage; Assign the same initial value of the multi-domain resource allocation method in the model training phase to each terminal k To eliminate the influence of resource allocation differences; Assign the same initial value of the multi-domain resource allocation method in the model aggregation phase to each terminal k Step S4-2: Select the shortest path search algorithm to optimize the model splitting method selection and cooperative node selection in sub-problem 1; Specifically: Construct a weighted directed graph \(G\) for each terminal \(k\) in sub - problem 1 train =(O train , E train ); Among them, O train is a set of nodes, It is the first-layer node, representing the first layer to the v0 layer of the training model of terminal k; Is a second-layer node, representing base station n x From the v1-th layer to the v2-th layer of the trained model; is the node of the third layer, representing base station n y the v3-th to the V-th layers of the training model; e train is the edge between nodes, e train ∈E train represents the selection method of cooperative nodes, and its weight represents the transmission and computing delays; Add a pair of virtual start / end vertices s k ,e k as the source node and destination node in the graph. The weights of the edges connecting to other nodes in the graph are 0, and the optimization goal becomes searching for the shortest path from s k to e k . All the vertices on this path form the corresponding optimal model segmentation method and cooperative node selection; Even if Among them respectively represent the optimal access base station selection, cooperative base station selection, and model splitting method of terminal k; The operation process of selecting the shortest path search algorithm for sub-problem 1 is: S4-2-1: The input terminal k transmits the output data of the v-th layer of the model to each base station n x transmission delay Each base station n x Transmit the output data of the v-th layer of the model to other base stations n y Transmission delay Processing delay of the v-th layer of the terminal k computing model and each base station n x the processing delay of the v-th layer of the computing model S4-2-2: Calculate each node in the third layer Split overhead And let Indicates the third-layer relay node The splitting overhead is 0; S4-2-3: The second-layer nodes According to the starting splitting layer v1 and the base station n x It is divided into multiple subsets, and the splitting overhead is calculated within each subset first S4-2-4: Obtain the optimal segmentation cost within each subset Corresponding termination splitting layer and the corresponding third-tier base station And let Indicates the second-layer relay node The splitting overhead is 0, and it selects any relay node in the third layer Connection; S4-2-5: Calculate each node in the first layer Segmentation overhead S4-2-6: Select the termination splitting layer that minimizes the splitting overhead and obtain the corresponding second-layer base station Corresponding to the second-layer termination splitting layer and the corresponding third-level base station S4-2-7: Shortest Path of Output Terminal k Based on the optimal model segmentation method and the collaborative node selection method Optimize the multi-domain resource allocation method in the model training stage Transform sub-problem 1 into a convex optimization problem P2' as shown below: s.t. C1, C2′, C3, C5′; Among them, And The optimizations among them do not affect each other; Step S4-3: Select the shortest path search algorithm to optimize the model splitting method selection and cooperative node selection in sub-problem 2.

9. The adaptive split federated learning method for a 6G computing power network for distributed AI training services according to claim 8, characterized in that: In step S4-3, the shortest path search algorithm is selected to optimize the model splitting method selection and cooperative node selection in sub-problem 2. Specifically: Construct a weighted directed graph \(G\) for each terminal \(k\). aggr,tran =(O aggr,tran , E aggr,tran ), the node set where o k is the only first-layer node, representing terminal k; It is a second-layer node representing all terminals; is the node of the third layer, representing all base stations; The edge e between nodes aggr,tran ∈E aggr,tran represents the selection method of collaborative nodes, and its weight represents the transmission delay; Add a pair of virtual start / end vertices s k ,e k as the source node and destination node in the graph, and the weights of the edges connecting them to other nodes in the graph are 0; From s k to e k all the vertices on the shortest path form the corresponding optimal selection method of collaborative nodes, that is, let If Then Among them, respectively represent the optimal terminal selection and base station selection of terminal k; The operation process of selecting the shortest path search algorithm for sub-problem 1 is: Step S4-3-1: The input terminal k transmits the model parameters to each terminal k x transmission delay Each terminal k x transmits the model parameters to each base station n y transmission delay Step S4-3-2: Calculate each node in the third layer transmission overhead Step S4-3-3: Calculate each node in the second layer transmission overhead And let That is, the delay for the terminal to transmit model parameters to itself is 0; Step S4-3-4: The first-layer node o k Select the second-layer node that minimizes the transmission overhead and further obtain the corresponding third-level base station Step S4-3-5: Output the shortest path s of the output terminal k k ,o k , Based on the optimal collaborative node selection method Optimize the multi-domain resource allocation method in the model aggregation stage Transform sub-problem 2 into a convex optimization problem P3′ as follows: s.t. C2″, C4, C5″; Among them, And The optimizations among them do not affect each other.

Images