Segmentation federal learning method and system based on aggregation gradient broadcast

Abstract

The invention provides a segmentation federal learning method and system based on aggregation gradient broadcast, and the method comprises the steps: initializing a global model, and segmenting the global model into a server model and a client model according to the capability limitation of a local device; issuing the client model to all local equipment terminals; the multiple local devices execute forward propagation at the same time, and shredded data are obtained through calculation; transmitting the shredded data and the training data label to an edge server; performing forward propagation in parallel according to the shredded data, calculating the loss of each local equipment end in combination with a corresponding training data label, and performing back propagation to obtain a loss function and a gradient of the shredded data; aggregating the gradient of the shredded data and broadcasting to all local equipment; updating the server side model and the client side model, and aggregating the server side model; and the iterative learning loop is repeatedly executed until convergence or the maximum communication round is reached. According to the invention, the communication overhead in segmentation federal learning is saved, and the model training efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the fields of communications and federated segmentation learning technology, and specifically to a segmentation federated learning method and system based on aggregated gradient broadcasting. Background Art

[0002] With significant progress in artificial intelligence, future 6G networks are expected to disperse AI from the central cloud to edge networks, enabling the transition from "connected things" to "connected intelligence" to support edge AI vision and emerging intelligent applications such as extended reality, intelligent transportation systems, and the Internet of Things.

[0003] Distributed collaborative machine learning has become a key technology, with federated learning (FL) and split learning (SL) attracting significant attention. Federated learning enables collaborative training of complete machine learning models without sharing local client data. However, resource-constrained devices in edge networks struggle to meet the computational demands of full model training. Split learning offloads the computational complexity to the server, partitioning the complete model and deploying it separately on the client and server. However, the original split learning sequential model training method suffers from latency and can lead to catastrophic forgetting.

[0004] To overcome these issues, split federated learning (SFL) combines the advantages of FL and SL, enabling parallel model training and reducing the computational burden on clients. It holds great potential for training and inference at the edge of 6G networks. However, SFL suffers from high communication overhead caused by frequent information exchange. Despite recent attention, efforts to mitigate this overhead have been limited. Summary of the Invention

[0005] The present application provides a split federated learning method and system based on aggregated gradient broadcasting, which can solve the technical problem of frequent information exchange leading to high communication overhead in split federated learning in the prior art.

[0006] In a first aspect, the present application provides a segmentation federated learning method based on aggregated gradient broadcasting, comprising the following steps: Step S1: Initialize the global model and divide the initialized global model into a server-side model and a client-side model according to the local device capability limitation; Step S2: Send the client model to all local devices; Step S3: Based on the client model, multiple local devices simultaneously perform forward propagation to calculate the crushed data; Step S4: Transmitting the crushed data and training data labels of the local device to the edge server; Step S5: Perform forward propagation in parallel based on the received shredded data from different local devices, calculate the loss of each local device in combination with the corresponding training data labels, and perform backpropagation to obtain the gradient of the loss function and the shredded data; Step S6: Aggregate the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcast the aggregated gradient of the crushed data to all local devices; Step S7: The server and the local device update the server-side model and the client-side model according to the loss function gradient and the aggregated gradient, respectively, and aggregate the server-side model; Step S8: Repeat steps S3 to S7 until the global model converges or the maximum communication round is reached, and obtain the global model obtained by segmentation federated learning based on aggregated gradient broadcast.

[0007] Furthermore, the step S1: initializing the global model, dividing the initialized global model into a server-side model and a client-side model according to the local device capability limitation: The global model is a machine learning model used to process a specific task , the size is ,Include layer; Segmentation is performed at layer v of the global model, where ; The client model size is , the model parameters are ; The server-side model size is , the model parameters are .

[0008] Furthermore, the step S2: sending the client model to all local devices, the sending process is carried out through the network between the local device and the server; Send the client model to all local devices to obtain the initial client model .

[0009] Furthermore, in step S3: based on the client model, multiple local devices simultaneously perform forward propagation to calculate the crushed data: Small batch data sample set of local device n From its local training dataset Randomly selected from The calculation formula for crushing data is: ,in For local devices No. The crushed data obtained by forward propagation calculation after rounds of training, is the mapping function from input data to crushed data on the local device side, For the Client model parameters after round of training, is the data sample set.

[0010] Furthermore, in step S5, forward propagation is performed in parallel based on the received crushed data from different local devices, the loss of each local device is calculated in combination with the corresponding training data label, and the loss function and the gradient of the crushed data are obtained by backpropagation: The loss function of each client is ,in The local device during the tth round of training The loss function value of is the local training dataset of local device n, is the loss function, For the data sample set The samples, is the input training data, are the training data labels, For the Client model parameters after round of training, is the data sample set, For local devices In the Server-side model parameters after rounds of training; The gradient calculation formula of the loss function is ,in represents the loss function gradient of the server-side model of local device n during the t-th round of training, Represents the server-side model parameters Find the gradient, is the loss function, whose input includes the server-side model parameters , client model parameters And local data samples ; The gradient calculation formula for crushed data is , For local devices No. The gradient of the crushed data calculated by forward propagation after one round of training, For local devices No. Crushed data calculated by forward propagation after one round of training.

[0011] Furthermore, the step S6: aggregating the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcasting the aggregated gradient of the crushed data to all local devices: The calculation formula for the aggregation gradient is: ,in is the aggregate gradient of the crushed data after the tth round of training, is the weighting coefficient, , is the local training dataset, The number of local devices.

[0012] Furthermore, in step S7, the server and the local device update the server-side model and the client-side model according to the loss function gradient and the aggregation gradient, respectively, and aggregate the server-side model: For device n, the update formula of the server model and the client model in the tth communication round is: ,in is the complete model parameter of the local device n after the tth round of training, are the server-side model parameters of local device n after round t training, To share model parameters between clients after the tth round of training, is the learning rate, is the number of local epochs, represents the gradient of the loss function of the server-side model of the local device n at the i-th iteration during the t-th round of training, The tth round of training The client gradient of the iteration; The server-side model aggregation formula is , is the server-side aggregate model parameter after the tth round of training, are the server-side model parameters of the local device n after round t of training.

[0013] Furthermore, in step S8, steps S3 to S7 are repeatedly executed until the global model converges or the maximum communication round is reached, and the global model obtained by the segmentation federated learning based on the aggregated gradient broadcast is obtained: The global model obtained by segmentation federated learning based on aggregated gradient broadcast is: ,in is the server-side aggregate model parameter after the tth round of training, The client shares model parameters after the tth round of training.

[0014] In a second aspect, the present application provides a segmentation federated learning system based on aggregated gradient broadcasting, comprising: The model initialization and segmentation module is used to initialize the global model and segment the initialized global model into a server-side model and a client-side model based on the local device capability limitations. A client model delivery module, communicating with the model initialization and segmentation module, for delivering the client model to all local devices; a crushing data acquisition module, in communication with the client model delivery module, configured to simultaneously execute forward propagation on multiple local devices based on the client model to calculate crushing data; A data transmission module, in communication with the crushing data acquisition module, for transmitting the crushing data and training data labels of the local device to the edge server; a loss and gradient acquisition module, communicatively connected to the data transmission module, configured to perform forward propagation in parallel based on the received shredded data from different local devices, calculate the loss of each local device in combination with the corresponding training data labels, and perform backpropagation to obtain the loss function and the gradient of the shredded data; an aggregated gradient acquisition and broadcast module, in communication with the loss and gradient acquisition module, configured to aggregate the gradients of the crushed data to obtain aggregated gradients of the crushed data, and broadcast the aggregated gradients of the crushed data to all local devices; A segmentation model update and aggregation module, which is in communication with the aggregation gradient acquisition and broadcast module, is used to update the server-side model and the client-side model according to the loss function gradient and the aggregation gradient, respectively, and aggregate the server-side model; The global model iterative learning module is communicatively connected to the crushed data acquisition module, the data transmission module, the loss and gradient acquisition module, the aggregated gradient acquisition and broadcast module, and the segmentation model update and aggregation module, and is used to repeatedly execute steps S3 to S7 until the global model converges or reaches the maximum communication round, and obtain the global model obtained by the segmentation federated learning based on the aggregated gradient broadcast.

[0015] In a third aspect, the present application provides a computer-readable storage medium, on which a segmented federated learning program based on aggregated gradient broadcast is stored. When the segmented federated learning program based on aggregated gradient broadcast is executed by a processor, the steps of the segmented federated learning method based on aggregated gradient broadcast as described above are implemented.

[0016] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least: By splitting the global model into client and server parts, only the fragmented data generated by forward propagation needs to be transmitted instead of the complete model parameters. Combined with the gradient aggregation and broadcasting of the edge server, the amount of data transmission between the device and the server is reduced. By using edge servers to process shredded data from multiple devices in parallel and dynamically adjusting the aggregation time window to optimize synchronization efficiency, the serial bottleneck of traditional federated learning is avoided. By crushing data gradient aggregation and layered model updates, the deviation problem caused by data distribution differences between devices is alleviated. The gradient clipping technology is introduced to suppress the interference of heterogeneous gradients on the model and enhance convergence stability.

[0017] The model segmentation strategy is dynamically adjusted based on device capabilities to prevent low-performance devices from becoming training bottlenecks; and the gradient broadcast mechanism replaces traditional multi-round parameter synchronization, reducing the communication complexity when large-scale devices collaborate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a segmentation federated learning method based on aggregated gradient broadcasting provided in an embodiment of the present application; Figure 2 This is a main framework diagram of the segmentation federated learning method based on aggregated gradient broadcasting provided in an embodiment of the present application; Figure 3 A comparison chart of communication overhead results for different solutions provided in the embodiments of this application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0021] In the description of the embodiments of the present application, the words "exemplary," "for example," or "for example" are used as examples, illustrations, or explanations. Any embodiment or design described in the embodiments of the present application as "exemplary," "for example," or "for example" should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0022] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0023] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0024] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0025] First, as Figures 1-2 As shown, the embodiment of the present application provides a segmentation federated learning method based on aggregated gradient broadcasting, including the following steps: Step S1: The server initializes the global model and, based on the local device capability limitations, divides the initialized global model into a server-side model and a client-side model; Step S2: Send the client model to all local devices; Based on steps S3-S7, perform segmented federated iterative optimization learning based on aggregated gradient broadcast on the global model: Step S3: Based on the client model, multiple local devices simultaneously perform forward propagation to calculate the intermediate feature representation of the local data sample, i.e., the shredded data; Step S4: Transmitting the crushed data and training data labels of the local device to the edge server; Step S5: The server performs forward propagation in parallel based on the received shredded data from different local devices, calculates the loss of each local device in combination with the corresponding training data labels, and performs backpropagation to obtain the gradient of the loss function and the shredded data; Step S6: Aggregate the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcast the aggregated gradient of the crushed data to all local devices; Step S7: The server and the local device update the server-side model and the client-side model according to the loss function gradient and the aggregated gradient, respectively, and aggregate the server-side model; Step S8: Repeat steps S3 to S7 until the global model converges or the maximum communication rounds (or maximum training rounds) are reached, and obtain the global model obtained by segmentation federated learning based on aggregated gradient broadcast.

[0026] This application splits the global model into two parts, the client and the server, so that only the fragmented data generated by the forward propagation needs to be transmitted instead of the complete model parameters. The gradients are creatively aggregated on the server before being broadcast to the local device, thus reducing the amount of data transmitted between the device and the server. By using edge servers to process shredded data from multiple devices in parallel and dynamically adjusting the aggregation time window to optimize synchronization efficiency, the serial bottleneck of traditional federated learning is avoided. By shattering data gradient aggregation and layered model updates, the deviation problem caused by data distribution differences between devices is alleviated. The introduction of gradient clipping technology suppresses the interference of heterogeneous gradients on the model and enhances convergence stability. The model segmentation strategy is dynamically adjusted based on device capabilities to prevent low-performance devices from becoming training bottlenecks; and the gradient broadcast mechanism replaces traditional multi-round parameter synchronization, reducing the communication complexity when large-scale devices collaborate.

[0027] In one embodiment, the step S1: initializing the global model, and dividing the initialized global model into a server-side model and a client-side model according to the local device capability limitation: The global model is a machine learning model used to process a specific task , the size is ,Include layer; Segmentation is performed at layer v of the global model, where ; The client model size is , the model parameters are ; The server-side model size is , the model parameters are ; The number of local devices is N. The local device capabilities are limited to the computing power and storage capacity of the worst device. Its memory size needs to be able to support the forward propagation and reverse calculation of the client model.

[0028] In one embodiment, the step S2: sending the client model to all local devices, the sending process is performed through the network between the local device and the server; Send the client model to all local devices to get the initialized client model .

[0029] In one embodiment, in step S3, based on the client model, multiple local devices simultaneously perform forward propagation to calculate the crushed data: All local devices participating in the training each perform training on a small batch of data sample sets. The small batch data sample set of local device n From its local training dataset Randomly selected from The calculation formula for crushing data after the tth round of training is: ,in For local devices No. The crushed data obtained by forward propagation calculation after rounds of training, is the mapping function from input data to crushed data on the local device side, For local devices In the Client model parameters after round of training, is the data sample set.

[0030] In one embodiment, step S4: transmitting the shredded data and the training data labels of the local device to the edge server, and transmitting the shredded data and the corresponding training data labels through the network to calculate the loss of the client.

[0031] In one embodiment, step S5: performing forward propagation in parallel based on the received shredded data from different local devices, calculating the loss of each local device in combination with the corresponding training data label, and backpropagating to obtain the loss function and the gradient of the shredded data: Server calculation local device side The loss function of the crushed data after forward propagation on the server side is ,in The local device during the tth round of training The loss function value of is the local training dataset of local device n, is the loss function, for The samples, is the input training data, are the training data labels, For the Client model parameters after round of training, is the data sample set, For local devices In the Server-side model parameters after rounds of training; The gradient calculation formula of the loss function of the local device n is , used for updating the local device n model, Represents the server-side model parameters of local device n during round t of training The gradient, Represents the server-side model parameters Find the gradient, is the loss function, whose input includes the server-side model parameters , client model parameters , and local data samples ; The formula for calculating the crushing data gradient is: , is the gradient of the crushed data generated by the local device n after the tth round of training, is the crushed data generated by the local device n after the tth round of training.

[0032] In one embodiment, step S6: aggregating the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcasting the aggregated gradient of the crushed data to all local devices: The calculation formula for the aggregation gradient is: ,in is the aggregate gradient of the crushed data after the tth round of training, is the weighting coefficient, , is the local training dataset, The number of local devices.

[0033] In one embodiment, in step S7, the server and the local device update the server-side model and the client-side model according to the loss function gradient and the aggregation gradient, respectively, and aggregate the server-side model: Update the server-side model and client-side model of all devices through gradient descent: Each local device is based on the aggregation gradient Calculate the gradient of the local device model as , in the tth communication round, the complete model of the local device n can be updated through multiple epochs, expressed as ,in is the complete model parameter of the local device n after the tth round of training, are the server-side model parameters of local device n after round t training, To share model parameters between clients after the tth round of training, is the learning rate, is the number of local epochs, Represents the server-side model of the local device n at the i-th iteration of the t-th round of training The gradient of the loss function, The tth round of training The client gradient of the iteration; The server-side model aggregation formula is , is the server-side aggregate model parameter after the tth round of training, are the server-side model parameters of the local device n after round t training; The complete global model obtained after training in the tth communication round .

[0034] The following will introduce the communication efficiency of this method through experimental analysis.

[0035] Training Setup: Experiments were conducted in an environment with a server and N = 10 devices. The learning task was to train convolutional neural network (CNN) models for different classification tasks. To evaluate our approach, experiments were conducted on three different datasets: MNIST, FMNIST, and CIFAR-10. The model architecture consisted of a CNN with two 5x5 convolutional layers (the first with 32 channels, the second with 64 channels, each followed by 2x2 max pooling), a fully connected layer with 512 units and ReLu activation, and a final softmax output layer. The maximum CPU cycle frequency of each client was 0.1 GHz, and the total CPU cycle frequency of the server was 100 GHz. We assumed a computational workload of 5.6 MFlops per client and 86.01 MFlops per server, respectively.

[0036] Communication Network Setup: Assume that the path loss of the wireless channel between the device and the edge server is given by 128.1 + 37.6log10(d) (in dB), where d represents the distance in kilometers (km). We assume a thermal noise spectral density of -174 dBm. The maximum transmit power budgets for each client and server are 25 dBm and 33 dBm, respectively. The total bandwidth is 20 MHz.

[0037] Figure 3 (a) is a comparison chart of communication overhead results obtained from experiments on the MNIST dataset; Figure 3 (b) is a comparison chart of the communication overhead results obtained from experiments on the FMNIST dataset; Figure 3 (c) is a comparison chart of communication overhead results obtained from experiments on the CIFAR-10 dataset; Figure 3 The relationship between communication overhead and convergence performance for both IID and non-IID data partitioning settings is shown, and the proposed method (named SFL-GA) is compared with existing schemes, including traditional SFL and PSL. It is observed that the proposed method exhibits higher communication efficiency compared to traditional SFL and PSL, with significantly lower communication overhead while maintaining comparable test accuracy with the other methods. For example, on the MNIST dataset, SFL-GA achieves approximately 94% test accuracy with a communication overhead of less than 1000 Mb, while traditional SFL consumes over 4000 Mb. This demonstrates the effectiveness of the gradient aggregation scheme in SFL-GA in alleviating SFL's communication overhead.

[0038] In a second aspect, the present application provides a segmentation federated learning system based on aggregated gradient broadcasting, comprising: The model initialization and segmentation module is used to initialize the global model and segment the initialized global model into a server-side model and a client-side model based on the local device capability limitations. A client model delivery module, communicating with the model initialization and segmentation module, for delivering the client model to all local devices; a crushing data acquisition module, in communication with the client model delivery module, configured to simultaneously execute forward propagation on multiple local devices based on the client model to calculate crushing data; A data transmission module, in communication with the crushing data acquisition module, for transmitting the crushing data and training data labels of the local device to the edge server; a loss and gradient acquisition module, communicatively connected to the data transmission module, configured to perform forward propagation in parallel based on the received shredded data from different local devices, calculate the loss of each local device in combination with the corresponding training data labels, and perform backpropagation to obtain the loss function and the gradient of the shredded data; an aggregated gradient acquisition and broadcast module, in communication with the loss and gradient acquisition module, configured to aggregate the gradients of the crushed data to obtain aggregated gradients of the crushed data, and broadcast the aggregated gradients of the crushed data to all local devices; A segmentation model update and aggregation module, which is in communication with the aggregation gradient acquisition and broadcast module, is used to update the server-side model and the client-side model according to the loss function gradient and the aggregation gradient, respectively, and aggregate the server-side model; The global model iterative learning module is communicatively connected to the crushed data acquisition module, the data transmission module, the loss and gradient acquisition module, the aggregated gradient acquisition and broadcast module, and the segmentation model update and aggregation module, and is used to repeatedly execute steps S3 to S7 until the global model converges or reaches the maximum communication round, and obtain the global model obtained by the segmentation federated learning based on the aggregated gradient broadcast.

[0039] In a third aspect, the present application provides a computer-readable storage medium, characterized in that a segmented federated learning program based on aggregated gradient broadcast is stored on the computer-readable storage medium, wherein when the segmented federated learning program based on aggregated gradient broadcast is executed by a processor, the steps of the segmented federated learning method based on aggregated gradient broadcast as described above are implemented.

[0040] Among them, the functional implementation of each module in the above-mentioned segmented federated learning system based on aggregated gradient broadcast corresponds to the steps in the above-mentioned segmented federated learning method embodiment based on aggregated gradient broadcast, and their functions and implementation processes are not repeated here one by one.

[0041] In a third aspect, an embodiment of the present application provides a segmented federated learning device based on aggregated gradient broadcasting. The segmented federated learning device based on aggregated gradient broadcasting can be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0042] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces interconnect components within the aggregated gradient broadcast-based segmented federated learning device, as well as interfaces that connect the device to other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet, fiber, or ATM interfaces; user devices can include displays and keyboards.

[0043] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0044] The processor may be a general-purpose processor that can invoke a segmentation federated learning program based on aggregated gradient broadcast stored in a memory and execute the segmentation federated learning method based on aggregated gradient broadcast provided in the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the segmentation federated learning program based on aggregated gradient broadcast is invoked can be referenced to the various embodiments of the segmentation federated learning method based on aggregated gradient broadcast in the present application and will not be further described here.

[0045] In a fourth aspect, an embodiment of the present application also provides a readable storage medium.

[0046] The readable storage medium of the present application stores a segmentation federated learning program based on aggregated gradient broadcasting, wherein when the segmentation federated learning program based on aggregated gradient broadcasting is executed by a processor, the steps of the segmentation federated learning method based on aggregated gradient broadcasting as described above are implemented.

[0047] Among them, the method implemented when the segmentation federated learning program based on aggregated gradient broadcast is executed can refer to the various embodiments of the segmentation federated learning method based on aggregated gradient broadcast in this application, and will not be repeated here.

[0048] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0049] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of this application.

[0050] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A segmentation federated learning method based on aggregated gradient broadcasting, characterized in that: The following steps are involved: Step S1: Initialize the global model and divide the initialized global model into a server-side model and a client-side model according to the local device capability limitation; Step S2: Send the client model to all local devices; Step S3: Based on the client model, multiple local devices simultaneously perform forward propagation to calculate the crushed data; Step S4: Transmitting the crushed data and training data labels of the local device to the edge server; Step S5: Perform forward propagation in parallel based on the received shredded data from different local devices, calculate the loss of each local device in combination with the corresponding training data labels, and perform backpropagation to obtain the gradient of the loss function and the shredded data; Step S6: Aggregate the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcast the aggregated gradient of the crushed data to all local devices; Step S7: The server and the local device update the server-side model and the client-side model according to the loss function gradient and the aggregated gradient, respectively, and aggregate the server-side model; Step S8: Repeat steps S3 to S7 until the global model converges or the maximum communication round is reached, and obtain the global model obtained by segmentation federated learning based on aggregated gradient broadcast.

2. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1, characterized in that Step S1: Initialize the global model, and divide the initialized global model into a server-side model and a client-side model according to the local device capability limitation: The global model is a machine learning model used to process a specific task , the size is ,Include layer; Segmentation is performed at layer v of the global model, where ; The client model size is , the model parameters are ; The server-side model size is , the model parameters are .

3. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that Step S2: Sending the client model to all local devices, the sending process is carried out through the network between the local device and the server; Send the client model to all local devices to obtain the initial client model .

4. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that In step S3, based on the client model, multiple local devices simultaneously perform forward propagation to calculate the crushed data: Small batch data sample set of local device n From its local training dataset Randomly selected from The calculation formula for crushing data is: ,in For local devices No. The crushed data obtained by forward propagation calculation after rounds of training, is the mapping function from input data to crushed data on the local device side, For the Client model parameters after round of training, is the data sample set.

5. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that Step S5: Perform forward propagation in parallel based on the received crushed data from different local devices, calculate the loss of each local device in combination with the corresponding training data label, and backpropagate to obtain the loss function and the gradient of the crushed data: The loss function of each client is ,in The local device during the tth round of training The loss function value of is the local training dataset of local device n, is the loss function, For the data sample set The samples, is the input training data, are the training data labels, For the Client model parameters after round of training, is the data sample set, For local devices In the Server-side model parameters after rounds of training; The gradient calculation formula of the loss function is ,in represents the gradient of the loss function of the server-side model of the local device n during the t-th round of training, Represents the server-side model parameters Find the gradient, is the loss function, whose input includes the server-side model parameters , client model parameters And local data samples ; The gradient calculation formula for crushed data is , For local devices No. The gradient of the crushed data calculated by forward propagation after one round of training, For local devices No. Crushed data calculated by forward propagation after one round of training.

6. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that Step S6: Aggregate the gradient of the crushed data to obtain the aggregated gradient of the crushed data, and broadcast the aggregated gradient of the crushed data to all local devices: The calculation formula for the aggregation gradient is: ,in is the aggregate gradient of the crushed data after the tth round of training, is the weighting coefficient, , is the local training dataset, The number of local devices.

7. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that Step S7: The server and the local device update the server-side model and the client-side model respectively according to the loss function gradient and the aggregation gradient, and aggregate the server-side model: For device n, the update formula of the server model and the client model in the tth communication round is: ,in is the complete model parameter of the local device n after the tth round of training, are the server-side model parameters of local device n after round t training, To share model parameters between clients after the tth round of training, is the learning rate, is the number of local epochs, represents the gradient of the loss function of the server-side model of the local device n at the i-th iteration during the t-th round of training, The tth round of training The client gradient of the iteration; The server-side model aggregation formula is , is the server-side aggregate model parameter after the tth round of training, are the server-side model parameters of the local device n after round t of training.

8. The segmentation federated learning method based on aggregated gradient broadcasting according to claim 1 is characterized in that Step S8: Repeat steps S3 to S7 until the global model converges or the maximum communication round is reached, and obtain the global model obtained by the segmentation federated learning based on the aggregated gradient broadcast: The global model obtained by segmentation federated learning based on aggregated gradient broadcast is: ,in is the server-side aggregate model parameter after the tth round of training, The client shares model parameters after the tth round of training.

9. A segmentation federated learning system based on aggregated gradient broadcast, characterized in that include: The model initialization and segmentation module is used to initialize the global model and segment the initialized global model into a server-side model and a client-side model based on the local device capability limitations. A client model delivery module, communicating with the model initialization and segmentation module, for delivering the client model to all local devices; a crushing data acquisition module, in communication with the client model delivery module, configured to simultaneously execute forward propagation on multiple local devices based on the client model to calculate crushing data; A data transmission module, in communication with the crushing data acquisition module, for transmitting the crushing data and training data labels of the local device to the edge server; a loss and gradient acquisition module, communicatively connected to the data transmission module, configured to perform forward propagation in parallel based on the received shredded data from different local devices, calculate the loss of each local device in combination with the corresponding training data labels, and perform backpropagation to obtain the loss function and the gradient of the shredded data; an aggregated gradient acquisition and broadcast module, in communication with the loss and gradient acquisition module, configured to aggregate the gradients of the crushed data to obtain aggregated gradients of the crushed data, and broadcast the aggregated gradients of the crushed data to all local devices; A segmentation model update and aggregation module, which is in communication with the aggregation gradient acquisition and broadcast module, is used to update the server-side model and the client-side model according to the loss function gradient and the aggregation gradient, respectively, and aggregate the server-side model; The global model iterative learning module is communicatively connected to the crushed data acquisition module, the data transmission module, the loss and gradient acquisition module, the aggregated gradient acquisition and broadcast module, and the segmentation model update and aggregation module, and is used to repeatedly execute steps S3 to S7 until the global model converges or reaches the maximum communication round, and obtain the global model obtained by the segmentation federated learning based on the aggregated gradient broadcast.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a segmentation federated learning program based on aggregated gradient broadcasting, wherein when the segmentation federated learning program based on aggregated gradient broadcasting is executed by a processor, the steps of the segmentation federated learning method based on aggregated gradient broadcasting according to any one of claims 1 to 8 are implemented.

Images