Deep learning model collaborative training method and device based on distributed federated learning
By employing a distributed federated learning approach, cloud servers and clients collaboratively train deep learning models, solving the problems of data leakage and low training efficiency, and achieving efficient and secure model training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG ELECTRIC POWER COMM CO LTD
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-14
AI Technical Summary
Existing technologies for training deep learning models suffer from problems such as data leakage and low training efficiency.
A distributed federated learning approach is adopted, in which a deep learning model is trained collaboratively by a cloud server and a client to generate global model parameters. The client updates its local model parameters based on its own training samples and optimizes the model without sharing the original data. The cloud server performs a weighted summation of the model parameters of each client to generate global model parameters.
This approach improves the training efficiency of deep learning models without leaking data, reduces data transmission overhead, and enhances the accuracy and efficiency of model training.
Smart Images
Figure CN122390003A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, and storage medium for collaborative training of deep learning models based on distributed federated learning. Background Technology
[0002] Deep learning models are important tools in the field of artificial intelligence, used to solve complex problems such as pattern recognition, natural language processing, and computer vision.
[0003] In related technologies, there are problems such as data leakage and low training efficiency when training deep learning models. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and computer-readable storage medium for collaborative training of deep learning models based on distributed federated learning, which can improve the training efficiency of deep learning models and avoid data leakage, in order to address the above-mentioned technical problems.
[0005] In a first aspect, this application provides a method for collaborative training of deep learning models based on distributed federated learning, applied to a cloud server, wherein the cloud server communicates with at least one client, and the method includes:
[0006] Based on the local model parameters of each client and the number of training samples, global model parameters are generated and sent to each client. Each client uses the global model parameters to update the local model parameters of the deep learning model, obtaining the updated local model parameters. The local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples. The training samples include text and images.
[0007] Based on the updated local model parameters of each client, the process returns the steps of generating global model parameters based on the local model parameters of each client and the number of training samples, and sending the global model parameters to each client, until the training termination condition is met, resulting in a trained deep learning model.
[0008] In one embodiment, global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0009] The total number of training samples is obtained based on the number of training samples for each client.
[0010] Divide the number of training samples for each client by the total number of training samples to obtain the percentage of training samples for each client.
[0011] Based on the proportion of training samples in each client, the local model parameters of each client are weighted and summed to obtain the global model parameters.
[0012] In one embodiment, the cloud server is communicatively connected to at least one edge server, and each edge server is communicatively connected to at least one client. Global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0013] Receive local model parameters sent by each edge server; the local model parameters are obtained by the edge server based on the local model parameters sent by each client and the number of training samples.
[0014] Aggregate local model parameters to obtain global model parameters.
[0015] In one embodiment, sending global model parameters to each client includes:
[0016] During the training iteration of the deep learning model, the network state of each client is obtained for each training iteration;
[0017] Based on the network status of each client and the number of training samples for each client, target clients are selected from each client.
[0018] Send global model parameters to the target client.
[0019] In one embodiment, target clients are selected from each client based on their online status and the number of training samples for each client, including:
[0020] Clients whose network status is online and whose number of training samples is greater than a preset threshold are identified as target clients.
[0021] In one embodiment, before generating global model parameters based on the local model parameters of each client and the number of training samples, the method further includes:
[0022] Receive encrypted data packets sent by each client;
[0023] The encrypted data packets are parsed to obtain the local model parameters and the number of training samples for each client.
[0024] In one embodiment, the client is configured to: Before receiving encrypted data packets from each client, the client is configured to:
[0025] Obtain your own business data;
[0026] Preprocess the business data to obtain training samples; preprocessing includes, but is not limited to, data cleaning, feature engineering, and annotation and desensitization.
[0027] Secondly, this application also provides a collaborative training device for deep learning models based on distributed federated learning, applied to a cloud server, wherein the cloud server communicates with at least one client, and the device includes:
[0028] The global model parameter generation module is used to generate global model parameters based on the local model parameters of each client and the number of training samples, and then send the global model parameters to each client. Each client is used to update the local model parameters of the deep learning model based on the global model parameters to obtain the updated local model parameters. The local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples. The training samples include text and images.
[0029] The model training module is used to return the updated local model parameters of each client, generate global model parameters based on the local model parameters of each client and the number of training samples, and send the global model parameters to each client until the training termination condition is met, resulting in a trained deep learning model.
[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps of the first aspect.
[0031] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps of the first aspect.
[0032] The aforementioned method, apparatus, computer device, and computer-readable storage medium for collaborative training of deep learning models based on distributed federated learning, involves generating global model parameters based on the local model parameters of each client and the number of training samples, and then sending the global model parameters to each client. Each client updates its local model parameters based on the global model parameters, obtaining updated local model parameters. These local model parameters are obtained by training the deep learning model using its own training samples, including text and images. The process continues with the steps of generating global model parameters based on the updated local model parameters of each client and the number of training samples, and sending the global model parameters to each client, until the training termination condition is met, resulting in a trained deep learning model. As can be seen from the above, distributed federated learning enables multiple data holders (clients) to collaboratively train a deep learning model without sharing the original data, thus avoiding data leakage. Furthermore, since only local and global model parameters are transmitted between the client and the cloud server, without transmitting the original data, data transmission overhead is reduced, thereby improving the training efficiency of the deep learning model. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 This is an application environment diagram of a deep learning model collaborative training method based on distributed federated learning in one embodiment;
[0035] Figure 2 This is a flowchart illustrating a collaborative training method for deep learning models based on distributed federated learning in one embodiment.
[0036] Figure 3 This is an application environment diagram for a deep learning model collaborative training method based on distributed federated learning, as described in another embodiment.
[0037] Figure 4 This is a temporal interaction diagram of collaborative training of a deep learning model based on distributed federated learning in one embodiment;
[0038] Figure 5 This is a structural block diagram of a deep learning model collaborative training device based on distributed federated learning in one embodiment;
[0039] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0041] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0042] The deep learning model collaborative training method based on distributed federated learning provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, cloud server 102 is communicatively connected to at least one client 104. Cloud server 102 generates global model parameters based on the local model parameters of each client 104 and the number of training samples, and sends the global model parameters to each client 104. Each client 104 updates its local model parameters of the deep learning model based on the global model parameters to obtain updated local model parameters. The local model parameters are obtained by the client 104 training the deep learning model with initial model parameters based on its own training samples. The training samples include text and images. Based on the updated local model parameters of each client 104, the process of generating global model parameters based on the local model parameters of each client and the number of training samples, and sending the global model parameters to each client 104 continues until the training termination condition is met, resulting in a trained deep learning model.
[0043] In one embodiment, such as Figure 2 As shown, a method for collaborative training of deep learning models based on distributed federated learning is provided. This embodiment applies this method to... Figure 1 The cloud server 102 in the example is used for illustration. In this embodiment of the application, the method includes the following steps:
[0044] Step S210: Generate global model parameters based on the local model parameters of each client and the number of training samples, and send the global model parameters to each client; each client is used to update the local model parameters of the deep learning model based on the global model parameters to obtain the updated local model parameters; the local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples; the training samples include text and images.
[0045] In this embodiment, the cloud server uniformly distributes the initial model parameters of the deep learning model to each client. Each client initializes its own deep learning model based on the initial model parameters. Each client then trains the initialized deep learning model using training sample data to obtain the trained deep learning model. The model parameters (local model parameters) of the trained deep learning model are then sent to the cloud server.
[0046] The cloud server aggregates the local model parameters sent by each client to obtain the global model parameters. Specifically, the cloud server performs a weighted sum of the local model parameters sent by each client based on the number of training samples sent by each client to obtain the global model parameters.
[0047] The cloud server distributes global model parameters of the deep learning model to each client. Each client optimizes the global model parameters based on the training sample data to obtain updated local model parameters. The updated local model parameters are then sent back to the cloud server.
[0048] Step S220: Based on the updated local model parameters of each client, return the steps of generating global model parameters based on the local model parameters of each client and the number of training samples, and send the global model parameters to each client until the training termination condition is met, and obtain the trained deep learning model.
[0049] The training termination conditions include either the convergence of the loss function value of the deep learning model or the completion of a preset number of training iterations.
[0050] In this embodiment, the cloud server aggregates the updated local model parameters sent by each client to obtain the updated global model parameters. Specifically, the cloud server performs a weighted sum of the updated local model parameters sent by each client based on the number of training samples sent by each client to obtain the updated global model parameters.
[0051] The cloud server distributes updated global model parameters to each client. Each client optimizes the updated global model parameters based on the training sample data to obtain updated local model parameters. The updated local model parameters are then sent to the cloud server. This process is repeated until the training termination condition is met, resulting in a trained deep learning model.
[0052] The aforementioned collaborative training method for deep learning models based on distributed federated learning generates global model parameters based on the local model parameters of each client and the number of training samples, and sends these global model parameters to each client. Each client then updates its local model parameters based on the global model parameters, obtaining updated local model parameters. These local model parameters are obtained by training the deep learning model using its own training samples, including text and images. The process continues with the generation of global model parameters based on the updated local model parameters of each client, and the sending of these global model parameters to each client. This process continues until the training termination condition is met, resulting in a trained deep learning model. As can be seen from the above, distributed federated learning enables multiple data holders (clients) to collaboratively train a deep learning model without sharing the original data, thus avoiding data leakage. Furthermore, since only local and global model parameters are transmitted between the client and the cloud server, without transmitting the original data, data transmission overhead is reduced, thereby improving the training efficiency of the deep learning model.
[0053] In one embodiment, before generating global model parameters based on the local model parameters of each client and the number of training samples, the method further includes:
[0054] Step S211: Receive encrypted data packets sent by each client.
[0055] In this embodiment, each client packages, compresses, and encrypts its local model parameters and the number of training samples to obtain an encrypted data packet. The encrypted data packet is then sent to the cloud server.
[0056] Step S212: Parse the encrypted data packet to obtain the local model parameters and the number of training samples for each client.
[0057] In this embodiment of the application, the cloud server decompresses and decrypts the encrypted data packets sent by each client to obtain the local model parameters and the number of training samples for each client.
[0058] In the embodiments of this application, each client can ensure the secure transmission of local model parameters and training sample quantity by packaging, compressing, and encrypting them.
[0059] In one embodiment, the client is configured to: before receiving encrypted data packets from each client.
[0060] Step S2111: Obtain your own business data.
[0061] In this embodiment, the client's business data is determined based on the client's business type. For example, if the client's business type is insurance business, then the business data refers to policy information, claims records, etc.
[0062] Step S2112: Preprocess the business data to obtain training samples; preprocessing includes, but is not limited to, data cleaning, feature engineering, and annotation and desensitization.
[0063] Data cleaning includes missing value handling, outlier detection, and duplicate value handling, which are used to eliminate noise, correct errors, handle missing values, and ensure data quality.
[0064] Feature engineering includes numerical feature processing, categorical feature processing, and text feature processing, which are used to extract meaningful features from the raw data and improve the model's expressive power.
[0065] The annotation and desensitization process includes data anonymization, differential privacy, and annotation replacement, which are used to prevent the leakage of sensitive information.
[0066] In this embodiment of the application, each client preprocesses the collected business data to obtain training samples.
[0067] In one embodiment, global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0068] Step S310: Obtain the total number of training samples based on the number of training samples for each client;
[0069] Step S320: Divide the number of training samples for each client by the total number of training samples to obtain the proportion of training samples for each client.
[0070] Step S330: Based on the proportion of training samples in each client, the local model parameters of each client are weighted and summed to obtain the global model parameters.
[0071] In this embodiment, the proportion of training samples for each client is multiplied by the local model parameters to obtain the product result for each client. The product results for each client are then summed to obtain the global model parameters.
[0072] The embodiments of this application determine the contribution ratio of each client to the global model parameters based on the number of training samples of each client, thereby improving the accuracy of the global model parameters.
[0073] In one embodiment, such as Figure 3 As shown, cloud server 102 is communicatively connected to at least one edge server 106, and each edge server 106 is communicatively connected to at least one client 104. Global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0074] Step S410: Receive local model parameters sent by each edge server; the local model parameters are obtained by the edge server based on the local model parameters sent by each client and the number of training samples.
[0075] Each edge server communicates with a certain number of clients, and the cloud server communicates with each edge server.
[0076] In the embodiments of this application, such as Figure 4 As shown, the cloud server sends the initial model parameters to each client via the edge server. Each client initializes its own deep learning model based on the initial model parameters. Each client then optimizes the initial model parameters based on the training sample data to obtain local model parameters. The local model parameters and the number of training samples are then sent to the corresponding edge server.
[0077] The edge server obtains the total number of training samples based on the number of training samples from each client, divides the number of training samples from each client by the total number of training samples to obtain the proportion of training samples from each client, and then performs a weighted summation of the local model parameters of each client based on the proportion of training samples from each client to obtain the local model parameters.
[0078] Each edge server sends local model parameters to the cloud server.
[0079] Step S420: Aggregate the local model parameters to obtain the global model parameters.
[0080] In this embodiment of the application, the local model parameters sent by each edge server are averaged to obtain the global model parameters.
[0081] In one embodiment, sending global model parameters to each client includes:
[0082] Step S510: During the training iteration of the deep learning model, obtain the network state of each client for each training iteration.
[0083] The network status includes online status and offline status.
[0084] In this embodiment of the application, the cloud server obtains the network status of each client in real time after each update of the global model parameters.
[0085] Step S520: Select target clients from each client based on the network status of each client and the number of training samples for each client;
[0086] Step S530: Send the global model parameters to the target client.
[0087] In this embodiment of the application, the cloud server selects a portion of target clients from all clients based on the network status of each client and the number of training samples of each client. The model parameters of the deep learning model are updated through the interaction between the target clients and the cloud server.
[0088] In one embodiment, target clients are selected from each client based on their online status and the number of training samples for each client, including:
[0089] Step S522: The client whose network status is online and whose number of training samples is greater than a preset threshold is identified as the target client.
[0090] The preset quantity threshold can be set according to actual needs.
[0091] In this embodiment of the application, the cloud server identifies clients with a large number of training samples and online status as target clients. By training the model parameters through the target clients, the convergence speed of the model parameters can be improved, thereby increasing the training efficiency of the deep learning model.
[0092] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0093] Based on the same inventive concept, this application also provides a device for collaborative training of deep learning models based on distributed federated learning, which implements the aforementioned method for collaborative training of deep learning models based on distributed federated learning. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for collaborative training of deep learning models based on distributed federated learning provided below can be found in the limitations of the method for collaborative training of deep learning models based on distributed federated learning described above, and will not be repeated here.
[0094] In one exemplary embodiment, please refer to Figure 5 This paper provides a collaborative training device for deep learning models based on distributed federated learning, applied to a cloud server. The cloud server communicates with at least one client, including:
[0095] The global model parameter generation module 510 is used to generate global model parameters based on the local model parameters of each client and the number of training samples, and send the global model parameters to each client. Each client is used to update the local model parameters of the deep learning model based on the global model parameters to obtain the updated local model parameters. The local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples. The training samples include text and images.
[0096] The model training module 520 is used to return the steps of generating global model parameters based on the updated local model parameters of each client and the number of training samples, and sending the global model parameters to each client, until the training termination condition is met, and a trained deep learning model is obtained.
[0097] In one embodiment, global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0098] The total number of training samples is obtained based on the number of training samples for each client.
[0099] Divide the number of training samples for each client by the total number of training samples to obtain the percentage of training samples for each client.
[0100] Based on the proportion of training samples in each client, the local model parameters of each client are weighted and summed to obtain the global model parameters.
[0101] In one embodiment, the cloud server is communicatively connected to at least one edge server, and each edge server is communicatively connected to at least one client. Global model parameters are generated based on the local model parameters of each client and the number of training samples, including:
[0102] Receive local model parameters sent by each edge server; the local model parameters are obtained by the edge server based on the local model parameters sent by each client and the number of training samples.
[0103] Aggregate local model parameters to obtain global model parameters.
[0104] In one embodiment, sending global model parameters to each client includes:
[0105] During the training iteration of the deep learning model, the network state of each client is obtained for each training iteration;
[0106] Based on the network status of each client and the number of training samples for each client, target clients are selected from each client.
[0107] Send global model parameters to the target client.
[0108] In one embodiment, target clients are selected from each client based on their online status and the number of training samples for each client, including:
[0109] Clients whose network status is online and whose number of training samples is greater than a preset threshold are identified as target clients.
[0110] In one embodiment, before generating global model parameters based on the local model parameters of each client and the number of training samples, the method further includes:
[0111] Receive encrypted data packets sent by each client;
[0112] The encrypted data packets are parsed to obtain the local model parameters and the number of training samples for each client.
[0113] In one embodiment, the client is configured to: Before receiving encrypted data packets from each client, the client is configured to:
[0114] Obtain your own business data;
[0115] Preprocess the business data to obtain training samples; preprocessing includes, but is not limited to, data cleaning, feature engineering, and annotation and desensitization.
[0116] The modules in the aforementioned deep learning model collaborative training device based on distributed federated learning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0117] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a collaborative training method for deep learning models based on distributed federated learning. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0118] Those skilled in the art will understand that Figure 6 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the aforementioned method for collaborative training of deep learning models based on distributed federated learning. The steps of this method for collaborative training of deep learning models based on distributed federated learning can be steps from one of the aforementioned embodiments of the method for collaborative training of deep learning models based on distributed federated learning.
[0119] In one embodiment, a computer-readable storage medium is provided, storing a computer program that, when executed by a processor, causes the processor to perform the steps of the aforementioned method for collaborative training of deep learning models based on distributed federated learning. The steps of this method for collaborative training of deep learning models based on distributed federated learning can be steps from one of the methods described in the various embodiments above.
[0120] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the steps of the aforementioned method for collaborative training of deep learning models based on distributed federated learning. The steps of this method for collaborative training of deep learning models based on distributed federated learning can be steps from one of the methods described in the various embodiments above.
[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0124] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for collaborative training of deep learning models based on distributed federated learning, characterized in that, Applied to a cloud server, wherein the cloud server communicates with at least one client, the method includes: Based on the local model parameters and the number of training samples of each client, global model parameters are generated and sent to each client. Each client is used to update the local model parameters of the deep learning model according to the global model parameters to obtain the updated local model parameters. The local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples. The training samples include text and images. Based on the updated local model parameters of each client, the process of generating global model parameters based on the local model parameters of each client and the number of training samples, and sending the global model parameters to each client, continues until the training termination condition is met, resulting in a trained deep learning model.
2. The method according to claim 1, characterized in that, The step of generating global model parameters based on the local model parameters of each client and the number of training samples includes: The total number of training samples is obtained based on the number of training samples for each client. Divide the number of training samples for each client by the total number of training samples to obtain the percentage of training samples for each client. The global model parameters are obtained by weighted summing of the local model parameters of each client based on the proportion of training samples of each client.
3. The method according to claim 1, characterized in that, The cloud server is communicatively connected to at least one edge server, and each edge server is communicatively connected to at least one client. The generation of global model parameters based on the local model parameters of each client and the number of training samples includes: The edge server receives local model parameters sent by each of the aforementioned edge servers; the local model parameters are obtained by the edge server based on the local model parameters sent by each of the aforementioned clients and the number of training samples. The local model parameters are aggregated to obtain the global model parameters.
4. The method according to claim 1, characterized in that, Sending the global model parameters to each of the clients includes: During the training iteration of the deep learning model, the network state of each client is obtained for each training iteration. Based on the network status of each client and the number of training samples for each client, target clients are selected from each client. The global model parameters are sent to the target client.
5. The method according to claim 4, characterized in that, The step of selecting target clients from each client based on their online status and the number of training samples for each client includes: Clients whose network status is online and whose number of training samples is greater than a preset threshold are identified as the target clients.
6. The method according to claim 1, characterized in that, Before generating global model parameters based on the local model parameters of each client and the number of training samples, the method further includes: Receive encrypted data packets sent by each of the aforementioned clients; The encrypted data packets are parsed to obtain the local model parameters and the number of training samples for each client.
7. The method according to claim 6, characterized in that, Before receiving encrypted data packets sent by each of the clients, the client is configured to: Obtain your own business data; The business data is preprocessed to obtain training samples; the preprocessing includes, but is not limited to, data cleaning, feature engineering, and annotation and desensitization.
8. A collaborative training device for deep learning models based on distributed federated learning, characterized in that, Applied to a cloud server, the cloud server communicating with at least one client, the device includes: A global model parameter generation module is used to generate global model parameters based on the local model parameters of each client and the number of training samples, and send the global model parameters to each client; each client is used to update the local model parameters of the deep learning model based on the global model parameters to obtain the updated local model parameters; the local model parameters are obtained by the client training the initial model parameters of the deep learning model based on its own training samples; the training samples include text and images; The model training module is used to return the steps of generating global model parameters based on the updated local model parameters of each client and the number of training samples, and sending the global model parameters to each client, until the training termination condition is met, and a trained deep learning model is obtained.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.