Asynchronous federated learning training method and device, computer equipment and storage medium
By correcting the gradient update mechanism using sample weights and parameter weights in asynchronous joint learning, and combining it with adaptive threshold gradient compression, the problem of user data privacy leakage is solved, and the computational efficiency and model accuracy of edge devices are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 新奥新智科技有限公司
- Filing Date
- 2021-07-01
- Publication Date
- 2026-05-29
AI Technical Summary
Existing federated learning suffers from user data privacy issues, especially when edge devices have limited data and train independently, making them prone to getting stuck in local optima and allowing gradient data to leak user information.
By calculating sample weights and parameter weights, correcting gradients, updating global parameters, and adaptively compressing gradients with a threshold, asynchronous joint learning is achieved, ensuring data privacy and security.
It improves edge computing efficiency, prevents user data privacy leaks, and enhances the security and accuracy of model training.
Smart Images
Figure CN115564055B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to an asynchronous joint learning training method, apparatus, computer device, and storage medium. Background Technology
[0002] Traditional federated learning can protect user data privacy during joint model training. The central node and local participants continuously update the model, uploading and distributing model parameters to complete joint training. Horizontal federated training can combine more data to improve model accuracy. Federation learning performs joint modeling without disclosing or aggregating the original data from all parties, maintaining the decentralization and distribution of training data.
[0003] Because joint learning mechanisms do not require data aggregation before model training, data can be computed at the edge before uploading the corresponding parameters. However, the data from a single edge device is limited, and the model for each participant may get stuck in a local optimum during individual training. Furthermore, the gradient data learned by nodes in joint learning can indirectly reflect training sample information, thereby potentially leaking user data privacy. Summary of the Invention
[0004] In view of this, embodiments of the present invention provide an asynchronous joint learning training method, apparatus, computer device, and storage medium to solve the problem of user data privacy leakage in the prior art.
[0005] A first aspect of this invention provides an asynchronous joint learning training method, comprising: performing a first calculation step, calculating corresponding sample weights and parameter weights based on node learning information; performing a correction step, correcting the gradient uploaded by the node based on the sample weights and parameter weights to obtain a corrected gradient; performing an update step, updating global parameters based on the corrected gradient to obtain updated parameters; performing a comparison step, comparing the updated parameters with historical parameters to calculate a target threshold; and performing a compression step, compressing the gradient uploaded by the node based on the target threshold to obtain a compressed gradient.
[0006] A second aspect of this invention provides an asynchronous joint learning training apparatus, comprising: a calculation module configured as an initialization module configured to execute a first calculation step, calculating corresponding sample weights and parameter weights based on node learning information; a correction module configured to execute a correction step, correcting the gradient uploaded by the node based on the sample weights and parameter weights to obtain a corrected gradient; an update module configured to execute an update step, updating global parameters based on the corrected gradient to obtain updated parameters; a comparison module configured to execute a comparison step, comparing the updated parameters with historical parameters to calculate a target threshold; and a compression module configured to execute a compression step, compressing the gradient uploaded by the node based on the target threshold to obtain a compressed gradient.
[0007] A third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.
[0008] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0009] The beneficial effects of this invention compared to existing technologies are as follows: By executing a first calculation step, corresponding sample weights and parameter weights are calculated based on node learning information; a correction step is executed, based on the sample weights and parameter weights, the gradient uploaded by the node is corrected to obtain the corrected gradient; an update step is executed, based on the corrected gradient, the global parameters are updated to obtain the updated parameters; a comparison step is executed, the updated parameters are compared with historical parameters to calculate the target threshold; and a compression step is executed, based on the target threshold, the gradient uploaded by the node is compressed to obtain the compressed gradient. This invention improves edge-side computational efficiency without compromising user data privacy. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a block diagram of the asynchronous joint learning framework according to an embodiment of the present invention;
[0012] Figure 2 This is a flowchart of an asynchronous joint learning training method provided in an embodiment of the present invention;
[0013] Figure 3 This is a flowchart of another asynchronous joint learning training method provided in an embodiment of the present invention;
[0014] Figure 4 This is a block diagram of an asynchronous joint learning training device provided in an embodiment of the present invention;
[0015] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0016] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.
[0017] The joint learning referred to in this invention can be used to support multi-user, multi-party collaboration, and to mine data value through AI technology to establish intelligent joint modeling. Intelligent joint modeling includes:
[0018] 1) Participating nodes control their own data in a weakly centralized joint training mode to ensure data privacy and security during the co-creation of intelligence;
[0019] 2) In different application scenarios, utilize the selection and / or combination of AI algorithms and privacy-preserving computation to establish multiple model aggregation and optimization strategies to obtain high-level, high-quality models;
[0020] 3) Under the premise of ensuring data security and user privacy, based on multiple model aggregation optimization strategies, obtain methods to improve the performance of the federated learning engine. The performance improvement methods can be achieved by solving problems such as parallel computing architecture, information interaction under large-scale cross-domain networks, intelligent perception, and anomaly handling mechanisms to improve the overall performance of the federated learning engine.
[0021] 4) Obtain the needs of multiple users in various scenarios, and through a mutual trust mechanism, determine and reasonably assess the true contribution of each participating party and allocate incentives accordingly;
[0022] Based on the above approach, an AI technology ecosystem based on collaborative learning can be established, fully leveraging the value of industry data and promoting the implementation of scenarios in vertical fields.
[0023] Figure 1This is a block diagram of the asynchronous joint learning framework according to an embodiment of the present invention. The asynchronous joint learning framework may include a central node layer (also called a parameter server layer) and an edge node layer.
[0024] The threshold adaptation module resides in the edge node layer, while the asynchronous joint learning module spans both the central node layer (also known as the parameter server layer) and the edge node layer. The central node is responsible for integrating the parameters from the edge nodes, updating them, and then distributing the updated parameters back to the edge nodes. The edge nodes only communicate with the central node.
[0025] The parameter server layer includes an asynchronous joint learning module, which comprises a parameter update unit, a gradient correction unit, and a dual weight calculation unit. The dual weight calculation unit calculates the corresponding sample weights and parameter weights based on the node's learning information. The gradient correction unit corrects the gradients uploaded by the nodes based on the dual weights. The parameter update unit uses the corrected gradients to update the global parameters.
[0026] The edge node layer includes an asynchronous joint learning module and a threshold adaptation module. The asynchronous joint learning module includes a node learning state monitoring unit, and the threshold adaptation module includes an adaptive threshold calculation unit and a gradient compression unit. The adaptive threshold calculation unit calculates the threshold based on the latest parameter changes. The gradient compression unit uses the threshold to compress communication that does not conform to the gradient. The node learning state monitoring unit is responsible for monitoring the node learning state, such as the number of learning rounds and the number of samples.
[0027] An asynchronous joint learning training method and apparatus according to an embodiment of the present invention will now be described in detail with reference to the accompanying drawings.
[0028] Figure 2 This is a flowchart of an asynchronous joint learning training method provided in an embodiment of the present invention. Figure 2 The asynchronous joint learning training method can be executed by the server. For example... Figure 2 As shown, the asynchronous joint learning training method includes:
[0029] S201, Execute the first calculation step, calculate the corresponding sample weights and parameter weights based on the node learning information;
[0030] S202, Perform the correction step, based on the sample weights and parameter weights, to correct the gradient uploaded by the node and obtain the corrected gradient;
[0031] S203, Perform the update step, update the global parameters based on the corrected gradient, and obtain the updated parameters;
[0032] S204, Perform the comparison step, compare the updated parameters with the historical parameters, and calculate the target threshold;
[0033] S205, Perform the compression step, compress the gradient uploaded by the node based on the target threshold, and obtain the compressed gradient.
[0034] Specifically, the process involves the following steps: First, a calculation step is performed to calculate the corresponding sample weights and parameter weights based on the node's learning information; Second, a correction step is performed to correct the gradient uploaded by the node based on the sample weights and parameter weights, resulting in a corrected gradient; Third, an update step is performed to update the global parameters based on the corrected gradient, resulting in updated parameters; Fourth, a comparison step is performed to compare the updated parameters with historical parameters and calculate the target threshold; Fifth, a compression step is performed to compress the gradient uploaded by the node based on the target threshold, resulting in a compressed gradient.
[0035] According to the technical solution provided in the embodiments of the present invention, the following steps are performed: First, a calculation step is performed to calculate the corresponding sample weights and parameter weights based on the node learning information; a correction step is performed to correct the gradient uploaded by the node based on the sample weights and parameter weights, resulting in a corrected gradient; an update step is performed to update the global parameters based on the corrected gradient, resulting in updated parameters; a comparison step is performed to compare the updated parameters with historical parameters and calculate a target threshold; finally, a compression step is performed to compress the gradient uploaded by the node based on the target threshold, resulting in a compressed gradient. This invention improves edge-side computational efficiency without compromising user data privacy.
[0036] Sample weights are determined by the proportion of a node's sample size to the total sample size. Parameter weights are influenced by the temporal similarity between the target parameters of the gradient and the current global parameters.
[0037] The gradients uploaded by the nodes are corrected using dual weights (sample weights and parameter weights). These gradients can be generated through gradient communication between the node and the server, thus reducing the number of communications between a single node and the parameter server. Larger dual weight values result in more gradient communications.
[0038] The corrected gradient replaces the corresponding parameter in the global parameters.
[0039] Adaptive threshold calculation requires obtaining the updated parameters and comparing them with historical parameters. The adaptive threshold calculation function calculates the threshold based on the latest parameter changes; historical parameters can be obtained by the server via wired or wireless means.
[0040] Gradient compression can compress communication that does not conform to the gradient by using a threshold. The threshold can be used as a criterion for gradient checking. Generally, a fixed threshold is used for criterion. This invention uses an adaptive threshold as the criterion.
[0041] In some embodiments, a second calculation step is performed to calculate the corresponding sample weights and parameter weights based on the compressed gradient; the correction step, update step, comparison step, compression step, and second calculation step are then performed until a preset condition is met and execution stops.
[0042] In some embodiments, the sample weights are calculated by the following steps: determining the sample weights based on the ratio of the number of node samples in the node learning information to the total number of samples.
[0043] In some embodiments, a discrimination condition is generated based on a target threshold; the gradient uploaded by the node is compressed based on the discrimination condition to obtain a compressed gradient.
[0044] A threshold can be used as the discrimination condition for gradient checking, and generally a fixed threshold is used. This invention uses an adaptive threshold as the discrimination condition. In some embodiments, it is determined whether the target node meets the target condition; in response to determining that the target node does not meet the target condition, the target node accumulates gradient information locally; in response to the amount of gradient information of the target node meeting the target value, the gradient information of the target node is uploaded to the central node.
[0045] Specifically, the target node can be any node, and the target conditions can be pre-defined, such as the gradient of the target node being within a threshold range.
[0046] As an example, nodes automatically adapt to changes in gradients during each round of model training and can calculate appropriate thresholds to compress gradients. Nodes that meet the criteria are granted the right to communicate with the parameter server for the corresponding round; otherwise, they accumulate gradient information locally for the next learning iteration. Eventually, sufficient gradient information is accumulated and uploaded to the central node. Regardless of whether a node is granted communication privileges for that round, a gradient check must be performed after the next learning round; that is, gradient self-checking is a continuous process throughout the node's learning journey.
[0047] In some embodiments, the target threshold is obtained by the following steps: inputting node learning information into the node auto-adaptation model to obtain the target threshold.
[0048] The node auto-adaptation model can be pre-trained. It can be based on neural network training.
[0049] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0050] Figure 3 This is a flowchart of another asynchronous joint learning training method provided in an embodiment of the present invention. Figure 3 The asynchronous joint learning training method can be executed by the server. For example... Figure 3As shown, the asynchronous joint learning training method includes:
[0051] S301, Execute the first calculation step, calculate the corresponding sample weights and parameter weights based on the node learning information;
[0052] S302, Perform the correction step, based on the sample weights and parameter weights, to correct the gradient uploaded by the node and obtain the corrected gradient;
[0053] S303, execute the update step, update the global parameters based on the corrected gradient, and obtain the updated parameters;
[0054] S304, Perform the comparison step to compare the updated parameters with the historical parameters and calculate the target threshold;
[0055] S305, Perform the compression step, compress the gradient uploaded by the node based on the target threshold, and obtain the compressed gradient;
[0056] S306, Perform the second calculation step: Calculate the corresponding sample weights and parameter weights based on the compressed gradient;
[0057] S307, continue executing the correction step, update step, comparison step, compression step and second calculation step until the preset conditions are met and execution stops.
[0058] Specifically, the process involves the following steps: First, a calculation step is performed to calculate the corresponding sample weights and parameter weights based on the node's learning information; Second, a correction step is performed to correct the gradient uploaded by the node based on the sample weights and parameter weights, resulting in a corrected gradient; Third, an update step is performed to update the global parameters based on the corrected gradient, resulting in updated parameters; Fourth, a comparison step is performed to compare the updated parameters with historical parameters and calculate the target threshold; Fifth, a compression step is performed to compress the gradient uploaded by the node based on the target threshold, resulting in a compressed gradient; Sixth, a second calculation step is performed to calculate the corresponding sample weights and parameter weights based on the compressed gradient; Seventh, the correction step, update step, comparison step, compression step, and second calculation step are continued until a preset condition is met and execution stops.
[0059] According to the technical solution provided in the embodiments of the present invention, the following steps are performed: a first calculation step is executed to calculate the corresponding sample weights and parameter weights based on the node learning information; a correction step is executed to correct the gradient uploaded by the node based on the sample weights and parameter weights to obtain the corrected gradient; an update step is executed to update the global parameters based on the corrected gradient to obtain the updated parameters; a comparison step is executed to compare the updated parameters with historical parameters to calculate the target threshold; a compression step is executed to compress the gradient uploaded by the node based on the target threshold to obtain the compressed gradient; a second calculation step is executed to calculate the corresponding sample weights and parameter weights based on the compressed gradient; the correction step, update step, comparison step, compression step, and second calculation step are continued until a preset condition is met and execution stops. This invention improves edge computing efficiency without compromising user data privacy.
[0060] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.
[0061] Figure 4 This is a schematic diagram of an asynchronous joint learning training device provided in an embodiment of the present invention. Figure 4 As shown, the asynchronous joint learning training device includes:
[0062] The calculation module 401 is configured as an initialization module and is configured to execute the first calculation step, which calculates the corresponding sample weights and parameter weights based on the node learning information.
[0063] The correction module 402 is configured to perform a correction step, which corrects the gradient uploaded by the node based on the sample weights and parameter weights, and obtains the corrected gradient.
[0064] The update module 403 is configured to perform an update step, which updates the global parameters based on the corrected gradient to obtain the updated parameters;
[0065] Comparison module 404 is configured to perform a comparison step, comparing the updated parameters with historical parameters to calculate the target threshold.
[0066] Compression module 405 is configured to perform a compression step, compressing the gradient uploaded by the node based on a target threshold to obtain a compressed gradient.
[0067] According to the technical solution provided in the embodiments of the present invention, the following steps are performed: a first calculation step is executed to calculate the corresponding sample weights and parameter weights based on the node learning information; a correction step is executed to correct the gradient uploaded by the node based on the sample weights and parameter weights to obtain the corrected gradient; an update step is executed to update the global parameters based on the corrected gradient to obtain the updated parameters; a comparison step is executed to compare the updated parameters with historical parameters to calculate the target threshold; and a compression step is executed to compress the gradient uploaded by the node based on the target threshold to obtain the compressed gradient. This invention improves the computational efficiency at the edge without compromising user data privacy.
[0068] In some embodiments, the asynchronous joint learning training device further includes: a recalculation module 406, configured to perform a second calculation step, calculating the corresponding sample weights and parameter weights based on the compressed gradient; and a stop module 407, configured to continue executing the correction step, update step, comparison step, compression step, and second calculation step until a preset condition is met and execution stops.
[0069] In some embodiments, the sample weights are calculated by the following steps: determining the sample weights based on the ratio of the number of node samples in the node learning information to the total number of samples.
[0070] In some embodiments, the compression module 405 is further configured to: generate discrimination conditions based on a target threshold; and compress the gradient uploaded by the node based on the discrimination conditions to obtain the compressed gradient.
[0071] In some embodiments, the asynchronous joint learning training apparatus is further configured to: determine whether a target node meets the target conditions; in response to determining that the target node does not meet the target conditions, the target node accumulates gradient information locally; and in response that the amount of gradient information of the target node meets the target value, upload the gradient information of the target node to the central node.
[0072] In some embodiments, the node auto-adaptation model is obtained through the following steps: inputting node learning information into the node auto-adaptation model to obtain the target threshold.
[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0074] Figure 5 This is a schematic diagram of the computer device 5 provided in an embodiment of the present invention. Figure 5As shown, the computer device 5 in this embodiment includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the various method embodiments described above. Alternatively, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in the various device embodiments described above.
[0075] Exemplarily, computer program 503 may be divided into one or more modules / units, which are stored in memory 502 and executed by processor 501 to perform the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 503 in computer device 5.
[0076] Computer device 5 may be a desktop computer, laptop, handheld computer, cloud server, or other similar computer device. Computer device 5 may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of computer device 5 and does not constitute a limitation on computer device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.
[0077] Processor 501 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0078] The memory 502 can be an internal storage unit of the computer device 5, such as a hard disk or RAM of the computer device 5. The memory 502 can also be an external storage device of the computer device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 5. Furthermore, the memory 502 can include both internal and external storage units of the computer device 5. The memory 502 is used to store computer programs and other programs and data required by the computer device. The memory 502 can also be used to temporarily store data that has been output or will be output.
[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0080] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0081] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments of the invention herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.
[0082] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0084] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0085] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. An asynchronous joint learning training method, applied to an asynchronous joint learning framework including a parameter server layer and an edge node layer; The parameter server layer includes: An asynchronous joint learning module, comprising: a parameter update unit, a gradient correction unit, and a dual weight calculation unit; the edge node layer comprising: an asynchronous joint learning module and a threshold adaptation module, wherein the asynchronous joint learning module comprises: a node learning state monitoring unit, and the threshold adaptation module comprises: an adaptive threshold calculation unit and a gradient compression unit; characterized in that the method comprises: The first calculation step is performed by the dual weight calculation unit to calculate the corresponding sample weight and parameter weight based on the node learning information. The gradient correction unit performs a correction step to correct the gradient uploaded by the node based on the sample weights and the parameter weights, thereby obtaining the corrected gradient. The update step is performed by the parameter update unit to update the global parameters based on the corrected gradient, and the updated parameters are obtained. The adaptive threshold calculation unit performs a comparison step to compare the updated parameters with the historical parameters and calculate the target threshold. The gradient compression unit performs a compression step to compress the gradient uploaded by the node based on the target threshold, thereby obtaining a compressed gradient; wherein gradient compression refers to using a threshold to compress communications that do not conform to the gradient. The node learning status monitoring unit is responsible for monitoring the node learning status, including the number of learning rounds and the number of samples.
2. The method according to claim 1, characterized in that, The method further includes: The second calculation step is performed to calculate the corresponding sample weights and parameter weights based on the compressed gradient; Continue executing the correction step, the update step, the comparison step, the compression step, and the second calculation step until the preset conditions are met and execution stops.
3. The method according to claim 1, characterized in that, The sample weights are calculated through the following steps: The sample weight is determined based on the ratio of the number of node samples to the total number of samples in the node learning information.
4. The method according to claim 1, characterized in that, The compression step, which compresses the gradient uploaded by the node based on the target threshold to obtain the compressed gradient, includes: Based on the target threshold, discrimination conditions are generated; The gradient uploaded by the node is compressed based on the discrimination condition to obtain the compressed gradient.
5. The method according to claim 1, characterized in that, The method further includes: Determine whether the target node meets the target conditions; In response to the determination that a target node does not meet the target conditions, the target node accumulates gradient information locally; In response to the target node's gradient information content meeting the target value, the target node's gradient information is uploaded to the parameter server.
6. The method according to claim 1, characterized in that, The target threshold is obtained through the following steps: The node learning information is input into the node auto-adaptation model to obtain the target threshold. The node auto-adaptation model is pre-trained.
7. An asynchronous joint learning training device, applied to an asynchronous joint learning framework including a parameter server layer and an edge node layer; The parameter server layer includes: An asynchronous joint learning module, comprising: a parameter update unit, a gradient correction unit, and a dual weight calculation unit; the edge node layer comprising: an asynchronous joint learning module and a threshold adaptation module, wherein the asynchronous joint learning module comprises: a node learning state monitoring unit, and the threshold adaptation module comprises: an adaptive threshold calculation unit and a gradient compression unit; characterized in that the device comprises: The calculation module is configured as an initialization module, and is configured to perform a first calculation step through the dual weight calculation unit to calculate the corresponding sample weights and parameter weights based on the node learning information. The correction module is configured to perform a correction step through the gradient correction unit, and correct the gradient uploaded by the node based on the sample weights and the parameter weights to obtain the corrected gradient. The update module is configured to perform an update step through the parameter update unit, update the global parameters based on the corrected gradient, and obtain the updated parameters. The comparison module is configured to perform a comparison step through the adaptive threshold calculation unit, comparing the updated parameters and historical parameters to calculate the target threshold; The compression module is configured to perform a compression step through the gradient compression unit, compressing the gradient uploaded by the node based on the target threshold to obtain a compressed gradient; wherein gradient compression refers to using a threshold to compress communication that does not conform to the gradient. The device is also used to monitor the learning status of the node through the node learning status monitoring unit, including the number of rounds and the number of samples in the learning process.
8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.