A joint learning aggregation method, device, equipment and medium
By receiving and processing the model parameters of each participant, calculating the marginal contribution degree of the model, and determining the parameters of the aggregation model based on these contribution degrees, the problem of poor joint learning aggregation effect is solved, and more accurate and efficient model aggregation is achieved.
Patent Information
- Application Number
- CN202411708003.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-04-01
AI Technical Summary
The aggregation effect of joint learning in the prior art is poor, which is affected by factors such as the non-uniformity of data distribution, the difference in model updates, and noise during communication.
By receiving the model parameters of the trained model sent by the device of each participant, the model parameters for each set containing the participant and other participants are determined, the marginal contribution of each participant to the set is calculated, and the parameters of the aggregate model are determined based on the degree of contribution.
The aggregation effect of joint learning is improved, the contribution degree of each participant is accurately determined, and the parameters of the aggregation model are effectively determined based on the weight of model parameters and contribution degree.
Smart Images
Figure CN119740676A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model technology, and in particular to a joint learning aggregation method, device, equipment and medium. Background Art
[0002] As society continues to develop and progress, federated learning, as a distributed machine learning paradigm, is becoming increasingly important. Within the framework of federated learning, multiple participants (such as different organizations or devices) collaborate to train models while protecting their respective data privacy, aiming to improve the model's overall performance and generalization capabilities. Model aggregation is a key step in federated learning, integrating the locally trained models of each participant to form a global model.
[0003] Currently, the most commonly used method for model aggregation is Federated Averaging (FedAvg). FedAvg is favored for its computational simplicity and high efficiency, particularly in scenarios with numerous participants and high network latency. However, despite its significant advantages in speed and feasibility, FedAvg's aggregation effectiveness can be affected by various factors, including but not limited to uneven data distribution, variability in model updates, and noise in the communication process. Summary of the Invention
[0004] The embodiments of the present application provide a joint learning aggregation method, apparatus, device, and medium for solving the problem of poor aggregation effect of joint learning in related technologies.
[0005] In a first aspect, an embodiment of the present application provides a joint learning aggregation method, the method comprising:
[0006] Receive model parameters of the trained model sent by each participant's device;
[0007] For each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; and determine the contribution of the participant based on the obtained marginal contribution of the participant to each set;
[0008] Parameters of the aggregation model are determined based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0009] In a second aspect, an embodiment of the present application further provides a joint learning aggregation device, the device comprising:
[0010] A receiving module, configured to receive model parameters of the trained model sent by a device of each participant;
[0011] a processing module configured to, for each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine a marginal contribution of the participant's model to the set based on a difference between the first accuracy and the second accuracy; and determine a contribution of the participant based on the obtained marginal contribution of the participant to each set;
[0012] The determination module is used to determine the parameters of the aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0013] In a third aspect, an embodiment of the present application further provides an electronic device, including:
[0014] a memory for storing program instructions;
[0015] The processor is configured to call the program instructions stored in the memory and execute the steps of any one of the above-mentioned joint learning aggregation methods according to the obtained program instructions.
[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method of joint learning aggregation as described in any one of the above items.
[0017] In an embodiment of the present application, an electronic device receives the model parameters of the trained model sent by the device of each participant; for each participant, determines each set including the participant and at least one participant in each participant; for each set, determines a first average parameter of the model parameters of each participant in the set, and determines a second average parameter of the model parameters of each participant in the set other than the participant; determines a first accuracy of sample identification when the parameter of the aggregate model is the first average parameter; and determines a second accuracy of sample identification when the parameter of the aggregate model is the second average parameter; determines the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; determines the contribution of the participant based on the obtained marginal contribution of the participant to each set; determines the parameters of the aggregate model based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant. Using the method provided in the embodiment of the present application, the electronic device can accurately determine the contribution of each participant, and based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant, accurately and effectively determine the parameters of the aggregate model, thereby improving the aggregation effect of joint learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 A schematic diagram of a process of a joint learning aggregation method provided in an embodiment of the present application;
[0020] Figure 2 A schematic diagram of a model aggregation process provided in an embodiment of the present application;
[0021] Figure 3 A schematic diagram of polymerization in a related technology provided in an embodiment of the present application;
[0022] Figure 4 A schematic diagram of a joint learning aggregation method provided in an embodiment of the present application;
[0023] Figure 5 A structural diagram of a joint learning aggregation device provided in an embodiment of the present application;
[0024] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solutions and advantages of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.
[0026] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.
[0027] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.
[0028] The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. The terms "first", "second", "third" and the like involved in this application are merely to distinguish similar objects, and do not represent a specific ordering of objects.
[0029] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0030] In order to solve the problem of poor aggregation effect of joint learning in related technologies, the embodiments of the present application provide a joint learning aggregation method, device, equipment and medium.
[0031] The joint learning aggregation method includes: an electronic device receiving model parameters of a trained model sent by a device of each participant; for each participant, determining each set including the participant and at least one participant among each participant; for each set, determining a first average parameter of the model parameters of each participant in the set, and determining a second average parameter of the model parameters of each participant in the set other than the participant; determining a first accuracy of sample identification when the parameters of the aggregation model are the first average parameters; and determining a second accuracy of sample identification when the parameters of the aggregation model are the second average parameters; determining the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; determining the contribution of the participant based on the obtained marginal contribution of the participant to each set; and determining the parameters of the aggregation model based on the weights determined corresponding to the model parameters of each participant and the contribution of each participant.
[0032] Example 1:
[0033] Figure 1 A schematic diagram of a process of a joint learning aggregation method provided in an embodiment of the present application, the process includes the following steps:
[0034] S101: Receive model parameters of a trained model sent by a device of each participant.
[0035] The joint learning aggregation method provided in the embodiment of the present application is applied to an electronic device, which may be a smart device such as a PC or a server. The server may be referred to as an aggregation server.
[0036] To achieve accurate and efficient aggregation, the electronic device can first issue training tasks to each participating device. These tasks may be based on different datasets, model architectures, or training strategies, thereby fully utilizing the computing resources of each participating device. Each participating device in the federated learning then conducts several rounds of local training to obtain a local model. After model training is complete, each participating device sends the model parameters of the trained model to the electronic device. The electronic device then receives the model parameters of the trained model sent by each participating device.
[0037] S102: For each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; determine the contribution of the participant based on the obtained marginal contribution of the participant to each set.
[0038] For each participant, the electronic device constructs different sets of combinations that include that participant and at least one other participant within each participant. For example, if three participants, A, B, and C, are currently being considered, the electronic device will determine the following sets: the set of participants A and B, the set of participants A and C, and the complete set that includes all participants A, B, and C.
[0039] The electronic device can determine the average parameter of the model parameters of each participant in each acquired set. For the sake of distinction, the average parameter can be referred to as the first average parameter. When the parameter of the aggregate model is determined to be the first average parameter, the sample is input into the aggregate model to obtain the output result of the aggregate model. Based on the output result of the aggregate model and the annotation result pre-saved for the sample, the accuracy of the aggregate model in identifying the sample is determined. For the sake of distinction, the accuracy can be referred to as the first accuracy. In one possible implementation, if the aggregate model is used for a classification task, the first accuracy can be determined based on whether the category predicted by the aggregate model is consistent with the labeled category; if the aggregate model is used for a regression task, the error between the predicted value and the labeled value of the aggregate model can be calculated, such as the mean square error, absolute error, etc.; specifically, how to calculate the accuracy of the aggregate model in identifying the sample is not limited here. The electronic device also determines the average parameter of the model parameters of each participant other than the participant in the set. For the sake of distinction, the average parameter can be referred to as the second average parameter. When the parameter of the aggregate model is determined to be the second average parameter, the sample is input into the aggregate model to obtain the output result of the aggregate model. Based on the output result of the aggregate model and the annotation result pre-saved for the sample, the accuracy of the aggregate model in identifying the sample is determined. For the sake of distinction, the accuracy can be referred to as the second accuracy.
[0040] For each set, after determining the first accuracy and the second accuracy of the set, the electronic device can determine the difference between the first accuracy and the second accuracy, and based on the difference, determine the marginal contribution of the participant's model to the set. In one possible implementation, the electronic device can determine the marginal contribution of the participant's model to each set.
[0041] After obtaining the marginal contribution of the participant's model for each set, the electronic device may determine the participant's contribution based on the obtained marginal contribution of the participant for each set. In one possible implementation, the electronic device may determine the participant's contribution as the average of the marginal contributions of the participant for each set.
[0042] S102: Determine parameters of an aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0043] After determining the contribution of each participant, the electronic device may determine parameters of the aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0044] In an embodiment of the present application, an electronic device receives the model parameters of the trained model sent by the device of each participant; for each participant, determines each set including the participant and at least one participant in each participant; for each set, determines a first average parameter of the model parameters of each participant in the set, and determines a second average parameter of the model parameters of each participant in the set other than the participant; determines a first accuracy of sample identification when the parameter of the aggregate model is the first average parameter; and determines a second accuracy of sample identification when the parameter of the aggregate model is the second average parameter; determines the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; determines the contribution of the participant based on the obtained marginal contribution of the participant to each set; determines the parameters of the aggregate model based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant. Using the method provided in the embodiment of the present application, the electronic device can accurately determine the contribution of each participant, and based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant, accurately and effectively determine the parameters of the aggregate model, thereby improving the aggregation effect of joint learning.
[0045] Example 2:
[0046] To improve the aggregation effect of joint learning, based on the above embodiment, in an embodiment of the present application, after determining each set including the participant and at least one of each participant, and before determining the contribution of the participant based on the obtained marginal contribution of the participant for each set, the method further includes:
[0047] Determine the weight of each participant to the set based on the number of participants included in the set;
[0048] Determining the contribution of the participant according to the obtained marginal contribution of the participant to each set includes:
[0049] The contribution degree of the participant is determined based on the obtained marginal contribution of the participant to each set and the weight of the participant to each set.
[0050] In order to improve the aggregation effect of joint learning, the electronic device may further determine the weight of the participant for each set, and then determine the contribution of the participant based on the weight of the participant for each set.
[0051] Specifically, the electronic device can determine the weight of the participant to the set based on the number of participants included in the set. In one possible implementation, the electronic device can determine the weight pre-saved for the number as the weight of the participant to the set.
[0052] After determining the participant's weight for each set, the electronic device may determine the participant's contribution based on the obtained marginal contribution of the participant for each set and the participant's weight for each set. Specifically, for each set, the electronic device may determine the product of the participant's marginal contribution to the set and the participant's weight for the set. After determining the product corresponding to each set, the electronic device may determine the sum of each obtained product and determine the sum as the participant's contribution.
[0053] The electronic device can determine the contribution of the participant using the following formula:
[0054]
[0055] in, is the contribution of the i-th participant, S iFor each participant, a large set is formed by each set that contains the participant. |s| is the number of elements in set s, ω(|s|) is the weight of the participant on the set, v(s)-v(s\{i}) is the marginal contribution of the participant to the set, v(s) is the first accuracy, s\{i} is the set after removing element i (that is, the participant), and v(s\{i}) is the second accuracy.
[0056] Example 3:
[0057] In order to improve the aggregation effect of joint learning, based on the above embodiments, in the embodiment of the present application, the weight of each participant to the set is determined according to the number of participants included in the set, including:
[0058] Determining a first difference between the number of participants included in the set and a predetermined value, and determining a second difference between the number of each participant and the number of participants included in the set;
[0059] determining a product of a factorial of the first difference and a factorial of the second difference;
[0060] The ratio of the product to the factorial of the number of each participant is determined as the weight of the participant to the set.
[0061] To improve the aggregation effect of joint learning, the electronic device may determine the difference between the number of participants included in the set and a preset value. For ease of distinction, this difference may be referred to as a first difference, where the preset value may be 1. Furthermore, the electronic device may determine the difference between the number of each participant and the number of participants included in the set. For ease of distinction, this difference may be referred to as a second difference.
[0062] The electronic device may determine the factorial of the first difference and the factorial of the second difference, and after determining the factorial of the first difference and the factorial of the second difference, determine the product of the factorial of the first difference and the factorial of the second difference. Furthermore, the electronic device may determine the factorial of the number of each participant, and determine a ratio of the product to the factorial of the number of each participant as the weight of the participant for the set.
[0063] Specifically, the electronic device may determine the weight of the participant to the set by the following formula:
[0064]
[0065] Wherein, ω(|s|) is the weight of the participant on the set, |s| is the number of participants included in the set, 1 is a preset value, and n is the number of each participant.
[0066] The embodiment of the present application is equivalent to determining the weight of each participant's model based on the Shapley Value. Among them, Shapley Value is a core concept in cooperative game theory, which is used to measure the contribution of each player in the cooperation. It has four desired properties: balance, symmetry, additivity and zero elements, and can comprehensively and fairly evaluate the contribution of each participant. In recent years, Shapley Value has been widely used in the field of machine learning, especially in feature selection, data valuation, multi-agent reinforcement learning and other aspects. It has shown significant advantages. By introducing Shapley Value, the contribution of each client in joint learning can be evaluated more accurately and fairly, thereby optimizing the model aggregation strategy. A method of fairly distributing benefits based on the average value of the marginal contribution generated by individuals joining the alliance.
[0067] The current mainstream federated learning aggregation method is Fed-avg, which generally includes the following four main steps:
[0068] Selection of participants: The electronic device can randomly select the required participants from the device pool, or use a custom algorithm to select the participants.
[0069] Parameter broadcast: The electronic device broadcasts global model parameters to the devices of selected participants. Local model training: The client synchronizes model parameters using its local data and then begins model training. This step may include local data preprocessing, such as data alignment, feature sampling, and cleaning, as well as the calculation of local models or intermediate results. Model aggregation: The participating devices send their trained local model parameters back to the electronic device, and the model parameters are aggregated into the global model. During aggregation, the model parameters of all participating devices are averaged.
[0070] The above steps will be iterated n times as required until the model converges or meets other preset requirements.
[0071] This embodiment of the application proposes a reinforcement learning-driven federated learning aggregation method based on Shapley Value space. The method is mainly divided into two phases: In the first phase, each federated learning participant trains its own model on a local dataset, uploads it to the aggregation server, and then calculates the Shapley Value of each participant. In the second phase, the reinforcement learning initial environment is constructed based on the Shapley Value space, and reinforcement learning is then used to perform model aggregation, ultimately achieving better model aggregation results.
[0072] Example 4:
[0073] In order to accurately determine the parameters of the aggregation model, based on the above embodiments, in the embodiment of the present application, the parameters of the aggregation model are determined based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant, including:
[0074] For each participant, determine the product of the model parameter of the participant and the weight determined corresponding to the contribution of the participant;
[0075] The sum of the products of each participant obtained is determined as the parameter of the aggregation model.
[0076] To accurately determine the parameters of the aggregation model, the electronic device can determine, for each participant, a weight corresponding to the participant's contribution, wherein the participant's contribution can be determined as the participant's corresponding weight, and the product of the participant's model parameters and the participant's weight can be determined. In this way, the electronic device can determine the product for each participant.
[0077] After determining the product of each participant, the electronic device may determine the sum of the obtained products of each participant, and determine the sum as a parameter of the aggregation model.
[0078] In order to improve the recognition effect of the model, based on the above embodiments, in the embodiment of the present application, the method further includes:
[0079] The parameters of the aggregate model are sent to the device of each participant, so that the device of each participant adjusts the model parameters of the trained model to the parameters of the aggregate model.
[0080] In an embodiment of the present application, after determining the parameters of the aggregate model, the electronic device may transmit the aggregate model parameters to each participating device. Each participating device then adjusts its own model parameters based on the received aggregate model parameters and continues the model training process. Once training is complete, the updated model parameters are transmitted back to the electronic device. This process is repeated until the aggregate model reaches convergence or a preset training termination condition is met.
[0081] It should be noted that each participant can use the reinforcement learning mechanism to train its local model. As a method that highly simulates human learning patterns, reinforcement learning is based on a trial-and-error strategy. The trial-and-error process includes two key steps: first, the agent interacts with a specific environment and observes the resulting results, such as rewards or penalties; then, the agent memorizes and adjusts its strategy based on these interaction results to optimize its own behavior. This optimization strategy aims to maximize rewards, that is, strive to obtain more rewards and reduce penalties, which is quite similar to the instinct of animals in nature to seek benefits and avoid harm. Therefore, reinforcement learning is a goal-oriented learning model that relies on interaction. Although reinforcement learning has shown significant results in many fields, because it requires exploration from scratch, the learning process is long and time-consuming, which is particularly undesirable in scenarios where efficient federated learning models are aggregated. In the embodiment of the present application, the aggregated model parameters are distributed to the devices of each participant through an electronic device, which significantly shortens the exploration cycle of the device of the participant and accelerates the learning process.
[0082] Figure 2 A schematic diagram of the model aggregation process provided in an embodiment of the present application.
[0083] Depend on Figure 2 It can be seen that the electronic device sends the parameters of the aggregation model to each participant, which is equivalent to selecting an initial state based on the Shapley value space. The local agent of the participant performs an action. t ) from the current state (State S t )Transfer to the next state (State S t+1 ), and in the process maximize the reward (r t ).
[0084] Example 5:
[0085] In order to improve the aggregation effect of joint learning, based on the above embodiments, in the embodiment of the present application, the first average parameter of the model parameter of each participant in the set is determined, including:
[0086] If the model parameter includes multiple parameters, for each pre-stored parameter type, a first average parameter of the model parameter of the parameter type for each participant in the set is determined.
[0087] In an embodiment of the present application, if the model parameters of the model include multiple parameters, when determining the first average parameter, the electronic device can determine the first average parameter of the model parameters of the parameter type for each participant in the set for each pre-saved parameter type.
[0088] After determining each first average parameter, the electronic device may determine a first accuracy of sample identification when each parameter of the aggregation model is the first average parameter.
[0089] In order to improve the aggregation effect of joint learning, based on the above embodiments, in the embodiment of the present application, the second average parameter of the model parameter of each participant other than the participant in the set is determined, including:
[0090] If the model parameter includes multiple parameters, for each pre-stored parameter type, a second average parameter of the model parameter of the parameter type for each participant in the set other than the participant is determined.
[0091] In an embodiment of the present application, if the model parameters of the model include multiple parameters, when determining the second average parameter, the electronic device can determine the second average parameter of the model parameters of the parameter type for each participant other than the participant in the set for each pre-saved parameter type.
[0092] After determining each second average parameter, the electronic device may determine a second accuracy of sample identification when each parameter of the aggregation model is the second average parameter.
[0093] The embodiment of the present application is equivalent to providing a joint learning aggregation method driven by Shapley Value reinforcement learning. It should be noted that joint learning generally uses the average fusion method (FedAvg) when aggregating models. Although this method is fast, it will affect the aggregation effect. Based on this, the present application considers using reinforcement learning to drive the aggregation process of joint learning, and the reinforcement learning module gives the aggregation weight to achieve intelligent decision tuning. At the same time, due to the problem of low learning efficiency in reinforcement learning, the present application constructs a reinforcement learning problem space with the help of Shapley Value, takes Shapley Value as a benchmark, skips the meaningless initial exploration process, greatly improves the learning efficiency of reinforcement learning, and promotes efficient and reasonable aggregation of joint learning.
[0094] The Fed-avg algorithm determines the weight of model aggregation based on the amount of data. Although simple and fast, it is difficult to achieve the optimal solution because it is only labeled by the amount of data. Due to the heterogeneity problem, the large amount of data on a certain node does not mean that the model generated by training this node is beneficial to the global model. The following situations may exist: 1) There is a lot of data but the quality is very poor, or even there are errors in the data labeling; 2) There is a lot of data but the scenario is single, resulting in category imbalance. This will cause the global model to lose generalization and low accuracy. Based on the above two reasons, the present invention first proposes to use reinforcement learning to drive the model fusion process of federated learning, and improve the effect of model fusion through the continuous trial and error and learning process of reinforcement learning. At the same time, in order to improve the detection efficiency of reinforcement learning in the federated learning model aggregation space, it is proposed to use Shapley Value space to guide the initial exploration process of reinforcement learning, thereby improving the efficiency of reinforcement learning.
[0095] Figure 3 A schematic diagram of aggregation in a related technology provided in an embodiment of the present application.
[0096] Depend on Figure 3 It can be seen that the electronic device receives the model parameters of the model sent by the device of each participant, and determines the average value of each received model parameter.
[0097] Figure 4 A schematic diagram of a joint learning aggregation method provided in an embodiment of the present application.
[0098] Depend on Figure 4 As can be seen, the electronic device receives the model parameters of the model sent by each participant's device, determines the contribution of each participant, and determines the parameters of the aggregate model based on the weights determined corresponding to the model parameters and contribution of each participant. The corresponding weights can be determined by structured value-based reinforcement learning (SV-based RL).
[0099] Example 6:
[0100] Based on the same inventive concept, the present application embodiment provides a joint learning aggregation device, please refer to Figure 5 , the device comprises:
[0101] A receiving module 501 is configured to receive model parameters of a trained model sent by a device of each participant;
[0102] Processing module 502 is configured to determine, for each participant, each set that includes the participant and at least one of the participants; determine, for each set, a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine, based on the difference between the first accuracy and the second accuracy, the marginal contribution of the participant's model to the set; and determine the participant's contribution based on the obtained marginal contribution of the participant to each set;
[0103] The determination module 503 is configured to determine parameters of the aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0104] In a possible implementation, the processing module 502 is further configured to determine the weight of the participant to the set according to the number of participants included in the set;
[0105] The processing module 502 is specifically configured to determine the contribution of the participant according to the acquired marginal contribution of the participant for each set and the weight of the participant for each set.
[0106] In one possible implementation, the processing module 502 is specifically configured to determine a first difference between the number of parties included in the set and a preset value, and determine a second difference between the number of each party and the number of parties included in the set; determine the product of the factorial of the first difference and the factorial of the second difference; and determine a ratio of the product to the factorial of the number of each party as the weight of the party to the set.
[0107] In one possible implementation, the determination module 503 is specifically used to determine, for each participant, the product of the model parameters of the participant and the weight determined corresponding to the contribution of the participant; and determine the sum of the products obtained for each participant as the parameters of the aggregation model.
[0108] In a possible implementation, the processing module 502 is further configured to send the parameters of the aggregate model to the device of each participant, so that the device of each participant adjusts the model parameters of the trained model to the parameters of the aggregate model.
[0109] In a possible implementation, the processing module 502 is specifically configured to determine, for each pre-stored parameter type, a first average parameter of the model parameters of the parameter type for each participant in the set if the model parameter includes multiple parameters.
[0110] In a possible implementation, the processing module 502 is specifically configured to determine, for each pre-saved parameter type, a second average parameter of the model parameter of the parameter type for each participant other than the participant in the set if the model parameter includes multiple parameters.
[0111] Example 7:
[0112] Based on the same inventive concept, the embodiment of the present application provides an electronic device that can implement the function of updating the above intentions. Please refer to Figure 6 The device includes a processor 601, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 communicate with each other through the communication bus 604.
[0113] The memory 603 stores a computer program. When the program is executed by the processor 601, the processor 601 performs the following steps:
[0114] Receive model parameters of the trained model sent by each participant's device;
[0115] For each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; and determine the contribution of the participant based on the obtained marginal contribution of the participant to each set;
[0116] Parameters of the aggregation model are determined based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0117] In a possible implementation, after determining each set including the participant and at least one of each of the participants, and before determining the contribution of the participant based on the obtained marginal contribution of the participant for each set, the method further includes:
[0118] Determine the weight of each participant to the set based on the number of participants included in the set;
[0119] Determining the contribution of the participant according to the obtained marginal contribution of the participant to each set includes:
[0120] The contribution degree of the participant is determined based on the obtained marginal contribution of the participant to each set and the weight of the participant to each set.
[0121] In a possible implementation, determining the weight of the participant policy to the set based on the number of participants included in the set includes:
[0122] Determining a first difference between the number of participants included in the set and a predetermined value, and determining a second difference between the number of each participant and the number of participants included in the set;
[0123] determining a product of a factorial of the first difference and a factorial of the second difference;
[0124] The ratio of the product to the factorial of the number of each participant is determined as the weight of the participant to the set.
[0125] In a possible implementation, determining the parameters of the aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant includes:
[0126] For each participant, determine the product of the model parameter of the participant and the weight determined corresponding to the contribution of the participant;
[0127] The sum of the products of each participant obtained is determined as the parameter of the aggregation model.
[0128] In one possible implementation, the method further includes:
[0129] The parameters of the aggregate model are sent to the device of each participant, so that the device of each participant adjusts the model parameters of the trained model to the parameters of the aggregate model.
[0130] In a possible implementation, determining a first average parameter of a model parameter of each participant in the set includes:
[0131] If the model parameter includes multiple parameters, for each pre-stored parameter type, a first average parameter of the model parameter of the parameter type for each participant in the set is determined.
[0132] In a possible implementation, determining a second average parameter of the model parameters of each participant in the set other than the participant includes:
[0133] If the model parameter includes multiple parameters, for each pre-stored parameter type, a second average parameter of the model parameter of the parameter type for each participant in the set other than the participant is determined.
[0134] The communication bus mentioned in the server above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.
[0135] The communication interface 602 is used for communication between the electronic device and other devices.
[0136] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk memory. Alternatively, the memory may be at least one storage device located away from the processor.
[0137] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.
[0138] Example 8:
[0139] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device. When the program is executed on the electronic device, the electronic device implements the following steps:
[0140] Receive model parameters of the trained model sent by each participant's device;
[0141] For each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameters of each participant in the set, and determine a second average parameter of the model parameters of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameters of the aggregate model are the first average parameter; and determine a second accuracy of sample identification when the parameters of the aggregate model are the second average parameter; determine the marginal contribution of the participant's model to the set based on the difference between the first accuracy and the second accuracy; and determine the contribution of the participant based on the obtained marginal contribution of the participant to each set;
[0142] Parameters of the aggregation model are determined based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant.
[0143] In a possible implementation, after determining each set including the participant and at least one of each of the participants, and before determining the contribution of the participant based on the obtained marginal contribution of the participant for each set, the method further includes:
[0144] Determine the weight of each participant to the set based on the number of participants included in the set;
[0145] Determining the contribution of the participant according to the obtained marginal contribution of the participant to each set includes:
[0146] The contribution degree of the participant is determined based on the obtained marginal contribution of the participant to each set and the weight of the participant to each set.
[0147] In a possible implementation, determining the weight of the participant policy to the set based on the number of participants included in the set includes:
[0148] Determining a first difference between the number of participants included in the set and a predetermined value, and determining a second difference between the number of each participant and the number of participants included in the set;
[0149] determining a product of a factorial of the first difference and a factorial of the second difference;
[0150] The ratio of the product to the factorial of the number of each participant is determined as the weight of the participant to the set.
[0151] In a possible implementation, determining the parameters of the aggregation model based on the model parameters of each participant and the weight determined corresponding to the contribution of each participant includes:
[0152] For each participant, determine the product of the model parameter of the participant and the weight determined corresponding to the contribution of the participant;
[0153] The sum of the products of each participant obtained is determined as the parameter of the aggregation model.
[0154] In one possible implementation, the method further includes:
[0155] The parameters of the aggregate model are sent to the device of each participant, so that the device of each participant adjusts the model parameters of the trained model to the parameters of the aggregate model.
[0156] In a possible implementation, determining a first average parameter of a model parameter of each participant in the set includes:
[0157] If the model parameter includes multiple parameters, for each pre-stored parameter type, a first average parameter of the model parameter of the parameter type for each participant in the set is determined.
[0158] In a possible implementation, determining a second average parameter of the model parameters of each participant in the set other than the participant includes:
[0159] If the model parameter includes multiple parameters, for each pre-stored parameter type, a second average parameter of the model parameter of the parameter type for each participant in the set other than the participant is determined.
[0160] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0161] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0162] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0164] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.
Claims
1. A joint learning aggregation method, characterized in that: The method comprises: Receive model parameters of the trained model sent by each participant's device; For each participant, determine each set that includes the participant and at least one of the participants; for each set, determine a first average parameter of the model parameter of each participant in the set, and determine a second average parameter of the model parameter of each participant in the set other than the participant; determine a first accuracy of sample identification when the parameter of the aggregate model is the first average parameter; and determine a second accuracy of sample identification when the parameter of the aggregate model is the second average parameter; determine the marginal contribution of the model of the participant to the set based on the difference between the first accuracy and the second accuracy; determine the contribution of the participant based on the acquired marginal contribution of the participant to each set; Based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant, the parameters of the aggregation model are determined.
2. The method according to claim 1, characterized in that After determining each set including the participant and at least one of each of the participants, and before determining the contribution of the participant according to the acquired marginal contribution of the participant for each set, the method further includes: According to the number of participants included in the set, determine the weight of the participant to the set; Determining the contribution of the participant according to the acquired marginal contribution of the participant to each set includes: The contribution of the participant is determined based on the obtained marginal contribution of the participant to each set and the weight of the participant to each set.
3. The method according to claim 2, characterized in that Determining the weight of the participant to the set according to the number of participants included in the set includes: Determine a first difference between the number of participants included in the set and a preset value, and determine a second difference between the number of each participant and the number of participants included in the set; determining a product of a factorial of the first difference and a factorial of the second difference; The ratio of the product to the factorial of the number of each participant is determined as the weight of the participant to the set.
4. The method according to claim 1, characterized in that: The determining of the parameters of the aggregation model based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant includes: For each participant, determine the product of the model parameter of the participant and the weight determined corresponding to the contribution of the participant; The sum of the products of each participant obtained is determined as the parameter of the aggregation model.
5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The parameters of the aggregate model are sent to the device of each participant, so that the device of each participant adjusts the model parameters of the trained model to the parameters of the aggregate model.
6. The method according to claim 1, characterized in that The determining of a first average parameter of a model parameter of each participant in the set includes: If the model parameter includes multiple parameters, then for each pre-saved parameter type, a first average parameter of the model parameter of the parameter type for each participant in the set is determined.
7. The method according to claim 1, characterized in that The determining of a second average parameter of the model parameter of each participant in the set other than the participant includes: If the model parameter includes multiple parameters, for each pre-saved parameter type, a second average parameter of the model parameter of the parameter type for each participant in the set other than the participant is determined.
8. A joint learning aggregation device, characterized in that: The device comprises: A receiving module, used for receiving the model parameters of the trained model sent by the device of each participant; A processing module, configured to determine, for each participant, each set including the participant and at least one of the participants; for each set, determine a first average parameter of a model parameter of each participant in the set, and determine a second average parameter of a model parameter of each participant other than the participant in the set; determine a first accuracy of sample identification when the parameter of the aggregate model is the first average parameter; and determine a second accuracy of sample identification when the parameter of the aggregate model is the second average parameter; determine a marginal contribution of the model of the participant to the set based on a difference between the first accuracy and the second accuracy; and determine a contribution of the participant based on the acquired marginal contribution of the participant to each set; The determination module is used to determine the parameters of the aggregation model based on the model parameters of each participant and the weights determined corresponding to the contribution of each participant.
9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in the method of joint learning aggregation according to any one of claims 1 to 7 according to the obtained program instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the steps included in the method for joint learning aggregation as claimed in any one of claims 1 to 7.