An artificial intelligence aggregation method with robustness against malicious nodes

Abstract

The application relates to an artificial intelligence aggregation method with robustness against malicious nodes, and belongs to the technical field of mobile communication. The method changes the traditional federated learning aggregation mode, adopts a geometric median algorithm for global model aggregation, and simultaneously designs an asynchronous time weighting mechanism to utilize the obsolete global model parameters of users, so that the probability that a large number of malicious nodes cause breakdown of the geometric median aggregation algorithm during aggregation is reduced. The designed algorithm can reduce the influence of the global model under attack by malicious nodes, and has important practical application value.

Description

Technical Field

[0001] This invention belongs to the field of mobile communication technology and relates to an artificial intelligence aggregation method that is robust against malicious nodes. Background Technology

[0002] With the development of machine learning technology, federated learning, as a distributed learning method, has been widely applied in various scenarios involving privacy-sensitive data, such as personalized recommendations on mobile devices and the analysis of medical and health data. In practical applications of federated learning, asynchronous federated learning overcomes the problems of slow convergence speed and low time efficiency in synchronous federated learning, significantly improving the efficiency of federated learning. However, asynchronous federated learning also faces some challenges, one of the most important being the problem of malicious node attacks. In asynchronous federated learning, each participant can disrupt the global model's global aggregation update by sending corrupted global model parameters or tampering with communication data, thereby affecting the model's performance. Although there are many malicious node detection schemes in current federated learning, these schemes still have limitations and cannot detect all malicious nodes. Therefore, researching user participation mechanisms in asynchronous federated learning and designing a robust asynchronous federated learning user participation mechanism against malicious node attacks to reduce the impact on the global model when attacked by malicious nodes has significant theoretical and practical application value.

[0003] Malicious node detection is a critical task in federated learning, aiming to identify participating nodes that intentionally tamper with, interfere with, or disrupt the federated learning process. However, current malicious node detection mechanisms in federated learning have limitations. First, malicious node detection requires significant computational resources and communication overhead, reducing system efficiency and increasing the cost and complexity of federated learning. Second, malicious nodes employ adversarial strategies to circumvent detection mechanisms, such as forging legitimate gradient information or attacking verification mechanisms. Finally, when detection algorithms are inaccurate, false positives occur, incorrectly identifying ordinary nodes as malicious nodes, thus impacting the performance of federated learning. Since malicious node detection schemes cannot detect all malicious nodes, improving the robustness of asynchronous federated learning against malicious node attacks is crucial. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an artificial intelligence aggregation method that is robust against malicious nodes.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A robust AI aggregation method against malicious nodes, where the total number of users is... N ,user n The data uploaded to the central server is , For users n of k Value, that is It is based on the user's first The global model of the round is obtained through local training. This method includes the following steps:

[0007] Step 101: The server initiates a learning task to the client, sending the initial global model parameters to all users; and creates a collection to store all user-uploaded data.

[0008] Step 102: Check all user data in the collection. k The value determines whether to delete the user's data;

[0009] Step 103: Execute the adaptive waiting asynchronous federated learning algorithm to receive user data; when the central server collects the latest model parameters uploaded by the user, it checks whether the user's historical data exists in the set. If it exists, the historical data is replaced; if it does not exist, the data uploaded by the user is added to the set.

[0010] Step 104: End the adaptive waiting period for collecting user data; perform global model aggregation on the data in the collection;

[0011] Step 105: Check if the global model has met the task termination condition. If it has, stop the federated learning task; otherwise, return to step 102.

[0012] Furthermore, in the adaptive waiting asynchronous federated learning algorithm, the difference in accuracy between the final results with and without malicious nodes is reduced, specifically as follows:

[0013] In asynchronous federated learning where malicious nodes are involved, mitigating and reducing the impact of malicious node attacks on the global model is mainly reflected in reducing the difference in accuracy between the final results with and without malicious nodes participating in asynchronous federated learning. The optimization objectives are as follows:

[0014]

[0015]

[0016] In the formula, The accuracy of the global model without malicious nodes involved; The accuracy of the global model under conditions of malicious node involvement; This represents the number of malicious nodes. This represents the total number of all nodes. The number of rounds required to complete a federated learning task with the participation of malicious nodes; The number of rounds required to complete a federated learning task without the involvement of malicious nodes; For training The loss function for the global model parameters after round 1; and These are the thresholds for the average accuracy growth rate; C 1 represents the limit on the number of malicious nodes. C 2. Due to communication cost constraints, C 3 and C 4. Convergence conditions for federated learning with and without malicious nodes.

[0017] Furthermore, in step 102, the value of k represents the round in which the user obtains the model parameters through local training based on the global model in the kth round.

[0018] Further, in step 102, a threshold K is set; the k value of all user data in the set is checked, if... If the user's data is selected, it will be retained; otherwise, it will be deleted. The rounds of current federated learning.

[0019] Furthermore, in step 104, the global model aggregation adopts the geometric median algorithm, which obtains the latest round of global model parameters by solving the geometric median of all parameters in the set.

[0020] Furthermore, in step 104, the time weight for each user is calculated and applied to the calculation of the geometric median. The formula for calculating the time weight is: ;

[0021] in, For users Time weights; For users n of k value.

[0022] Furthermore, in step 104, the geometric median of the parameters in the set is calculated using the Weiszfeld algorithm.

[0023] Furthermore, in step 103, the adaptive waiting asynchronous federated learning algorithm includes:

[0024] Receive model parameters uploaded by users;

[0025] Check if the user's historical data exists in the collection;

[0026] If it exists, replace the historical data.

[0027] If it does not exist, add the data uploaded by the user to the collection.

[0028] Furthermore, in step 105, the task termination condition includes:

[0029] The global model achieves the preset accuracy;

[0030] Federated learning has reached the preset number of rounds;

[0031] Other preset conditions.

[0032] Furthermore, the server is a cloud server, and the client is a mobile terminal device.

[0033] The beneficial effects of this invention are as follows: This method changes the traditional federated learning aggregation method, adopts the geometric median algorithm for global model aggregation, and designs an asynchronous time weighting mechanism to make use of the user's outdated global model parameters, thereby reducing the probability that malicious nodes will cause the geometric median aggregation algorithm to break down due to their excessive number during a certain aggregation.

[0034] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0035] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0036] Figure 1 This is a diagram of the adaptive weighted federated learning model of the present invention;

[0037] Figure 2 This is a model diagram of the robust weighted aggregation asynchronous federated learning method in the present invention.

[0038] Figure 3 This is an example diagram of the RWAFL algorithm of the present invention. Detailed Implementation

[0039] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0040] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0041] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0042] like Figure 1 As shown, in order to improve the robustness of asynchronous federated learning and reduce the difference between the global model accuracy when malicious nodes are involved and when no malicious nodes are involved, Based on the robustness of the geometric median to outliers, a Robust Weighted Aggregation Asynchronous Federated Learning (RWAFL) algorithm was designed, which includes the following steps:

[0043] Let the total number of users be N ,user n The data uploaded to the central server is , For users n of k Value, that is It is based on the user's first The global model of the round is obtained through local training. For example... Figure 2 As shown, the algorithm flow is as follows:

[0044] 101: The central server initiates a federated learning task, setting the initial global model parameters... Send to all users; create a collection to store all user-uploaded data. Set threshold .

[0045] 102: Check Set All user data k Value, if If the user's data is selected, it will be retained; otherwise, it will be deleted. The rounds of current federated learning.

[0046] 103: Execute the adaptive waiting asynchronous federated learning algorithm to receive user data. When the central server collects the latest model parameters uploaded by the user, it checks whether the user's historical data exists in the set. If it exists, the historical data is replaced; otherwise, the user's uploaded data is added to the set. middle.

[0047] 104: End the adaptive waiting phase for collecting user data. (For the set...) The data in the middle is aggregated into a global model to obtain The following describes the global model aggregation and update process of the RWAFL algorithm.

[0048] In the Wheel, collection There is Data for each user, and for all parameters in the set, based on the corresponding parameters... The time weights for each user are calculated using the following formula. The formula for calculating its weights is:

[0049]

[0050] In the formula For users The time weight. The first In the wheel, the assembly There is The central server obtains the latest global model parameters by calculating the geometric median of all parameters in the set. The geometric median of all user parameters in the set. Parameters to the user The Euclidean distance in assigning its weights After that, the sum of the two numbers is minimized, that is:

[0051]

[0052] In the formula, This is the Euclidean norm. This invention utilizes the Weiszfeld algorithm to solve for the set... The geometric median of the median parameter.

[0053] 105: Check if the global model has met the task termination condition. If it has, stop the federated learning task; otherwise, return to step 102.

[0054] In asynchronous federated learning where malicious nodes are involved, mitigating and reducing the impact of malicious node attacks on the global model is mainly reflected in reducing the difference in accuracy between the final results with and without malicious nodes participating in asynchronous federated learning. The optimization objectives are as follows:

[0055]

[0056]

[0057] In the formula, The accuracy of the global model without malicious nodes involved; The accuracy of the global model under conditions of malicious node involvement; C 1. Regarding the limitation on the number of malicious nodes, in the study of the robustness of asynchronous federated learning in this invention, the number of malicious nodes is limited to no more than half of the total number of users participating in the overall federated learning. The number of malicious nodes; C 2 is limited by communication costs; The number of rounds required to complete a federated learning task with the participation of malicious nodes; The number of rounds required to complete a federated learning task without the involvement of malicious nodes; C 3 and C 4. Convergence conditions for federated learning with and without malicious nodes.

[0058] Let the total number of users be N ,user n The data uploaded to the central server is , For users n of k Value, that is It is based on the user's first The global model of the round is obtained through local training. For example... Figure 2 As shown, the algorithm flow is as follows:

[0059] 101: The central server initiates a federated learning task, setting the initial global model parameters... Send to all users; create a collection to store all user-uploaded data. Set threshold .

[0060] 102: Check Set All user data k Value, if If the user's data is selected, it will be retained; otherwise, it will be deleted. The rounds of current federated learning.

[0061] 103: Execute the adaptive waiting asynchronous federated learning algorithm to receive user data. When the central server collects the latest model parameters uploaded by the user, it checks whether the user's historical data exists in the set. If it exists, the historical data is replaced; otherwise, the user's uploaded data is added to the set. middle.

[0062] 104: End the adaptive waiting phase for collecting user data. (For the set...) The data in the middle is aggregated into a global model to obtain The following section will detail the global model aggregation and update process of the RWAFL algorithm.

[0063] In the Wheel, collection There is Data for each user, and for all parameters in the set, based on the corresponding parameters... The time weights for each user are calculated using the following formula. The formula for calculating its weights is:

[0064]

[0065] In the formula For users The time weight. The first In the wheel, the assembly There is The central server obtains the latest global model parameters by calculating the geometric median of all parameters in the set. The geometric median of all user parameters in the set. Parameters to the user The Euclidean distance in assigning its weights Then, add them together to get the sum. , minimize ,Right now:

[0066]

[0067] In the formula, For the Euclidean norm; use the Weiszfeld algorithm to solve for the set. The geometric median of the median parameter.

[0068] The Weiszfeld algorithm is an iterative algorithm for finding weighted average locations. It iteratively finds the location that minimizes the weighted average of the distances from each location to a given point. The Weiszfeld algorithm is commonly used to solve location estimation, weighted averaging, and some optimization problems. The iterative process involves calculating the distance from each location to the target point and updating the location based on the distance and weights until convergence to a solution. This invention utilizes... -approximate as a pair of Weiszfeld algorithms The convergence criterion for solving this problem is that it satisfies the following:

[0069]

[0070] Weiszfeld's algorithm stops iteratively solving the problem. For the first The global model parameters are updated after the aggregation of the rounds. It is an arbitrary small value that is set.

[0071] 105: Check if the global model has met the task termination condition. If it has, stop the federated learning task; otherwise, return to step 102.

[0072] The following example illustrates the algorithm execution process. Figure 3 As shown, the horizontal axis represents the time axis, the vertical line represents the aggregation time of the central server, the solid rectangle represents the time spent on local training and data transmission by the user, the dashed rectangle represents the offline time of the user device, and the threshold... Set to 2. User devices may be offline due to factors such as having other computing tasks, insufficient energy, or unwillingness to participate in federated learning. In addition, the time for local training and data transmission may vary in different rounds due to factors such as the user's remaining energy, the presence of other computing tasks, changes in location, and channel conditions.

[0073] At the start, users A, B, C, D, and E obtain the global model parameters. At this time, the set There are no user parameters in the data. In the first round, after user D's data arrives at the central server, the central server waits for... After that, no other user data arrived. Update the collection. The global model parameters uploaded by users A and D in the first round and Store in collection At this time, the set The central server utilizes a collection The parameters of users A and D are used to perform the first round of global model aggregation to obtain the global model. and will Send to users A and D. In the first round, user C's device went offline, and users E and B timed out.

[0074] In the second round, the central server first checks the set. Does the database contain parameters to be deleted, specifically those for users D and A? The value is 0, and the difference between this value and the current aggregation round number minus 1 does not exceed the threshold of 2, therefore the parameter... It will not be deleted. The central server received the parameters from user A. User B's parameters User E's parameters Update the collection ,Will Replace with ,Will , Add to collection .at this time The central server accesses the collection. The intermediate parameters are aggregated to obtain the global model parameters. .Will Send to users A, B, and E.

[0075] In the third round, the retrieval set The k values ​​of parameters for users A, B, D, and E are within the threshold range and do not need to be deleted. The central server receives the parameters from user A. User C's parameters Parameters of user D Update the collection ,Will Add to collection ,Will Replace with , Replace with At this time, the set Using sets The parameters are aggregated globally to obtain the result. Then the global model Send to users A, C, and D.

[0076] Similarly, in the fourth round, the retrieval set... The parameters for users B, C, D, and E, among which... , , The difference between the k value and the current round minus 1 is 3, which exceeds the threshold, therefore it is unavailable. , , Delete, user D's parameters It's available, so there's no need to delete it. The fourth round received parameters from user A. Update the collection , Replace with At this time, the set Therefore, the fourth round uses , Perform global model aggregation update to obtain and will Send to user A. The RWAFL algorithm is shown in Table 1.

[0077] Table 1

[0078]

[0079] The geometric median is robust to malicious nodes accounting for no more than 50% of the total. In the RWAFL algorithm, even if the number of malicious nodes exceeds 50% of the total number of users received in a particular aggregation update round, the parameters of other users' historical models are used in the global model aggregation. Although most users did not upload their updated model parameters in this round, or the central server did not receive model parameters from most users in this round, the central server utilizes these users' historical parameters to participate in the latest round of global aggregation. This effectively prevents malicious nodes from disrupting the geometric median aggregation due to their excessive number. Furthermore, applying appropriate time weights to these parameters not only balances the lagging effect of old parameters on global model aggregation but also promotes global model aggregation by assigning time weights to the old parameters.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A robust AI aggregation method against malicious nodes, where the total number of users is... N ,user n The data uploaded to the central server is , For users n of k value, k The value represents the user's value based on the first... k The round in which the model parameters are obtained through local training of the global model; that is... It is based on the user's first The model parameters obtained by locally training the global model of the wheel are characterized by: The method includes the following steps: Step 101: The server initiates a learning task to the client, sending the initial global model parameters to all users; and creates a collection to store all user-uploaded data. Step 102: Check all user data in the collection. k The value determines whether to delete the user's data; Step 103: Execute the adaptive waiting asynchronous federated learning algorithm to receive user data; when the central server collects the latest model parameters uploaded by the user, it checks whether the user's historical data exists in the set. If it exists, the historical data is replaced; if it does not exist, the data uploaded by the user is added to the set. Step 104: End the adaptive waiting period for collecting user data; perform global model aggregation on the data in the collection; Step 105: Check if the global model has met the task termination condition. If it has, stop the federated learning task; otherwise, return to step 102.

2. The robust artificial intelligence aggregation algorithm against malicious nodes according to claim 1, characterized in that: In the adaptive waiting asynchronous federated learning algorithm, the difference in accuracy between the final result with and without malicious nodes is reduced. The optimization objectives are as follows: In the formula, The accuracy of the global model without malicious nodes involved; The accuracy of the global model under conditions of malicious node involvement; This represents the number of malicious nodes. This represents the total number of all nodes. The number of rounds required to complete a federated learning task with the participation of malicious nodes; The number of rounds required to complete a federated learning task without the involvement of malicious nodes; For training The loss function for the global model parameters after round 1; and These are the thresholds for the average accuracy growth rate; C 1 represents the limit on the number of malicious nodes. C 2. Due to communication cost constraints, C 3 and C 4. Convergence conditions for federated learning with and without malicious nodes.

3. The AI ​​aggregation method with robustness against malicious nodes according to claim 1, characterized in that: In step 102, a threshold is set. K Check all user data in the collection. k Value, if satisfied If the user's data is selected, it will be retained; otherwise, it will be deleted. The rounds of current federated learning.

4. The AI ​​aggregation method with robustness against malicious nodes according to claim 1, characterized in that: In step 104, the global model aggregation uses the geometric median algorithm, which obtains the latest round of global model parameters by solving the geometric median of all parameters in the set.

5. The AI ​​aggregation method with robustness against malicious nodes according to claim 4, characterized in that: In step 104, the time weight of each user is calculated and applied to the calculation of the geometric median. The formula for calculating the time weight is as follows: ; in, For users Time weights; For users n of k value.

6. The AI ​​aggregation method with robustness against malicious nodes according to claim 4, characterized in that: In step 104, the geometric median of the parameters in the set is calculated using the Weiszfeld algorithm.

7. The AI ​​aggregation method with robustness against malicious nodes according to claim 1, characterized in that: In step 105, the task termination conditions include: The global model achieves the preset accuracy; Federated learning has reached the preset number of rounds; Other preset conditions.

8. The AI ​​aggregation method with robustness against malicious nodes according to claim 1, characterized in that: The server is a cloud server, and the client is a mobile terminal device.

Images