Federal learning method and system for coping with hostile attack in non-independent identically distributed scene

Abstract

The invention belongs to the technical field of artificial intelligence, discloses a federal learning method and system for coping with hostile attacks in a non-independent identically distributed scene, and designs a user contribution degree model based on marginal contribution and a user credibility model based on a rank sum ratio. According to the method, the concept of evolutionary clustering is adopted, scores of all users in the round are calculated by utilizing the contribution degree of each user in the round and the user credibility calculated in the previous round, and on the basis, the users participating in aggregation are selected by utilizing the idea of a greedy algorithm to be added into a set of a user aggregation model one by one. Meanwhile, a root data set is maintained at a server side, and a server model trained by using the root data set is used as an initial aggregation model of each round. In a non-independent identically distributed scene, the performance of the algorithm is evaluated on two public data sets with different proportions of malicious users. A large number of experimental results show that the accuracy of the algorithm in a test set is superior to that of various baseline algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a federated learning method and system for coping with malicious attacks in non-independent and identically distributed scenarios. Background Art

[0002] With the explosive growth of data in recent years, machine learning has been well developed and applied. Now machine learning has penetrated into all walks of life. Traditional machine learning needs to collect user data in one place for model training. However, due to the emergence of more and more privacy and security issues, countries and industries have introduced increasingly stringent data privacy protection regulations, which makes it difficult to collect user data in one place. Training data is becoming more and more fragmented, forming multiple "data island" distributed databases. In order to train machine learning models as much as possible while protecting user privacy, Google proposed Federated Learning (FL) in 2016. Federated learning is a special type of distributed machine learning. Its essence is that a large number of clients jointly train a global model under the coordination of the server. Specifically, each client device trains a local model with its own local data and uploads the model parameters to the central server. After receiving the local model parameters, the server aggregates them into a global model according to a certain strategy, and then sends the global model to each client. The client updates its local model with the global model and starts the next round of training. This process is repeated until the global model converges. Since the client uploads model parameters instead of original data in each round, it can prevent user privacy leakage and save a lot of communication resources.

[0003] Federated learning has undoubtedly opened up a new research direction for artificial intelligence. It provides a new training method to build personalized models without infringing user privacy. However, since federated learning is a relatively complex open system, there are a large number of terminal devices in the system, which can dynamically join or leave. This feature gives malicious attackers an opportunity to manipulate the training of local models and ultimately affect the accuracy of the global model. Some poisoning attacks attempt to prevent the model from being learned and deviate from the original goal. They often affect the aggregation of the global model by injecting malicious clients into the federated learning system, or tampering with the data of the malicious client, or even directly creating wrong model parameters, such as label reversal attacks and model poisoning attacks. In addition, due to the large differences between the local data of each client, the client data presents the characteristics of non-independent and identically distributed. Non-independent and identically distributed refers to the situation where the data distribution between different client data sets is inconsistent and does not obey the same sampling. This will cause the local model to be updated to the local optimal point during the local training stage, and the local optimal point is far away from the global optimal point. Therefore, non-independent and identically distributed causes the local model to converge in different directions, which will not only reduce the convergence speed of the global model, but also reduce the performance of the global model. Since most malicious attack detection algorithms are implemented through the differences between different users, and non-IID itself will lead to differences between users, non-IID provides malicious users with greater attack and hiding convenience. Therefore, how to deal with malicious attacks in non-IID scenarios and ensure the accuracy of the global model is an urgent problem to be solved in the field of federated learning.

[0004] The existing technology has the following technical problems and defects in industrial applications:

[0005] 1. Openness and complexity lead to vulnerability to malicious attacks: The openness and dynamic nature of the federated learning system makes it extremely vulnerable to malicious attacks in the process of collecting and aggregating user model parameters. Malicious attackers can manipulate the training of local models, inject malicious clients or tamper with data, or even create incorrect model parameters, causing the learning direction of the global model to deviate from the original goal. These poisoning attacks will not only prevent the model from being learned correctly, but also seriously affect the convergence speed and accuracy of the global model, thereby weakening the reliability and effectiveness of the entire system.

[0006] 2. Non-IID data characteristics: Due to the large differences between the data of each client, the data presents non-IID characteristics. Non-IID data makes it easy for the local model to fall into the local optimal point during the training process, and these local optimal points are far away from the global optimal point, affecting the convergence speed and performance of the global model. This inconsistency in data distribution not only reduces the performance of the global model, but also increases the uncertainty and complexity of the training process.

[0007] 3. Difficulty in detecting malicious attacks: The non-independent and identically distributed data characteristics increase the difficulty of malicious attack detection. Most existing malicious attack detection algorithms rely on differences between users to identify malicious behavior. However, non-independent and identically distributed data itself will lead to differences in model update directions and results between users, which greatly reduces the effectiveness of algorithms based on differences to detect malicious users. Therefore, in this case, it is easier for malicious users to hide their attack behaviors, which brings greater security risks to the federated learning system.

[0008] 4. Accuracy and convergence issues of the global model: In a federated learning system, ensuring the accuracy and fast convergence of the global model is key. However, the presence of non-IID data and malicious attacks will seriously affect the convergence speed and final performance of the global model. The inconsistency of the model update direction and the interference of malicious data will cause the learning process of the global model to become unstable, reducing the accuracy and effectiveness of the global model. In this case, the existing federated learning technology is difficult to effectively meet the actual needs of complex industrial applications and urgently needs to be improved and optimized.

[0009] In summary, the main problems faced by existing technologies in industrial applications include: openness and complexity make them vulnerable to malicious attacks, non-independent and identically distributed data characteristics affect the convergence and performance of the model, malicious attack detection is difficult, and the accuracy and convergence of the global model are problematic. These problems limit the widespread application of federated learning technology in the industrial field, and effective solutions are urgently needed. Summary of the invention

[0010] In response to the problems existing in the prior art, the present invention provides a federated learning method and system for dealing with malicious attacks in a non-independent and identically distributed scenario.

[0011] The present invention designs a federated learning method for dealing with malicious attacks in non-IID scenarios. The method is applicable to non-IID scenarios and aims to reduce the negative impact of malicious users and enhance the robustness of the federated learning system. The method includes: using the root data set and marginal contribution to evaluate the contribution of each user model to the global model; using the rank sum ratio of user contribution to measure user credibility; using the contribution of this round and the credibility calculated in the previous round to calculate the user score, and selecting the federated learning user model to participate in aggregation according to the score.

[0012] In a federated learning system with N clients and one central server, each client uses C i (i=1, 2, ...N), and the central server is represented by C0. Each client has its own local data set Where |D i |For client C iThe present invention uses vector ω∈R d To represent the training model parameters, f(ω; b) is used to represent the loss of model ω on the sample data set b, and client C i The objective function can be expressed as The task of the central server is to find a model ω * Minimize the weighted average loss of all clients, that is,

[0013]

[0014] where α i It is the weight of the client model when it is aggregated.

[0015] Furthermore, the federated learning method for dealing with malicious attacks in the non-IID scenario includes the following steps:

[0016] Step 1: For a specific federated learning task, collect a clean small dataset D0, called the root dataset. Taking the digital image classification task as an example, the so-called clean means that the classification labels of all images are correct, which can be achieved by manually correcting the labels;

[0017] Step 2: Server C0 initializes a global model ω 0 ;

[0018] Step 3: The server sends the global model ω t-1 (i=1, 2, ...T) is broadcast to each client, where t represents the communication round. Client C i After receiving the global model, (i=1, 2, ...N) uses it to initialize its own local model, and uses the local data set to train the local model to obtain the updated local model ω i t , and ω i t Upload to the server. Among them, the model update process is the process of minimizing the loss function using stochastic gradient descent, that is,

[0019]

[0020] Where η is the learning rate.

[0021] Step 4: The server uses D0 to synchronously train the neural network model ω0 with each client as the server-side model;

[0022] Step 5: After receiving the models of all users, the server calculates the score of each user and selects the user to participate in the aggregation of the global model together with the server-side model to obtain the aggregated global model ω t .

[0023] Step 6: Test the global model on the test dataset t accuracy;

[0024] Step 7: Repeat steps 3 to 6 until the global model converges.

[0025] Furthermore, the user rating in step five is solved by designing a contribution model and a credibility model.

[0026] The contribution model is designed as follows: In Marginal Gain Profit Sharing, the benefit of a participant in a team is the utility the team gains when they join the team. The participants join in a certain order, and the marginal contribution of each participant is based on the marginal benefit calculated when they join the team in the same order.

[0027] Consider a set of participants Assumptions It is a subset of participants, used to complete a task together, the utility function is the utility of subset S for the task. The marginal contribution to a subset can be expressed as

[0028] The present invention defines the marginal contribution of users in federated learning. Participants are equivalent to clients in federated learning. They need to join the user set on the server side in a certain order to complete the aggregation task together. The utility function here is The marginal contribution of each client model is the marginal benefit of adding it to the current user set.

[0029] The present invention maintains a clean small data set on the server side, called the root data set D0. The server uses this data set to train synchronously with each user side. The obtained model is called the server model ω0. This model is used as the initial user subset for each round of aggregation, that is, S t , calculate the marginal contribution of each user when joining. In this paper, the accuracy of the aggregation model in the test is used as the utility function, that is,

[0030]

[0031] Among them, D T is the test set, t indicates that the current selection is the tth round of aggregation user, k indicates the kth user who needs to choose to join the aggregation, and i indicates the calculation user C i marginal contribution.

[0032] Since marginal contribution is a positive indicator, that is, the higher the marginal contribution, the greater the probability that the user is a good user, and the greater the help to improve the model performance. Therefore, the present invention uses the extreme difference normalization method of positive indicators to normalize the marginal contribution in the round of aggregated user selection:

[0033]

[0034] The above formula can map the data to [0, 1] without changing the data distribution. The normalized marginal contribution is used as the user's contribution in this selection.

[0035] The design of the credibility model is as follows:

[0036] The credibility model is based on the rank sum ratio method. The rank sum ratio (RSR) is a statistical information analysis method used in multi-index comprehensive evaluation and other aspects. Its basic idea is to obtain the dimensionless statistic RSR through rank conversion in a matrix of n rows (n evaluation objects) and m columns (m evaluation indicators or levels), and use RSR to measure the quality of the evaluation object. The general calculation steps of the rank sum ratio are as follows:

[0037] (1) Select the evaluation index according to the purpose of the evaluation, and determine whether the evaluation index is a benefit-based index or a cost-based index. The larger the value of the benefit-based index, the better, while the smaller the value of the cost-based index, the better. Construct a matrix X with n rows (n evaluation objects) and m columns (m evaluation indicators or levels).

[0038]

[0039] (2) Sort by the value of the evaluation index, with benefit-type indicators ranked from large to small and cost-type indicators ranked from small to large, and obtain the rank R of each indicator ij , using rank R ij To replace the original evaluation index value, we get the rank matrix R

[0040]

[0041] (3) Calculate the rank sum ratio (RSR) between each evaluation object and sort the evaluation objects according to the RSR value. The calculation expression of the rank sum ratio is as follows:

[0042]

[0043] Where i = 1, 2, ..., n, j = 1, 2, ..., m, W j is the weight of each evaluation index. RSR value is dimensionless and has a minimum value Maximum RSR max =1.

[0044] Assume that in a certain round of user aggregation, k (k < N) user models are selected. In this way, each time a user selects, the present invention will obtain a user contribution ranking. After the round of user selection, the present invention will obtain k groups of user contribution rankings. Here, the evaluation object of the present invention is all users. The evaluation index is the contribution of the user in each round. The evaluation matrix can be obtained:

[0045]

[0046] In the above formula, is the normalized contribution of the nth user in the mth user selection in the tth round (which can be regarded as the mth evaluation indicator).

[0047] Since contribution is a benefit-based indicator, sort the normalized contributions of the users in the jth column in descending order to obtain the rank R of each indicator value in this column. ij The same process is applied to the remaining columns, and then the original evaluation index value is replaced by the rank of the evaluation index value, and then the present invention can obtain the contribution rank matrix R V :

[0048]

[0049] Then the rank sum ratio of each user is calculated. Here, since each evaluation index is the contribution rank, the weight is set to 1, and the rank sum ratio of the user in the tth round is

[0050]

[0051] The higher the user's contribution ranking, that is, the smaller the RSR, the greater the probability that the user is a benign user, and the more significant the performance improvement of the global model. In order to keep the same dimension as the contribution, the present invention normalizes the RSR to obtain the user credibility Cr. Note that RSR is a cost-type indicator here, that is, the smaller the RSR, the higher the credibility. The user credibility of the tth round is

[0052]

[0053] From the previous analysis, we can see that contribution is a metric calculated when the current user makes a selection, and credibility is a metric calculated based on the user selection in the previous round. According to the principle of evolutionary clustering, when there are two conflicting metrics that need to be optimized at the same time, the following rules need to be met:

[0054] (1) Clustering at any point in time must follow the characteristics of the current results;

[0055] (2) The clustering cannot change much from one time slice to the next.

[0056] When applied to federated learning, each user selection needs to be made based on the currently calculated contribution, and the user with higher credibility calculated last time also needs to be considered. Therefore, the present invention assigns different weights to contribution and credibility to calculate the user score of the user selection this time. The weight of contribution is β, and the weight of credibility is 1-β. User C i The rating is

[0057]

[0058] When β=0, the user rating of this round depends entirely on the user credibility of the previous round; when β=1, the user rating of this round is based only on the contribution of the current round, so the present invention sets the β interval range to [0.5, 1].

[0059] Furthermore, the user selection algorithm in step five is based on the idea of ​​a greedy algorithm.

[0060] In each round of user selection, the present invention calculates the marginal contribution of each user model relative to the selected user subset and normalizes it to obtain the contribution of the user in that round Then based on the contribution and credibility of the previous round of users Calculate user score The user with the highest score is selected to join the aggregated user set. The present invention continues to calculate user scores and make selections until the aggregation threshold p is reached. Specifically, if the contribution of a user who has selected to join the aggregation stage for p consecutive times is less than or equal to 0, the present invention terminates the user selection process for this round. At the same time, based on the user's contribution, the present invention calculates the user credibility of this round. and used in subsequent calculations.

[0061] Furthermore, the user aggregation algorithm in step five is based on the FedAvg algorithm. Since the clients participating in the federated learning task have different data sets of different sizes and there is heterogeneity in the amount of data, the weight of each user model when aggregated into a global model should also be different. Specifically, users with large amounts of data should have a larger weight; users with smaller amounts of data should have a smaller weight. This is fair on the one hand, and conforms to the distribution law of data sets of centralized machine learning models on the other. Therefore, the present invention uses the ratio of the size of the client data set to the sum of the sizes of all client data sets as the weight of the client when participating in the global model aggregation, and the formula is expressed as:

[0062]

[0063] Among them, S t is a subset of users selected by a user selection algorithm.

[0064] Another object of the present invention is to provide a federated learning method and system for responding to malicious attacks in a non-independent and identically distributed scenario using the above method, wherein the federated learning method and system for responding to malicious attacks in a non-independent and identically distributed scenario include:

[0065] The contribution model module is used to measure the user's contribution to the global model when calculating the user rating. The contribution of each user is quantified by calculating the marginal contribution of each user to the user subset selected for aggregation when joining. The marginal contribution is calculated by the difference in accuracy of the aggregation model on the test data set before and after the user joins. In order to calculate the subsequent user rating, the Min-Max normalization method is used to uniformly normalize the marginal contribution value of each user. The test data set here is divided from the training set, and in specific application tasks, it can be collected from each client;

[0066] The credibility model module is used to measure the credibility of the user when calculating the user score. The higher the credibility, the higher the probability that the user is a benign user; the lower the credibility, the lower the probability that the user is a benign user. The credibility of the user is quantified by calculating the rank sum ratio of the user contribution in the previous communication round. That is, the smaller the rank sum ratio of the user contribution, the higher the credibility of the user, and the higher the probability of being selected in the subsequent user selection. Like the contribution model, the rank sum ratio of each user also needs to be normalized. It should be noted here that because the rank sum ratio is a cost-type indicator, in order to weight it with the contribution later, the Min-Max normalization method needs to be modified, and the normalized result is subtracted from 1 to obtain the final normalized rank sum ratio. The present invention uses the value of the normalized rank sum ratio as the credibility;

[0067] The user rating calculation module is used to comprehensively measure the user's contribution and credibility. Based on the principle of evolutionary clustering, the contribution of this round and the credibility of the previous round are considered at the same time, with the contribution of this round as the main factor. Different weights are assigned to contribution and credibility to calculate the user rating, and the user rating is recalculated after each user set of the aggregation model is updated;

[0068] The user selection module is used to select users to participate in the aggregation of the global model. In each round of communication, after the server receives the model updates of all users, it calculates the scores of each user, selects the user with the highest score to join the set of users of the aggregated model, and then updates the user score, and continues to select the next user to participate in the aggregation, until the contribution of the user who has been selected to join the aggregation stage for p consecutive times is less than or equal to 0, and the present invention terminates the user selection process of this round;

[0069] The local training module is used for user devices to use local data sets to train federated learning tasks and complete message interaction and model sharing between users and servers;

[0070] The model aggregation module calculates the aggregation weights of the users in the model aggregation set and performs weighted aggregation on the user models to obtain the updated global model and conduct the next round of training.

[0071] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the federated learning method for responding to malicious attacks in a non-independent and identically distributed scenario.

[0072] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the federated learning method for responding to malicious attacks in a non-independent and identically distributed scenario.

[0073] Another object of the present invention is to provide an information data processing terminal, which is used to implement the federated learning system for responding to malicious attacks in the non-independent and identically distributed scenario.

[0074] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0075] First, in the federated learning scenario, the security of the learning process is crucial. In order to improve the robustness of the federated learning system to cope with malicious user attacks, the present invention discloses a federated learning method for dealing with malicious attacks in a non-independent and identically distributed scenario, by adding a contribution model and a credibility model, and introducing the ideas of evolutionary clustering and greedy algorithms to exclude malicious user models from the global model aggregation. The present invention is applicable to a variety of application scenarios such as smart cities, smart communications, smart agriculture, smart medical care, and smart homes.

[0076] The present invention considers the poisoning attack scenario in the non-independent and identically distributed scenario. First, the non-independent and identically distributed characteristics between clients themselves will lead to large differences between client models, and the models will be updated in different directions, which will eventually affect the convergence and accuracy of the global model; secondly, the purpose of malicious attacks is to make the global model deviate from the correct direction. Taking poisoning attacks as an example, malicious attacks will cause the client model update direction to be significantly different from the correct model update direction or even opposite, which will also lead to large differences between client models. Therefore, the non-independent and identically distributed characteristics between clients will also increase the difficulty of identifying malicious users, making it impossible to distinguish malicious users from benign users by the differences between users. The present invention takes the accuracy of the global model as the goal, and selects users by the contribution and credibility of the user model to the global model. This is because according to past experience, the purpose of malicious users is to reduce the performance of the global model, so in the non-independent and identically distributed scenario, even if the differences between users are large, the contribution of malicious users is always relatively low, and thus the credibility is also relatively low. The present invention can thereby screen out users with a higher probability of benign users to participate in the aggregation of the global model, ultimately reducing the impact of malicious attacks and improving the robustness of federated learning in the non-independent and identically distributed scenario.

[0077] (1) Improved robustness: By designing a user selection algorithm, the performance of the global model of the federated learning system in the face of malicious poisoning attacks is greatly improved;

[0078] (2) Model performance improvement: By designing a user selection algorithm, the performance of the global model of the federated learning system in non-IID scenarios is greatly improved;

[0079] (3) User score evaluation: The present invention provides a method for evaluating the user scores of federated learning. The score of each user can be well evaluated by the user contribution of this round and the user contribution calculated in the previous round. The higher the score, the greater the probability that the user is a benign user.

[0080] In summary, the present invention proposes a robust federated learning method for dealing with malicious attacks in non-IID scenarios, which can significantly resist malicious user attacks in non-IID scenarios, while improving the accuracy of the global model, and has strong robustness and easy implementation. This will have a positive impact on the application fields of federated learning such as smart cities, smart healthcare, and financial services.

[0081] Second, the technical solution of the present invention provides a solution to deal with malicious attacks for the problem of federated learning in non-IID scenarios. After the technical solution of the present invention is transformed into a specific product, the expected benefits and commercial value are mainly reflected in the following aspects:

[0082] Data privacy protection: Federated learning avoids centralizing client data in one place, but instead shares client models, thus avoiding the leakage of private data. This privacy protection mechanism will attract many organizations and individuals to use this technology, especially in industries involving sensitive data, such as medical, financial, and personal privacy-related application fields;

[0083] Improve system security: Strict global model aggregation user screening conditions exclude malicious users from aggregation, reduce the impact of malicious attacks, make federated learning aggregation safer, and the accuracy of the global model is closer to the performance of the model trained by centralized machine learning;

[0084] Improve the accuracy of the model: In a distributed machine learning system, the data sets of terminal devices are mostly non-independent and identically distributed. This invention limits malicious users from participating in aggregation while ensuring the diversity of user models participating in aggregation, thereby improving the convergence speed and accuracy of the global model.

[0085] Potential market opportunities: With the rapid development of federated learning, this invention has potential market opportunities in many industries and fields. Healthcare, smart cities, financial services, the Internet of Things and other fields all require secure federated learning systems with privacy protection and efficient data collaboration capabilities, and these fields will become potential commercial application areas of this invention.

[0086] The technical solution of the present invention solves the problem of how to deal with malicious attacks in non-independent and identically distributed scenarios. In non-independent and identically distributed scenarios, due to the differences between client data sets, the update directions of the trained local models are also quite different. Most algorithms for detecting malicious users at home and abroad are based on the differences between users. In scenarios where malicious users account for a small proportion, users with small differences from the global model are considered to be benign users, and this algorithm is obviously not applicable in non-independent and identically distributed scenarios. The algorithm designed by the present invention does not rely on the differences between user models, so it is not affected by non-independent and identically distributed. At the same time, the present invention protects the diversity of users participating in aggregation to a certain extent, accelerates the convergence speed of the model, and improves the accuracy of the model.

[0087] The technical solution of the present invention overcomes the technical bias that in order for the detection algorithm to work in a federated learning system, the proportion of malicious users needs to be controlled within a certain range. It is more common that in the federated learning scenarios applicable to most algorithm regulations, the proportion of malicious users cannot exceed 50%. However, this is obviously unrealistic in the deployment and application scenarios of federated learning, because in actual application scenarios, malicious attacks are carried out in secret, and it is therefore impossible to count and control the proportion of malicious users. In scenarios where the proportion of malicious users is relatively high, the current algorithms do not work. The present invention proposes an innovative robust federated learning algorithm in non-independent and identically distributed scenarios, which can ensure that the model can achieve better performance under various malicious user proportions, overcoming the limitations of previous studies.

[0088] Third, the present invention proposes a new method for dealing with non-IID data and malicious attacks in a federated learning system. The following is a detailed description of the specific implementation scenarios of the invention, the problems solved, and the technical progress achieved:

[0089] First, the present invention is implemented in a federated learning system including multiple clients and a central server. In this system, each client holds a non-IID local dataset and contains malicious users who carry out poisoning attacks. To address this challenge, the present invention introduces a key step on the server side: using a clean small dataset, namely the root dataset, to train the neural network model synchronously with each client to build a more reliable server-side model.

[0090] Secondly, in response to poisoning attacks carried out by malicious users, the present invention proposes a scoring mechanism based on user contribution and credibility. After receiving the models uploaded by all users, the server first calculates the marginal contribution of each user model relative to the aggregated model set, and then selects the user model with the highest score for aggregation based on the user's historical credibility score. This mechanism can not only effectively eliminate the influence of malicious users, but also ensure the stability and accuracy of the global model.

[0091] Thirdly, the significant technical progress of the present invention lies in its ability to deal with non-IID data and malicious attacks. By using the root data set to build a server-side model, the present invention can correct the deviation caused by non-IID data to a certain extent. At the same time, through the user rating mechanism, the present invention can effectively identify and eliminate the interference of malicious users, thereby ensuring the reliability and robustness of the global model.

[0092] Finally, the implementation process of the present invention is simple and clear, and has good scalability and flexibility. Whether in terms of data scale, model complexity or system architecture, the present invention can be adjusted and optimized according to actual needs. In addition, through continuous iteration and updating, the present invention can continuously improve the performance of the global model to adapt to the ever-changing data environment and application scenarios.

[0093] Fourth, the present invention solves the technical problems of equipment maintenance and product quality control in existing industrial applications through intelligent predictive maintenance and intelligent quality control systems. In traditional industrial production, equipment failures and product quality problems are often difficult to predict, resulting in frequent production line shutdowns and high product defect rates. Due to the lack of real-time monitoring and prediction methods, production efficiency and product quality are difficult to guarantee, thereby increasing production costs and operational risks.

[0094] The intelligent predictive maintenance system significantly improves the foresight and accuracy of equipment maintenance through real-time data collection, data analysis and fault prediction models. Sensors monitor the operating status of key equipment in real time, and the data analysis module promptly identifies abnormal conditions and predicts the occurrence of faults through machine learning algorithms. This not only greatly reduces the unplanned downtime of the production line and improves production efficiency, but also reduces the direct and indirect losses caused by equipment failures, and significantly improves the reliability and stability of the production line.

[0095] The intelligent quality control system realizes real-time monitoring of the production process and precise control of product quality through data collection, quality control models and abnormality recognition modules. The system can collect various data of the production process in real time, and quickly detect and predict product quality problems through big data analysis technology and quality control models. Intelligent quality control not only effectively reduces the product defect rate and improves production quality and efficiency, but also reduces the cost and errors of manual intervention, and improves the automation level of the production process and customer satisfaction.

[0096] In summary, the present invention has achieved significant technological progress in industrial applications. By introducing big data analysis and intelligent technology, the efficiency and accuracy of equipment maintenance and product quality control have been greatly improved. The implementation of intelligent predictive maintenance system and intelligent quality control system not only optimizes the production process, reduces costs and risks, but also enhances the competitiveness of enterprises in the market, providing strong support for the continuous innovation and development of the industrial field. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 It is a scenario diagram of a federated learning system for responding to malicious attacks in a non-independent and identically distributed scenario provided by an embodiment of the present invention;

[0098] Figure 2It is a flow chart of a federated learning method for coping with malicious attacks in a non-independent and identically distributed scenario provided by an embodiment of the present invention;

[0099] Figure 3 It is a functional module diagram of a federated learning method for coping with malicious attacks in a non-independent and identically distributed scenario provided by an embodiment of the present invention;

[0100] Figure 4 It is a simulation result diagram of the parameter p in the federated learning method for coping with malicious attacks in a non-independent and identically distributed scenario provided by an embodiment of the present invention;

[0101] Figure 5a to Figure 5c This is a simulation comparison diagram of the accuracy of the model trained on the CIFAR-10 dataset on the test set provided by an embodiment of the present invention;

[0102] Figure 6a to Figure 6c This is a simulation comparison chart of the accuracy of the model trained on the Fashion-MNIST dataset on the test set provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0104] Reference Figure 1 , the specific implementation scenario of the present invention is further described in detail. In a federated learning system with N clients and one central server, each client uses C i (i=1, 2, ...N), and the central server is represented by C0. Each client has its own local data set Where |D i |For client C i The present invention uses vector ω∈R d To represent the training model parameters, f(ω; b) is used to represent the loss of model ω on the sample data set b, and client C i The objective function can be expressed as The task of the central server is to find a model ω * Minimize the weighted average loss of all clients, that is,

[0105]

[0106] where α i It is the weight of the client model when it is aggregated.

[0107] The present invention is applicable to scenarios where the client data is not independent and identically distributed and contains malicious users. Malicious users control the local model by implementing poisoning attacks, such as affecting the classification effect of the model by modifying the labels of the client data.

[0108] like Figure 2 As shown, the federated learning method for dealing with malicious attacks in a non-IID scenario provided by an embodiment of the present invention includes the following steps:

[0109] Step 1: For a specific federated learning task, collect a clean small dataset, called the root dataset;

[0110] Step 2: Initialize a global model on the server side;

[0111] Step 3: The server broadcasts the global model to each client. After receiving the global model, the client uses it to initialize its local model, trains the local model with the local dataset, obtains the updated local model, and uploads it to the server.

[0112] Step 4: The server uses the root data set to train the neural network model synchronously with each client as the server-side model, and adds the server-side model to the current round of aggregated model set;

[0113] Step 5. After receiving the models of all users, the server calculates the marginal contribution of each user to the aggregate model set as its contribution. If it reaches 20 rounds, the credibility of each user model is calculated, and the user score is calculated using the contribution of each user in this round and the credibility of the previous round. The user with the highest score is selected to join the aggregate model set, and the user score is updated to continue selecting users until the contribution of the selected user is less than 0 for p consecutive times. Then the user selection is stopped, and all user models in the aggregate model set are aggregated into a global model.

[0114] Step 6: Test the accuracy of the global model on the test data set;

[0115] Step 7: Repeat steps 3 to 6 until the global model converges.

[0116] Example 1: Medical image classification in a non-IID data scenario

[0117] In the task of medical image classification, the patient datasets owned by different hospitals or medical institutions (clients) often have significant differences, and the data are non-independent and identically distributed (non-IID). Some malicious users carry out poisoning attacks by tampering with labels, etc., in an attempt to affect the accuracy of the global model.

[0118] 1) Collect the root dataset:

[0119] Medical institutions jointly collect a small-scale and clean medical image dataset, called the root dataset.

[0120] The root dataset is used for synchronous model training on the server side.

[0121] 2) Initialize the global model:

[0122] The server initializes a global neural network model for image classification.

[0123] 3) Broadcasting the global model:

[0124] The server broadcasts the global model to each medical institution, which then uses it to initialize the local model.

[0125] 4) Local model training:

[0126] Each medical institution trains a local model using its local patient dataset and uploads the updated local model to the server.

[0127] 5) Synchronous training and marginal contribution calculation:

[0128] The server synchronously trains a global model using the root dataset and adds it to the aggregated model set.

[0129] After receiving the models of all institutions, the server calculates the marginal contribution of each institution's model relative to the aggregated model set as its contribution.

[0130] If 20 rounds are reached, the server calculates the credibility of each institution's model, and uses the contribution of this round and the credibility of the previous round to calculate the user score, selects the institution model with the highest score to add to the aggregated model set, updates the user score and continues to select users.

[0131] 6) Model aggregation and selection:

[0132] If the contribution of the selected model in a round of institutional model selection is less than 0 for p consecutive times, stop the model selection for that round.

[0133] All organization models in the aggregate model set are aggregated into a new global model.

[0134] 7) Model testing and convergence:

[0135] Test the accuracy of the global model on the test dataset.

[0136] Repeat steps 3 to 6 until the global model converges.

[0137] Through this implementation plan, it is possible to effectively respond to malicious attacks in non-independent and identically distributed data scenarios, improve the classification accuracy of the global model, and ensure the reliability and security of medical image classification tasks.

[0138] Example 2: Intelligent traffic prediction in non-IID data scenario

[0139] In the intelligent traffic prediction task, traffic data (clients) in different regions have different traffic patterns and data distributions, and the data is usually not independent and identically distributed. Some malicious users interfere with the prediction effect of the model by tampering with data labels.

[0140] 1) Collect the root dataset:

[0141] Regional traffic management departments jointly collect a small-scale and clean traffic dataset, called the root dataset.

[0142] The root dataset is used for synchronous training on the server side.

[0143] 2) Initialize the global model:

[0144] The server initializes a global model for traffic flow prediction.

[0145] 3) Broadcasting the global model:

[0146] The server broadcasts the global model to each traffic management department, which then uses it to initialize the local model.

[0147] 4) Local model training:

[0148] Each traffic management department trains a local model using its local traffic dataset and uploads the updated local model to the server.

[0149] 5) Synchronous training and marginal contribution calculation:

[0150] The server synchronously trains a global model using the root dataset and adds it to the aggregated model set.

[0151] After receiving the models of all departments, the server calculates the marginal contribution of each department model relative to the aggregate model set as its contribution.

[0152] If 20 rounds are reached, the server calculates the credibility of each department model, and uses the contribution of this round and the credibility of the previous round to calculate the user score, selects the department model with the highest score to add to the aggregate model set, updates the user score and continues to select users.

[0153] 6) Model aggregation and selection:

[0154] If the contribution of the selected model in a round of institutional model selection is less than 0 for p consecutive times, stop the model selection for that round.

[0155] All department models in the aggregate model set are aggregated into a new global model.

[0156] 7) Model testing and convergence:

[0157] Test the accuracy of the global model on the test dataset.

[0158] Repeat steps 3 to 6 until the global model converges.

[0159] Through this implementation plan, it is possible to effectively respond to malicious attacks in non-independent and identically distributed data scenarios, improve the prediction accuracy of the global model, and ensure the stability and security of the intelligent transportation system.

[0160] The above two embodiments demonstrate the specific application of the federated learning method of the present invention to deal with malicious attacks in non-independent and identically distributed data scenarios. By collecting clean root data sets, initializing global models, performing local model training and synchronous training, calculating contribution and credibility, model aggregation and selection, and testing the accuracy of the global model on the test data set, the security and robustness of the federated learning system can be effectively improved, and it is suitable for different application scenarios. The pseudo code representation of the algorithm is shown in Table 1.

[0161] Table 1

[0162] Algorithm 1 User selection algorithm

[0163]

[0164] like Figure 3 As shown, the federated learning method and system for dealing with malicious attacks in a non-IID scenario provided by an embodiment of the present invention include:

[0165] The contribution model module is used to measure the user's contribution to the global model when calculating the user score. The contribution of each user is quantified by calculating the marginal contribution of each user to the selected user subset for aggregation when joining;

[0166] The credibility model module measures the credibility of users when calculating user scores. The higher the credibility, the higher the probability that the user is a benign user; the lower the credibility, the lower the probability that the user is a benign user;

[0167] The user rating calculation module is used to comprehensively measure the user's contribution and credibility. Based on the principle of evolutionary clustering, the contribution of this round and the credibility of the previous round are considered at the same time, with the contribution of this round as the main factor. Different weights are assigned to contribution and credibility to calculate the user rating, and the user rating is recalculated after each user set of the aggregation model is updated;

[0168] The user selection module is used to select users to participate in the aggregation of the global model. In each round of communication, after the server receives the model updates of all users, it calculates the scores of each user, selects the user with the highest score to join the set of users of the aggregated model, and then updates the user score, and continues to select the next user to participate in the aggregation, until the contribution of the user who has been selected to join the aggregation stage for p consecutive times is less than or equal to 0, and the present invention terminates the user selection process of this round;

[0169] The local training module is used for user devices to use local data sets to train federated learning tasks and complete message interaction and model sharing between users and servers;

[0170] The model aggregation module calculates the aggregation weights of the users in the model aggregation set and performs weighted aggregation on the user models to obtain the updated global model and broadcast it to all users.

[0171] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0172] Example 1: Smart Medical System

[0173] This invention is applied to medical equipment in various hospitals, and federated learning is used to assist medical image diagnosis and disease prediction. On the one hand, it can protect the privacy data of patients in various hospitals and reduce the risk of medical data leakage; on the other hand, since the patients targeted by each hospital are different, data is generally non-independent and identically distributed. This method can deal with the problem of global model training errors caused by malicious users tampering with local training data in non-independent and identically distributed scenarios, thereby reducing the probability of misdiagnosis.

[0174] Example 2: Smart Consumer Finance System

[0175] The invention can be used to collect user information in different departments and companies, including consumers' consumption level, purchasing power, product characteristics and purchasing preferences, train the God General network model to better understand the consumption needs of users at different consumption levels, and customize better sales strategies. This method can ensure that the original data of user information is stored locally, avoid the transmission of original data, and break the barriers to cooperation between companies; at the same time, the invention can also deal with the poisoning attack problem of the system in non-independent and identically distributed scenarios, and ensure that the learned model is correct.

[0176] Example 3: Smart Education System

[0177] Federated learning can be used to integrate educational resources from various knowledge bases to achieve customized education. It can collaboratively build a universal learning plan model based on the data in students' personal mobile devices, which can formulate standardized learning plans for students with similar backgrounds. The present invention can protect students' privacy and security, while improving the accuracy of the model learned in complex scenarios where non-independent and identically distributed data and malicious attacks coexist.

[0178] Example 4: Intelligent Navigation System

[0179] The technical solution of the present invention can be applied in the scenario of vehicle navigation, learning different vehicle data under different satellite systems and different navigation systems, training corresponding models, and then more accurately recommending the optimal navigation route. At the same time, it also protects the privacy of each user's vehicle and improves the robustness of the system.

[0180] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. A person of ordinary skill in the art will appreciate that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVDROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0181] The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with data, charts, etc. of the test process.

[0182] 1. Simulation conditions:

[0183] The simulation experiment of the present invention is carried out on a Windows platform, and the main configurations are: the CPU is Intel(R) Core(TM) i58400 CPU@2.80GHz; the memory is 16G; the operating system is Windows 10; and the simulation software environment is Python 3.6.

[0184] 2. Simulation content and result analysis

[0185] use Figure 1 The simulation scenario includes multiple user devices and a central server. According to different simulation parameters, the federated learning method for dealing with malicious attacks in the non-independent and identically distributed scenario of the present invention is used for simulation. The present invention uses the CNN network that is often used for image classification tasks to perform simulation on the standard image classification data sets Fashion-MNIST and CIFAR-10.

[0186] The present invention designs a federated learning scenario with non-IID data and malicious users. In this setting, there are 100 clients and one server. The server randomly selects 100 images from the training data set as the root data set. The present invention uses a known general method to construct non-IID data, which is based on a parameter called the non-IID degree to adjust the non-IID level of the client data set, which ranges from [0.1, 1.0]. The larger the value, the higher the non-IID degree in the data set. Specifically, when the non-IID degree is 0.1, the client's training data is independent and identically distributed, and when the non-IID degree is 1.0, each client data set contains only one image. In the simulation experiment, the present invention sets the non-IID degree to 0.5 and assumes that the malicious user performs a poisoning attack by flipping the label of the local data. The present invention evaluates the model accuracy of each algorithm when the malicious user accounts for 20%, 50% and 70%. In terms of model parameter settings, the present invention sets the learning rate of each algorithm to 0.01, the batch sizes of Fashion-MNIST and CIFAR-10 to 32 and 64 respectively, the local epoch to 5, the communication round to 100, and the SGD optimizer. By default, the contribution weight is set to 0.7. The present invention calculates the credibility from the 20th iteration of FL training to avoid the influence of initial iteration instability. In this method, p is a key threshold.

[0187] Figure 4 The relationship between the performance of the algorithm of the present invention on CIFAR-10 and the number of communication rounds is described when p selects different values. It can be observed that as p increases, the accuracy of the method on the test data set first increases and then decreases, and the best performance is achieved when p = 3. This is because when p is small, the threshold condition becomes too strict, resulting in too few users selected in each round, which reduces the diversity of non-independent and identically distributed users participating in the aggregation. However, when p is too large, the threshold condition will also become too loose, resulting in some malicious users participating in the aggregation, thereby reducing the performance of the global model. Therefore, p = 3 is set in the subsequent simulation experiments of this paper.

[0188] To show the advantages of our algorithm, we compare it with the following baseline methods, which also solve the problems of non-IID data and malicious users in federated learning:

[0189] AFedSV+: An algorithm with adaptive weights and client importance sampling strategy, which can effectively solve poisoning attacks in non-IID scenarios;

[0190] FedAvg: A popular federated learning method in non-adversarial environments, which can alleviate the problems caused by non-IID data to a certain extent;

[0191] FedProx: This algorithm handles the problem of non-IID data by adding a proximal term to the loss function;

[0192] FLTrust: This algorithm uses root datasets and cosine similarity to deal with non-IID problems and poisoning attacks in simple dataset scenarios;

[0193] RFA: This method uses cosine similarity to detect and exclude malicious users, thereby mitigating the impact of malicious user attacks.

[0194] Figure 5 shows the accuracy of the global model trained by the algorithm of the present invention and the baseline algorithm on CIFAR-10 under three different malicious user ratios. For each algorithm, the present invention conducted 5 independent experiments and averaged and smoothed the results.

[0195] The research results show that in non-IID scenarios with different proportions of malicious users, the algorithm in the present invention is significantly better than the baseline algorithm. Taking the non-IID scenario experiment with malicious users accounting for 20% as an example, Figure 5aAs shown. After 100 rounds of federated learning communication, the average accuracy of the algorithm in the present invention on the test set reached 52.21%, which is 6.9%, 12.4%, 28.7%, 22.8% and 11.3% higher than AFedSV+, FedAvg, FedProx, FLTrust and RFA, respectively. Among them, the poor performance of FedAvg and FedProx is due to the inability to solve the problem of malicious user attacks. In particular, in FedProx, the aggregation weights of malicious users and benign users are comparable, so it is difficult to mitigate the impact of malicious users, and sometimes even performs worse than FedAvg. The low test accuracy of RFA is because it only focuses on malicious users without considering non-independent and identically distributed scenarios, and it only performs relatively well for a small number of malicious users. This is because it relies on the information of all user models, and once the number of malicious users dominates, the algorithm will fail. Similarly, FLTrust also relies on the global model to select and adjust the model gradient. When the proportion of malicious users is high, the global model becomes unreliable. AFedSV+ fails because user selection is significantly random, and although it can be adjusted by selection probability, it cannot avoid occasional aggregation of malicious users. The algorithm in this paper sets stricter user selection criteria, taking into account the contribution of users in the current round and the credibility of the previous round. It is less affected by non-IID data and user interactions, and aims to exclude malicious users in the aggregation stage, thereby achieving higher model accuracy.

[0196] When the proportion of malicious users reaches 70% or even higher, the model accuracy of the FedAvg, FedProx, FLTrust, and RFA algorithms shows a downward trend. This is because when the proportion of malicious users is too high, the algorithm cannot work properly and the model performance is even worse than the initial model.

[0197] Figure 6 shows the accuracy of the proposed algorithm and the baseline algorithm on the Fashion-MNIST dataset as a function of the number of communication rounds. The experimental setting is similar to the CIFAR-10 dataset, and the results are consistent with the CIFAR-10 dataset, where the proposed algorithm significantly outperforms the baseline algorithm.

[0198] Take the non-IID experimental scenario where malicious users account for 50% as an example. Figure 6bAs shown. After 100 rounds of federated learning communication, the average accuracy of the algorithm in the present invention on the test set is 88.16%, which is 8.1%, 45.8%, 41.4%, 43.9% and 48.9% higher than AFedSV+, FedAvg, FedProx, FLTrust and RFA, respectively. Among them, except for the algorithm in the present invention and the FLTrust algorithm, the accuracy of the remaining algorithms fluctuates greatly. The present invention analyzes that this is because as a data set for handwritten character recognition, the features of Fashion-MNIST are relatively simple, and a simple CNN network can achieve higher performance. Therefore, flipping labels can easily have a significant impact on the gradient, and once a malicious user is selected for aggregation, it will produce large fluctuations. In contrast, for the CIFAR-10 data set, due to its low accuracy on a simple CNN network, the impact caused by occasional malicious users participating in aggregation is relatively small.

[0199] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A federated learning method for dealing with malicious attacks in a non-IID scenario, characterized in that: The following steps are involved: Step 1: For a specific federated learning task, collect a clean small dataset as the root dataset; Step 2: Initialize a global model on the server side; Step 3: The server broadcasts the global model to each client. The client trains the local model using the local dataset and uploads the updated model to the server. Step 4: The server and each client synchronously train the neural network model as the server-side model; Step 5: The server uses the contribution and credibility of each client to calculate the user score, and selects users based on the score to participate in the aggregation of the global model together with the server-side model to obtain the aggregated global model; Step 6: Test the accuracy of the global model on the test data set; Step 7: Repeat steps 3 to 6 until the global model converges.

2. The federated learning method for dealing with malicious attacks in a non-IID scenario as claimed in claim 1, characterized in that: The contribution model in this method is implemented in the following way: Define marginal contribution as the improvement in accuracy or loss value on the test set after model aggregation brought about by the user joining the current user set; The server maintains a root dataset and uses it to train synchronously with each client. The resulting model is used as the server model. The server model is used as the initial user subset for each round of aggregation, and the marginal contribution of each user when joining is calculated; The marginal contribution is normalized using the range normalization method of positive indicators, and the standardized marginal contribution is used as the contribution of the user in this selection.

3. The federated learning method for dealing with malicious attacks in a non-IID scenario as claimed in claim 1, characterized in that: The credibility model in step 5 is calculated by a rank sum ratio (RSR) method, comprising the following steps: (1) Construct an evaluation matrix based on the user's contribution in each round; (2) Replace the evaluation index values ​​in the evaluation matrix with the ranks to obtain a rank matrix; (3) The rank sum ratio (RSR) of each user is calculated based on the rank matrix and normalized to obtain the user credibility.

4. The federated learning method for dealing with malicious attacks in a non-IID scenario as claimed in claim 1, characterized in that: The user rating calculation includes: The user score is calculated by assigning different weights to contribution and credibility, where the contribution weight is set to α and the credibility weight is set to 1-α, and the value range of α is between [0.5, 1]. The user score is determined based on the contribution of the current round and the credibility of the previous round.

5. The federated learning method for dealing with malicious attacks in a non-IID scenario as claimed in claim 1, characterized in that: The user selection method comprises: (1) Calculate the marginal contribution of each user model relative to the selected user subset and normalize it; (2) Calculate user ratings based on the user contribution in the current round and the user credibility in the previous round; (3) Select the user with the highest score to join the aggregation model set until the contribution of the user who has been selected to join for p consecutive times is less than or equal to 0.

6. The federated learning method for dealing with malicious attacks in a non-IID scenario as claimed in claim 1, characterized in that: The aggregation algorithm includes: (1) Aggregate user models based on the FedAvg algorithm; (2) determining the weight of the user during aggregation according to the ratio of the size of each selected user's data set to the sum of the sizes of all selected user data sets; (3) Aggregate user models by weight to obtain a global model.

7. A federated learning system for dealing with malicious attacks in a non-independent and identically distributed scenario implementing the federated learning method for dealing with malicious attacks in a non-independent and identically distributed scenario as described in any one of claims 1 to 6, characterized in that: The federated learning system for dealing with malicious attacks in the non-IID scenario includes: The contribution model module is used to measure the user's contribution to the global model when calculating the user score. The contribution of each user is quantified by calculating the marginal contribution of each user to the selected user subset for aggregation when joining; The credibility model module is used to measure the credibility of users when calculating user scores. The higher the credibility, the higher the probability that the user is a benign user; the lower the credibility, the lower the probability that the user is a benign user; The user rating calculation module is used to comprehensively measure the user's contribution and credibility. Based on the principle of evolutionary clustering, the contribution of this round and the credibility of the previous round are considered at the same time, with the contribution of this round as the main factor. Different weights are assigned to contribution and credibility to calculate the user rating, and the user rating is recalculated after each user set of the aggregation model is updated; The user selection module is used to select users to participate in the aggregation of the global model. In each round of communication, after the server receives the model updates of all users, it calculates the scores of each user, selects the user with the highest score to join the set of users of the aggregated model, and then updates the user score, and continues to select the next user to participate in the aggregation, until the contribution of the user who has been selected to join the aggregation stage for p consecutive times is less than or equal to 0, and the present invention terminates the user selection process of this round; The local training module is used for user devices to use local data sets to train federated learning tasks and complete message interaction and model sharing between users and servers; The model aggregation module calculates the aggregation weights of the users in the model aggregation set and performs weighted aggregation on the user models to obtain the updated global model and broadcast it to all users.

8. The federated learning system for dealing with malicious attacks in a non-IID scenario as claimed in claim 7, characterized in that: The federated learning system for dealing with malicious attacks in the non-IID scenario includes a server and N clients, including: Calculate the marginal contribution of each user relative to the aggregate model set to determine the contribution of each user; The credibility of the users in this round is determined by the rank sum ratio of the marginal contribution of each user's choice in this round; The idea of ​​evolutionary clustering is used to assign different weights to contribution and credibility to calculate user ratings; The idea of ​​greedy algorithm is adopted to select users one by one to join the collection of aggregation model; Aggregate the users in the aggregate model set to obtain the global model of this round.

9. A federated learning method for dealing with malicious attacks in a non-independent and identically distributed data scenario, characterized in that: include: Collect root datasets, where each client jointly collects a small and clean dataset for model validation and synchronous training on the server side; Initialize the global model. The server initializes a global model and broadcasts it to each client. After receiving the global model, the client uses it to initialize the local model. Local model training: Each client trains a local model using its local dataset and uploads the updated local model to the server; Synchronous training and marginal contribution calculation: The server uses the root dataset to synchronously train the global model and adds it to the aggregate model set. After receiving the models of all clients, the server calculates the marginal contribution of each client model relative to the aggregate model set as its contribution. Credibility calculation and user scoring: If the preset number of rounds is reached, the server calculates the credibility of each client model, and uses the contribution of this round and the credibility of the previous round to calculate the user score, and selects the client model with the highest score to join the aggregated model set. If the contribution of the selected client is less than 0 for a preset number of consecutive times, the selection of the model for this round is stopped; Model aggregation: all client models in the aggregated model collection are aggregated into a new global model; Model testing and convergence,The accuracy of the global model is tested on the test dataset, and the above steps are repeated until the global model converges.

10. An intelligent traffic prediction system for dealing with malicious attacks in a non-IID data scenario, characterized in that: include: The data collection module is used by traffic management departments in various regions to jointly collect a small-scale and clean traffic dataset, called the root dataset, which is used for model verification and synchronous training on the server side; The global model initialization module is used to initialize a global model for traffic flow prediction on the server side and broadcast it to each traffic management department. After receiving the global model, each department uses it to initialize the local model. A local model training module is used by each traffic management department to train a local model using its local traffic dataset and upload the updated local model to the server; The synchronous training and marginal contribution calculation module is used by the server to synchronously train the global model using the root data set and add it to the aggregate model set. After receiving the models of all departments, the marginal contribution of each department model relative to the aggregate model set is calculated as its contribution. The credibility calculation and user scoring module is used to calculate the credibility of each department model when the preset number of rounds is reached, and use the contribution of this round and the credibility of the previous round to calculate the user score, select the department model with the highest score to add to the aggregate model set, and stop selecting the department model for this round if the contribution of the selected department is less than 0 for a preset number of consecutive times; Model aggregation module, which is used to aggregate all department models in the model set into a new global model; The model testing and convergence module is used to test the accuracy of the global model on the test data set and repeat the above steps until the global model converges.

Images