Federal learning client selection method and system for intelligent manufacturing and related equipment

Abstract

The invention discloses an intelligent manufacturing-oriented federal learning client selection method and system and related equipment, and the method and system enable trained models to have better similarity through constructing a client combination with large data relevance and small difference between clients, and improve the accuracy of the data relevance between the clients. Therefore, the federated learning aggregation model obtained through training can effectively reduce the influence of data distribution deviation, and then the problem that when a client carries out federated training, due to the difference between local models caused by manufacturing industry heterogeneous data, deviation occurs in the aggregation model is effectively solved.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a federated learning client selection technology that can be used in the context of intelligent manufacturing. Background Art

[0002] As manufacturing becomes an increasingly important part of the economy, modern manufacturing is creating new products or services with features such as customization, adaptability and reliability at an unprecedented scale. These new products or services have become an integral part of our daily lives. The functionality and scalability of products or services make manufacturing systems increasingly complex, which is driving the manufacturing industry to become a new generation of cyber-physical systems and move towards network-enabled smart manufacturing. Industrial infrastructure, smart manufacturing technologies, and large-scale digitalization of assets and processes across industries are driving the manufacturing industry into a new era. The promotion and construction of smart manufacturing systems requires the digitization of modern manufacturing, which is achieved through data-driven analytical tools such as artificial intelligence, machine learning, digital twins, cloud computing, and big data analytics. These rapidly developing technologies, coupled with the exponential growth of manufacturing data, have enabled the development of data-hungry artificial intelligence and machine learning at an unprecedented scale.

[0003] The distributed nature of modern industrial IoT systems and traditional centralized machine learning techniques pose a direct threat to data sharing. Taking the object recognition and classification task in smart manufacturing as an example, the raw data collected by different cameras needs to be uploaded to a centralized server (such as a cloud server), which may lead to the potential leakage of confidential information (such as products being manufactured or personal information collected) during the transmission process. The concept of federated learning has recently been introduced to facilitate knowledge sharing among multiple smart terminal devices while protecting data privacy by avoiding sharing raw data between terminal devices and central servers. In a smart manufacturing system based on federated learning, multiple cameras can collect product images and train a local model separately. By not transmitting raw data, only the local model parameters are encrypted and uploaded to a central server to form a global model. By combining multiple local models, the global model is expected to provide more generalized and accurate predictions.

[0004] Different from the distributed collaborative modeling scenarios based on the general data center environment, federated learning is used in Figure 1The smart manufacturing scenario shown in the figure usually faces a complex computing environment with heterogeneity, and its heterogeneity factors will have an adverse effect on the model effect in the federated scenario. Existing methods for dealing with data heterogeneity factors often have many constraints and limitations, or lack a certain degree of versatility, and cannot take into account both computing overhead and accuracy assurance for complex smart manufacturing scenarios. For example, the FedAvg algorithm uses a random selection strategy to filter clients to reduce computing overhead, but this random selection of clients will aggravate the adverse effects of data heterogeneity. Summary of the invention

[0005] In view of the problems existing in the existing federated learning client selection scheme in the intelligent manufacturing system based on federated learning, the purpose of the present invention is to provide a federated learning client selection scheme for intelligent manufacturing. This scheme constructs a client combination with large data correlation and small difference between clients, and trains a model with better similarity, so that the obtained federated learning aggregation model can reduce the impact of data distribution deviation.

[0006] In order to achieve the above object, the present invention provides a method for selecting a federated learning client for intelligent manufacturing, comprising:

[0007] (1) Divide heterogeneous data. Divide industrial data into different levels of heterogeneous data for clients according to heterogeneity requirements;

[0008] (2) Client local pre-training: each client trains its own local model based on the received initial model parameters and using its own data;

[0009] (3) Collecting client model parameter change information, and collecting the model parameter difference between the first and last iteration training of the client local training;

[0010] (4) Construct a graph structure to calculate the similarity of the client model. Use the collected information to construct an undirected graph with the client as the node, calculate the similarity of the nodes, and select the client combination based on this;

[0011] (5) Train and save the results, perform federated learning training on the output new client group, and finally evaluate the aggregated model and save the evaluation results.

[0012] In some embodiments of the present invention, the federated learning client selection method divides heterogeneous data through Dirichlet distribution.

[0013] In some embodiments of the present invention, when constructing an undirected graph, the method for selecting a client for federated learning uses the total number of clients N as the number of nodes in the undirected graph, and each node represents each client C in the federated learning.n}, which constructs a complete graph G, in which there are edges corresponding to different nodes.

[0014] In some embodiments of the present invention, the federated learning client selection method is based on the node value represented by each client, uses multidimensional cosine similarity to calculate the weights of the edges between client nodes respectively, and then maps the model similarity between the two nodes based on the weights.

[0015] In order to achieve the above object, the present invention also provides a federated learning client selection system for intelligent manufacturing, the selection system includes a heterogeneous data division module, a client local pre-training module, a model parameter change information collection module, a client screening module and a client federated learning training module.

[0016] The module for dividing heterogeneous data is configured to divide the industrial data into heterogeneous data of different degrees for the client according to heterogeneity requirements;

[0017] The client local pre-training module is configured to interact with the data of the heterogeneous data division module, and can train each client to have its own local model based on the received initial model parameters and using its own data;

[0018] The model parameter change information collection module is configured to interact with the client local pre-training module data and collect the model parameter difference between the first and last iterative training of the client local training;

[0019] The client screening module is configured to interact with the model parameter change information collection module data, and calculate the client model similarity by constructing a graph structure, thereby screening the client combination accordingly;

[0020] The client federated learning training module is configured to interact with the client screening module and the heterogeneous data partitioning module, and can perform federated learning training on the screened client groups, and finally perform model evaluation on the aggregated model.

[0021] In some embodiments of the present invention, the module for dividing heterogeneous data divides the heterogeneous data by Dirichlet distribution.

[0022] In some embodiments of the present invention, the client screening module includes a client undirected graph construction submodule, a client model similarity calculation submodule, a client selection submodule,

[0023] The client undirected graph construction submodule is configured to construct an undirected graph with the client as a node;

[0024] The client model similarity calculation submodule is configured to interact with the client undirected graph construction submodule data, and determine the node value represented by each client by the constructed undirected graph, and use the multi-dimensional cosine similarity to calculate the weight of the edge between the client nodes respectively, and then map the model similarity between the two nodes based on the weight;

[0025] The client selection submodule is configured to interact with the client model similarity calculation submodule in data, screen the original clients based on the model similarity calculated by the client model similarity calculation submodule, and select a client set with closer similarity.

[0026] In order to achieve the above-mentioned object, the present invention also provides a processor, which is used to run a program, wherein the steps of the above-mentioned federated learning client selection method for intelligent manufacturing are executed when the program is running.

[0027] In order to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program code is loaded and executed by the processor to implement the steps of the above-mentioned federated learning client selection method for intelligent manufacturing.

[0028] In order to achieve the above-mentioned objectives, the present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned federated learning client selection method for intelligent manufacturing.

[0029] The federated learning client selection scheme for intelligent manufacturing provided by the present invention constructs a client combination with large data correlation and small differences between clients, so that the trained models have better similarity, so that the trained federated learning aggregation model can effectively reduce the impact of data distribution deviation, and then effectively solve the problem of deviation in the aggregation model caused by differences between local models caused by heterogeneous data in the manufacturing industry during client federated training.

[0030] In the federated learning client selection scheme for intelligent manufacturing provided by the present invention, a graph model established with the client as a node can effectively associate the clients, thereby facilitating relationship operations between the clients.

[0031] In the federated learning client selection scheme for intelligent manufacturing provided by the present invention, the similarity between federated learning clients is calculated by cosine similarity, which can further reduce the computational complexity.

[0032] The federated learning client selection scheme for intelligent manufacturing provided by the present invention establishes a corresponding client selection algorithm based on cosine similarity and graph theory. Under the setting that only parameters but not data are transmitted in federated learning, data characteristics are reflected by transmitting changes in client model parameters. The server uses graph theory and similarity to perform more accurate similarity calculations on parameter information, allowing each client to search for other clients that are more similar to it in data distribution and interact more with it, thereby helping to screen out more large and more similar training data, avoiding model skewness, and thereby improving the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0034] Figure 1 This is a structural diagram of the federated learning framework involved in the present invention;

[0035] Figure 2 This is a flowchart of the implementation principle of the federated learning client selection solution for intelligent manufacturing in the present invention;

[0036] Figure 3 A schematic diagram of the composition of the federated learning client selection system for intelligent manufacturing in the present invention;

[0037] Figure 4 This is a flow chart of using Dirichlet distribution to partition client data in an example of the present invention;

[0038] Figure 5 This is a flowchart for implementing the federated learning client selection of the algorithm FedMPCCS in the example of the present invention;

[0039] Figure 6 The following are distribution results of data in four different situations after data is divided in an example of the present invention.

[0040] Figure 7 This is a comparison chart of the loss reduction changes of the algorithm FedMPCCS in the example of the present invention and the classic federated learning algorithm FedAvg on the power system attack data set.

[0041] Figure 8 This is a comparison chart of the loss reduction changes of the algorithm FedMPCCS in the example of the present invention and the classic federated learning algorithm FedAvg on the hard disk fault detection dataset.

[0042] Fig. 9 This is a comparison chart of the accuracy increase of the algorithms FedMPCCS and FedAvg in the example of the present invention on the power system attack data set.

[0043] Fig.10The figure is a comparison chart of the accuracy increase of the algorithms FedMPCCS and FedAvg in the example of the present invention on the hard disk fault detection data set. DETAILED DESCRIPTION

[0044] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further described below with reference to specific diagrams.

[0045] The realization of artificial intelligence training in the field of intelligent manufacturing requires a large amount of multi-party data. Due to security and privacy constraints, the parties cannot exchange data with each other. In this regard, in modern industrial Internet of Things systems, the introduction of federated learning can protect data privacy by avoiding sharing original data between terminal devices and central servers, and can promote knowledge sharing among multiple intelligent terminal devices.

[0046] However, due to the data heterogeneity in the field of intelligent manufacturing, the performance of federated learning will decline. Specifically, when the client performs federated training, the differences between local models caused by heterogeneous data in the manufacturing industry will lead to deviations in the aggregated model.

[0047] To address this problem, the present invention provides a federated learning client selection solution for intelligent manufacturing. By constructing a client combination with high data correlation and small differences between clients, the trained models have better similarity, so that the trained federated learning aggregation model can effectively reduce the impact of data distribution deviation, and then effectively solve the problem of deviation in the aggregation model caused by differences between local models caused by heterogeneous data in the manufacturing industry during client federated training.

[0048] The following specifically describes the implementation process of the federated learning client selection solution for intelligent manufacturing provided by the present invention.

[0049] The intelligent manufacturing-oriented federated learning client selection scheme provided by the present invention is implemented in coordination with the client and the corresponding server in a modern industrial Internet of Things system.

[0050] See also Figure 2 , which is a flowchart of the implementation principle of the federated learning client selection solution for intelligent manufacturing in the present invention. Based on the diagram, this solution is specifically implemented by the following steps working together:

[0051] (1) Customize the partitioning of heterogeneous data;

[0052] This step divides the industrial data into different degrees of heterogeneity for the client according to its heterogeneity.

[0053] (2) Client-side local pre-training.

[0054] In this step, each client receives the initial model parameters sent by the system and uses its own data to train its own local model.

[0055] (3) The server collects the client's model parameter change information.

[0056] In this step, for the client local pre-training process in step (2), the parameter difference between the first and last iterative training of the client local training is transmitted to the server respectively.

[0057] (4) Construct a graph structure to calculate the similarity of client models.

[0058] In this step, the server constructs an undirected graph with the client as the node using the collected information, calculates the similarity of the nodes, outputs the node degree, and selects the client combination based on it.

[0059] (5) Train and save the results.

[0060] This step performs federated learning training on the output new client group, and finally performs model evaluation on the aggregated model and saves the evaluation results.

[0061] As a further illustration, the present federated learning client selection scheme preferably divides the heterogeneous data by Dirichlet distribution in step (1).

[0062] As an example, in a specific implementation, the industrial data may be customized using the distribution parameter α according to heterogeneity requirements to divide the data into different degrees of heterogeneity for the client.

[0063] As a further explanation, when the client constructs an undirected graph for the node in step (4), the total number of clients N is used as the number of nodes in the undirected graph, and each node represents each client C in the federated learning. n}, which constructs a complete graph G, in which there are edges corresponding to different nodes.

[0064] Furthermore, based on the constructed undirected graph, this federated learning client selection scheme uses multi-dimensional cosine similarity to calculate the weights of the edges between client nodes based on the node values ​​represented by each client, and then maps the model similarity between the two nodes based on the weights.

[0065] This federated learning client selection scheme for intelligent manufacturing can effectively overcome the problem of degraded federated learning performance caused by data heterogeneity in the field of intelligent manufacturing.

[0066] The federated learning client selection scheme for intelligent manufacturing provided by the present invention can be used to form a corresponding software program and a corresponding federated learning client selection system in specific applications. When the software program is running, it will execute the above-mentioned method steps for selecting a federated learning client for intelligent manufacturing and store them in a corresponding storage medium for the processor to retrieve and execute.

[0067] See also Figure 3 The thus formed federated learning client selection system 100 for intelligent manufacturing mainly includes five functional modules: a heterogeneous data division module 110, a client local pre-training module 120, a model parameter change information collection module 130, a client screening module 140 and a client federated learning training module 150.

[0068] The heterogeneous data division module 110 is configured to divide the industrial data into heterogeneous data of different degrees for the client according to heterogeneity requirements.

[0069] The client local pre-training module 120 in the system is configured to interact with the data partitioning module 110 and can train each client to form its own local model based on the received initial model parameters and using its own data.

[0070] The model parameter change information collection module 130 in the present system is configured to interact with the client local pre-training module 120 data, and can collect the model parameter difference between the first and last iteration training of the client local training.

[0071] The client screening module 140 in the present system is configured to interact with the model parameter change information collection module 130 for data, and calculate the client model similarity by constructing a graph structure, thereby screening the client combination accordingly.

[0072] The client federated learning training module 150 in the system is configured to interact with the client screening module 140 and the heterogeneous data partitioning module 110, so as to perform federated learning training on the screened client groups and finally perform model evaluation on the aggregated model.

[0073] As a further illustration, the module 110 for partitioning heterogeneous data in the present system is preferably configured to partition heterogeneous data by Dirichlet distribution.

[0074] On this basis, when industrial data is divided according to heterogeneity requirements based on the present heterogeneity data division module 110, the industrial data can be customized using the distribution parameter α according to the heterogeneity requirements to divide heterogeneous data of different degrees for the client.

[0075] As a further explanation, the client screening module 140 in the present system is specifically composed of a client undirected graph construction submodule 141 , a client model similarity calculation submodule 142 and a client selection submodule 143 , which cooperate with each other.

[0076] The client undirected graph construction submodule 141 is specifically configured to construct an undirected graph with the client as a node.

[0077] Specifically, the client undirected graph construction submodule 141 uses the total number of clients N as the number of nodes in the undirected graph, and each node represents each client C in the federated learning. n}, thus constructing a complete graph G, in which there are edges corresponding to different nodes.

[0078] The client model similarity calculation submodule 142 here is configured to interact with the client undirected graph construction submodule 141 for data, and to determine the node value represented by each client using the constructed undirected graph, and to use multi-dimensional cosine similarity to calculate the weights of the edges between the client nodes respectively, and then to map the model similarity between the two nodes based on the weights.

[0079] The client selection submodule 143 here is configured to interact with the client model similarity calculation submodule 142 for data, screen the original clients based on the model similarity calculated by the client model similarity calculation submodule 142, and select a client set with closer similarity.

[0080] Regarding the federated learning client selection scheme for intelligent manufacturing provided by the present invention, the implementation process and corresponding technical features of the federated learning client selection scheme are further explained below through specific application examples.

[0081] When this federated learning client selection scheme is applied in practice, a corresponding federated learning client selection system is constructed based on the aforementioned scheme, and the constructed federated learning client selection system is adapted to the modern industrial Internet of Things system, such as establishing corresponding data links between the client and the server in the modern industrial Internet of Things system.

[0082] On this basis, for industrial data, heterogeneous data is divided through Dirichlet distribution. This division can reflect different heterogeneous data, so that experimental data can simulate different situations and be closer to real manufacturing data scenarios. Specifically, the system can customize the industrial data according to the heterogeneity needs using parameter α to divide heterogeneous data of different degrees for the client.

[0083] See also Figure 4, which shows the flow chart of using Dirichlet distribution to partition client data in this example. First, the corresponding label distribution matrix is ​​constructed for the original data, then the corresponding sample index set is formed for the constructed label distribution matrix, then Dirichlet distribution partitioning is performed, and finally the output client data set is formed according to the partitioning result.

[0084] As an example, there are K category labels and N clients. The samples of each category label need to be divided into different clients according to different proportions, so that the distribution of sample labels on each client can be as different as possible.

[0085] Furthermore, let the matrix is the category label distribution matrix, whose row vector Represents the probability distribution vector of category k on different clients. Each dimension represents the proportion of samples of category k divided into different clients. The random vector is sampled from Dirichlet distribution.

[0086] Based on the preparation of the aforementioned solution, the federated learning client selection system formed in this example realizes client selection based on federated learning by constructing a client selection algorithm for intelligent manufacturing (i.e., FedMPCCS).

[0087] See also Figure 5 ,The process of selecting the federated learning client in this example is as follows:

[0088] Step 1: Collect server information.

[0089] In order to evaluate the similarity between clients, it is necessary to collect data information of each client. However, due to the constraint that federated learning only transmits parameters but not data, this example relies on collecting changes in model parameters to reflect data information, thereby achieving client similarity measurement.

[0090] Accordingly, in the process of selecting a client set in this example, the client information needs to be collected first. At this stage, the aggregation server must obtain the client's model transformation information.

[0091] The local model of the client is iterated by formula (1), and the model parameter ω of the client c is c The model parameter difference between the first and last iterations locally is recorded as It represents the model parameter ω of client c c The parameter changes that occur between e epochs. The model parameter changes of N clients can be obtained by local training performed in parallel by the clients. The aggregation server collects this information.

[0092]

[0093] Step 2: The client builds an undirected graph.

[0094] Since similarity calculation can only reflect the relationship between given vectors, but clients in the manufacturing industry have complex associations, if similarity calculation is directly performed on them based on the changes in client model parameters collected in Step 1, more complex relationships cannot be taken into account.

[0095] To this end, this example uses an undirected graph constructed with the client as the node, combined with similarity calculation to improve the accuracy and explainability of the calculation.

[0096] In this example, the undirected graph is constructed by taking the total number of clients N as the number of nodes in the undirected graph, and each node represents each client C in the federated learning process = {c1, c2, ..., c n}, which constructs a complete graph G, where there are edges between different nodes. The aggregation server collects the model parameter change information Processing is performed and parameter changes represent the node value of each client in the undirected graph.

[0097] Step 3: Calculate similarity and graph node degree.

[0098] In this step, the node values ​​of each client representative obtained in Step 2 are used to calculate the weight cosθ of the edges between client nodes using the multidimensional cosine similarity formula (2): ij (i≠j,i,j∈n), the weight can map the model similarity between two nodes, that is, the two clients, and can be stored and recorded using the adjacency matrix, as shown in formula (3).

[0099] Based on this, according to the weight cosθ between each node ij The undirected complete graph G generated in Step 2 is transformed into a weighted undirected complete graph Formula (4) is used to calculate the weights between a node and all its connected nodes to obtain the node degree d i , to indicate the connection strength or importance between this node and other nodes.

[0100]

[0101] d i =(cosθ i1 +...+cosβ in ) (4).

[0102] Step 4: Top-K outputs the client set.

[0103] The degree of each node in the statistical weighted undirected complete graph is calculated according to the degree weight. Figure 5 In Step 4, the Top-K selection mechanism is applied to filter the original clients according to the hyperparameter β∈(0,1) to set the number of output sets and select the client sets with closer similarity. Where K represents the set C K The number of clients is K = β·N, and the selection of the original client set C can be realized according to formula (5).

[0104]

[0105] For the above-mentioned specific application example scheme, the performance and effect are further verified through corresponding experimental data.

[0106] 1. Based on the Dirichlet distribution in the application example, this example uses two industrial data sets: (1) power system attack data set and (2) hard disk fault detection data set. The data results of the power system attack data set are shown in the figure. Figure 6 As shown, the parameter α can affect the deviation range of data generation. Figure 6 The client_id axis represents the ID of each participant in federated learning, the label axis represents the label type of each participant's sample set, and number represents the number of data samples of the corresponding category of each participant.

[0107] Figure 6 When the value of α in (a) is 1, the data distribution of each participant is severely skewed, the number of samples of different participants in the same category varies greatly, and the category distribution between different participants is also different.

[0108] Figure 6 When the value of α in (c) is 10, although the data distribution of each participant is skewed, the difference in change is small, and the participants still hold different numbers of category labels.

[0109] Figure 6 When the value of α in (d) is 50, the data distribution of each participant in this scenario is slightly skewed, and the differences between the participants are small, which is close to the distribution state of the IID data set.

[0110] 2. By comparing the random selection strategy of the FedAvg algorithm, the training effect and convergence of this application example solution are tested.

[0111] In order to fairly verify the difference between this application example method and FedAvg, the parameter β of the algorithm FedMPCCS in this application example scheme and the parameter C of FedAvg are set to correspond to ensure the same number of selected clients. The local model selects MLP, which consists of an input layer, two hidden layers, a dropout layer and an output layer. At the same time, 5 federated learning clients are set, and the data heterogeneity of each client is divided by Dirichlet distribution according to α=10 to construct a local data set with uneven label categories. Since different data sets have different communication rounds for convergence in federated learning, the global communication rounds are set to 200 and 50 respectively on the power system attack data set and hard disk fault detection data set.

[0112] See also Figure 7 This is the loss reduction curve of the algorithms FedMPCCS and FedAvg in this application example scheme with the same number of clients on the power system attack dataset:

[0113] Figure 7 In (a), the loss decrease rate of FedMPCCS from Round = 0 to Round = 50 is faster than that of FedAvg;

[0114] Figure 7 In (b), when the parameters are set to C = 0.6, β = 0.6, the loss reduction rate of FedMPCCS is significantly better than that of FedAvg after Round = 25, and the loss of FedAvg decreases slowly after Round = 100;

[0115] Figure 7 In (c), the loss of FedMPCCS decreases faster than that of FedAvg after Round = 50.

[0116] See also Figure 8 , which shows the loss reduction changes of the algorithm FedMPCCS and the classic federated learning algorithm FedAvg in this application example on the hard disk fault detection dataset.

[0117] In general, the loss reduction rate of FedMPCCS under the three parameter settings is better than that of FedAvg to varying degrees.

[0118] See also Fig. 9 , which shows the change in model accuracy of the algorithm FedMPCCS and the classic federated learning algorithm FedAvg in this application example scheme on the power system attack dataset with the same number of clients.

[0119] in, Fig. 9 In (a), the accuracy improvement rate of FedMPCCS between Round = 0 and Round = 100 is significantly higher than that of the hyperparameter FedAvg. Fig. 9In (b), the accuracy of FedMPCCS is significantly improved after Round = 75 compared with FedAvg, while the accuracy of FedAvg tends to fit.

[0120] See also Fig.10 , which shows the change in model accuracy of the algorithm FedMPCCS and the classic federated learning algorithm FedAvg in this application example solution on the hard disk fault detection dataset with the same number of clients.

[0121] in, Fig.10 In (a), FedMPCCS has a faster accuracy improvement than FedAvg before Round = 20, and the accuracy improvements of both algorithms tend to be flat after Round = 20. Fig.10 In (b), the accuracy of FedMPCCS is significantly higher than that of FedAvg before Round = 40. Fig.10 FedMPCCS in (c) outperforms FedAvg in most of the global communication rounds.

[0122] Furthermore, in order to compare the improvement of FedMPCCS with FedAvg on the two data sets, the present invention counts and compares the average improvement of global communication rounds in accuracy of FedMPCCS compared with FedAvg, and the results are shown in the table:

[0123]

[0124] It can be seen from the above example scheme that the scheme of the present invention constructs a client combination with large data correlation and small differences between clients, and the trained models have better similarity, so that the federated learning aggregation model obtained will reduce the impact of data distribution deviation. On this basis, the data is configured not to be transferred locally in federated learning, so that only model information and parameters can be transmitted, so that iterative parameter changes are obtained through local model training on the client, thereby reflecting the impact of data characteristics on model performance.

[0125] Furthermore, compared with the Euclidean distance which has the dimensionality curse problem in high-dimensional space, the solution of the present invention has lower computational complexity by calculating the similarity between federated learning clients through cosine similarity; at the same time, the relationship between all clients of federated learning is mapped out through the relationship between nodes and edges in graph theory, and the clients can be effectively associated through a graph model established with the clients as nodes, which facilitates the relationship calculation between clients.

[0126] Furthermore, the scheme of the present invention establishes a clear client selection algorithm based on cosine similarity and graph theory. Under the premise of only transmitting parameters but not data in federated learning, the data characteristics are reflected by transmitting the changes in client model parameters, and the server uses graph theory and similarity to perform more accurate similarity calculations on parameter information, so that each client can search for other clients that are more similar to it in data distribution and interact more, helping to screen out more and more similar training data, avoiding model skewness, and thus improving the training effect of the model.

[0127] Finally, it should be noted that

[0128] When the solution of the present invention is implemented specifically, a processor can also be provided, which is used to run a program, wherein the steps of the above-mentioned federated learning client selection solution for intelligent manufacturing are executed when the program is running.

[0129] When the solution of the present invention is implemented specifically, a terminal device can also be provided, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program code is loaded and executed by the processor to implement the steps of the above-mentioned federated learning client selection solution for intelligent manufacturing.

[0130] When the solution of the present invention is implemented specifically, it can also provide a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned federated learning client selection solution for intelligent manufacturing.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0133] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0138] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0139] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0141] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0142] The above-mentioned method of the present invention, or a specific system unit, or a part of the same, is a pure software architecture, which can be arranged on a physical medium, such as a hard disk, an optical disk, or any electronic device (such as a smart phone, a computer-readable storage medium) through program code. When the machine loads the program code and executes it (such as a smart phone loading and executing it), the machine becomes a device for implementing the present invention. The above-mentioned method and device of the present invention can also be transmitted in the form of program code through some transmission media, such as cables, optical fibers, or any transmission mode. When the program code is received, loaded and executed by a machine (such as a smart phone), the machine becomes a device for implementing the present invention.

[0143] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for selecting a federated learning client for intelligent manufacturing, characterized in that: include: (1) Divide heterogeneous data. Divide industrial data into different levels of heterogeneous data for clients according to heterogeneity requirements; (2) Client local pre-training: each client trains its own local model based on the received initial model parameters and using its own data; (3) Collecting client model parameter change information, and collecting the model parameter difference between the first and last iteration training of the client local training; (4) Construct a graph structure to calculate the similarity of the client model. Use the collected information to construct an undirected graph with the client as the node, calculate the similarity of the nodes, and select the client combination based on this; (5) Train and save the results, perform federated learning training on the output new client group, and finally evaluate the aggregated model and save the evaluation results.

2. The method for selecting a federated learning client according to claim 1, characterized in that: The federated learning client selection method divides heterogeneous data through Dirichlet distribution.

3. The method for selecting a federated learning client according to claim 1, characterized in that: When constructing an undirected graph, the method for selecting a client for federated learning uses the total number of clients N as the number of nodes in the undirected graph, and each node represents each client C in the federated learning. n }, which constructs a complete graph G, in which there are edges corresponding to different nodes.

4. The method for selecting a federated learning client according to claim 1, characterized in that: The federated learning client selection method is based on the node value represented by each client, uses multi-dimensional cosine similarity to calculate the weights of the edges between client nodes respectively, and then maps the model similarity between two nodes based on the weights.

5. The federated learning client selection system for intelligent manufacturing is characterized by: The selection system includes a heterogeneous data division module, a client local pre-training module, a model parameter change information collection module, a client screening module and a client federated learning training module. The module for dividing heterogeneous data is configured to divide the industrial data into heterogeneous data of different degrees for the client according to heterogeneity requirements; The client local pre-training module is configured to interact with the data of the heterogeneous data division module, and can train each client to have its own local model based on the received initial model parameters and using its own data; The model parameter change information collection module is configured to interact with the client local pre-training module data and collect the model parameter difference between the first and last iterative training of the client local training; The client screening module is configured to interact with the model parameter change information collection module data, and calculate the client model similarity by constructing a graph structure, thereby screening the client combination accordingly; The client federated learning training module is configured to interact with the client screening module and the heterogeneous data partitioning module, and can perform federated learning training on the screened client groups, and finally perform model evaluation on the aggregated model.

6. The federated learning client selection system according to claim 5, characterized in that: The module for dividing heterogeneous data divides heterogeneous data through Dirichlet distribution.

7. The federated learning client selection system according to claim 5, characterized in that: The client screening module includes a client undirected graph construction submodule, a client model similarity calculation submodule, and a client selection submodule. The client undirected graph construction submodule is configured to construct an undirected graph with the client as a node; The client model similarity calculation submodule is configured to interact with the client undirected graph construction submodule data, and determine the node value represented by each client by the constructed undirected graph, and use the multi-dimensional cosine similarity to calculate the weight of the edge between the client nodes respectively, and then map the model similarity between the two nodes based on the weight; The client selection submodule is configured to interact with the client model similarity calculation submodule in data, screen the original clients based on the model similarity calculated by the client model similarity calculation submodule, and select a client set with closer similarity.

8. A processor, the processor being used to run a program, characterized in that: When the program is running, the steps of the method for selecting a federated learning client for intelligent manufacturing described in any one of claims 1 to 4 are executed.

9. A terminal device, comprising a processor, a memory, and a program stored in the memory and executable on the processor, characterized in that: The program code is loaded and executed by the processor to implement the steps of the federated learning client selection method for intelligent manufacturing described in any one of claims 1-4.

10. A computer program product, characterized in that When executed on a data processing device, it is suitable for executing the steps of the federated learning client selection method for intelligent manufacturing described in any one of claims 1-4.

Images