Methods and systems for operating technical facilities using optimal models
By generating and training device-specific models and forming customer groups based on similarity functions, the method enhances federated learning accuracy for anomaly detection in industrial systems, addressing the issue of heterogeneous datasets.
Patent Information
- Application Number
- CN202110441372.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-04-23
- Filing Date
- 2021-04-23
- Publication Date
- 2024-12-24
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing federated learning models struggle with insufficient accuracy due to heterogeneous and varying dataset sizes across local nodes, leading to suboptimal performance in anomaly detection and classification in industrial systems.
A method involving generating a basic model, distributing it to control devices, training device-specific models, and using similarity functions to form customer groups for federated learning, where similarity is calculated using centroid models and weighted group models to enhance accuracy.
Improves the accuracy of anomaly detection and classification by grouping similar devices for federated learning, allowing for more effective model training and better anomaly identification in industrial systems.
Smart Images
Figure CN113743614B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method and a system for operating a technical installation using an optimal model, wherein the installation is part of a system having a first technical installation and at least one second technical installation, each installation comprising a control device with a memory and connected technical devices, and the system also having a server with a memory (MEM).
[0002] The invention also relates to an installation for operation using an optimization model, comprising a control device and a technical device with a memory. Background Art
[0003] Federated Learning (FL, also known as collaborative learning or composite learning) is a machine learning (ML) technique that trains algorithms via multiple decentralized edge devices or servers that contain local data samples without exchanging their data samples in the process.
[0004] This known scheme is different from conventional centralized machine learning techniques, in which all data samples are uploaded to a server, and from classical decentralized schemes, in which local data samples are assumed to be uniformly distributed.
[0005] With federated learning, multiple actors can create common, comprehensive machine learning models without sharing data and thus addressing key topics such as data protection, data security, data access rights, and access to heterogeneous data.
[0006] Federated learning aims to train algorithms for machine learning, such as deep neural networks, based on multiple local datasets contained in local nodes without exchanging data samples.
[0007] The general principle is to train local models with local data samples and exchange parameters, such as weights of deep neural networks, between these local models at a certain frequency to generate a global model.
[0008] The federated learning algorithm can use a central server that coordinates the different steps of the algorithm and acts as a reference clock, or if there is no such central server, the federated learning algorithm can be peer-to-peer. In the absence of peer-to-peer, the federated learning process can be divided into multiple rounds, where each round consists of four general steps.
[0009] The main difference between federated learning and distributed learning lies in the assumptions made about the properties of the local dataset, as distributed learning was originally designed to parallelize computing power, while federated learning was originally designed to train heterogeneous datasets.
[0010] Distributed learning also aims to train a single model on multiple servers, but it usually assumes that the local datasets are identically distributed and of roughly the same size. None of these assumptions are made for federated learning. Instead, the datasets are often heterogeneous and can vary in size by multiple orders of magnitude.
[0011] In order to ensure good task performance of the final central machine learning model, the composite learning is based on an iterative process that is divided into atomic groups of client-server interactions called composite learning rounds.
[0012] Each round of the process consists in transmitting the current global model state to the participating nodes, training the local models on the local nodes to create a series of potential model updates on each node, and then aggregating these local updates into a single global update and applying it to the global model.
[0013] In the following methodology, we use a central server for the aggregation, while the local nodes perform local training according to the instructions of the central server. However, other strategies can also lead to the same results in a peer-to-peer solution using the rumor method without a central server.
[0014] For initialization in an iterative approach, a statistical model (e.g. linear regression, neural network, boosting, etc.) is selected to be trained and initialized on the local node. The node is activated and waits for the central server to take over the computational tasks.
[0015] Iterative training performs the following steps for multiple iterations of so-called compound learning rounds:
[0016] Selection: Select a portion of local nodes to start training local data. These local nodes all get the same current statistical model from the current server. Other nodes wait for the next compound round.
[0017] Configuration: The central server initiates the selected nodes to train the model based on their local data in a predefined way, such as some batch updates for gradient descent.
[0018] Reporting: Each node returns locally learned incremental model updates to the central server. The central server aggregates all results and stores the new model. The new model also handles errors, such as lost connectivity to nodes during the training. The system returns to the selection phase.
[0019] End: If a pre-given termination criterion is met, such as a maximum number of rounds or a higher local accuracy than a target, the central server commands the iterative training process to terminate. The central server may contain a robust model that has been trained based on multiple heterogeneous data sources.
[0020] However, in the prior art, it may happen in certain applications that even a jointly trained model does not have sufficient accuracy. Summary of the invention
[0021] The task of the present invention is to overcome such drawbacks in the prior art and to further improve the accuracy of the FL model.
[0022] The object according to the invention is achieved by a method of the type mentioned at the outset, wherein the following method steps are performed:
[0023] Q1) Generate a basic model of the equipment for at least one of the technical equipment,
[0024] Q2) distributing the device basic model to the first control device and the at least one second control device,
[0025] Q3) generating, training and storing a first device model by the first control device and generating, training and storing at least one second device model by the at least one second control device, respectively, based on the device basic model,
[0026] Q4) providing the first device model and the at least one second device model to the server, and storing the first device model and the at least one second device model in a model memory of the server,
[0027] Q5) loading at least those device models that are not already present in the control device and providing them to the corresponding control device,
[0028] Q6) applying the device model provided in step Q5) to the corresponding control device and determining a corresponding similarity function for the first control device, and applying the first similarity model to the at least one second control device and determining a corresponding similarity function for the at least one second control device,
[0029] Q7) transmitting the corresponding similarity function to the server,
[0030] Q8) forming at least one customer group by applying the similarity function by the server and creating a corresponding group model by performing federated learning within the customer group,
[0031] Q9) An operating model is selected from at least one group model for the installation and loaded, the operating model is provided to a control device of the installation, and the installation is controlled with the aid of the operating model selected as the best model.
[0032] The device base model is understood as the ML model used for similarity calculation (English: similarity modelskeleton).
[0033] The device basic model, such as an autoencoder, is created taking into account the data structure of the customer, without requiring any knowledge of the data content in the process.
[0034] The basic device model can then be received, for example, by the first control device and trained based on the specific customer data, for example by adapting the weights of an autoencoder with the aid of a “back propagation” algorithm known to those skilled in the art.
[0035] The basic model of the device may describe the general operating behavior of the device, for example, without considering a special operating environment or a special parameter combination during operation.
[0036] The device model can, for example, describe a special operating behavior of the device, such as a special operating behavior in a special operating environment or taking into account a special parameter combination during operation. Therefore, compared with the device basic model, the first device model and the second device model have additional model parameters or can also be established based on a larger training data set.
[0037] The operating model can map influencing variables of the plant operation to the ML model, wherein this can be done on the basis of a basic plant model, ie, for example, a basic basic plant model can be extended with current parameters.
[0038] For example, a group model may be derived or formed from a plurality of device models, for example by expansion of parameters, combination of parameters or by application of a modified abstraction function or a modified similarity function.
[0039] The operating model can be selected in step Q9, for example, by the user defining target parameters for the model or by the method according to the invention obtaining an improved model for the selected facility from the group model. Based on the parameters or the selection of the facility, a corresponding criterion can be specified, which allows a corresponding selection of the group model. The model is selected from the set of available group models, for example, by applying the mentioned criteria or by selecting a dedicated facility for which an improved model, i.e. the best model, is to be determined by means of "federated learning".
[0040] In a further step, the device model can be derived specifically for the second control device and used to calculate the degree of similarity with respect to the first control device.
[0041] For example, model groups or customer groups can be defined based on particularly important operating characteristics of the system, such as temperature, pressure, etc., ambient conditions or user parameters.
[0042] A customer group can have several customers with corresponding facilities, wherein the facilities each include technical equipment, but only one model is used for the facility or the equipment.
[0043] In the simplest case, the similarity function may be a scalar, ie a similarity value. Thus, when forming the similarity function, the numerical similarity value between two similarity models is determined by applying a similarity method, as described in step Q6).
[0044] The similarity function may also include a plurality of similarity values, for example different similarities may exist for different operating modes of the device, wherein these similarities may be described with the aid of the similarity function.
[0045] In principle, it is not necessary for the assets to include only devices of the same design or only devices with the same operating behavior. However, it is obvious that the model can be improved more effectively if similar devices are taken into account in the method according to the invention.
[0046] The similarity of devices may be related to construction characteristics or operating characteristics. Individual parameters as well as overall properties can be considered to determine similarity, such as operating temperature, mechanical vibration or power consumption of a single component or the entire system.
[0047] For example, electric motors, pumps, milling machines, production machines, industrial plants, vehicles can each be similar to one another by having the same design, being operated in comparable operating environments or having the same power of individual components or systems.
[0048] In one development of the invention, it is provided that after step Q7) a further step is carried out:
[0049] Q7a) When using the similarity function, at least one centroid model is calculated based on the "federated averaging" method,
[0050] And in step Q8), at least one centroid model is taken into account when forming at least one group model.
[0051] When generating the group model, the centroid model can be used to give higher weights to those operating models that are closer or more similar to the centroid model, thereby further improving the accuracy.
[0052] Substitute models are available for the basic device model, which can include additional features, parameters or characteristic variables of the device. This makes it possible to provide a more accurate ML model for special features when adding other new devices.
[0053] In one development of the invention, it is provided that after step Q7) a further step is carried out:
[0054] Q7b) Storing the similarity function in a similarity matrix.
[0055] This makes it possible to perform a rapid search for similarities and to determine a subset of the matrix particularly efficiently and quickly and to take this into account for forming or using the model.
[0056] In principle, other data structures are also suitable for storing similarity values or similarity functions, but the data can be processed particularly efficiently with the aid of the matrix.
[0057] In one development of the invention, it is provided that in step Q8) at least one weighting function is used when forming the at least one group model.
[0058] This makes it possible to combine data that are related to different degrees, for example, depending on the application, in a simple manner.
[0059] In one development of the invention, it is provided that the server performs the generation of the device basic model.
[0060] Obviously, the device base model must exist before starting the method. A new customer can access this device base model.
[0061] As a result, the system model can be used in a particularly simple manner as a starting basis for subsequent learning rounds.
[0062] In one development of the invention, it is provided that the installation is connected to the control device via a local network and to the server via a public network.
[0063] Good data protection is achieved by the method, since no device data is sent to the server, but only model data. The device data can be protected, for example, by a firewall.
[0064] However, a common FL model can be trained and thereby multiple devices can be connected to each other to be able to learn from each other.
[0065] In one development of the invention, it is provided that the at least one second installation is respectively connected to the corresponding control device via a local network and is respectively connected to the server via a public network.
[0066] The object according to the invention is achieved by a system of the type mentioned at the outset, wherein the system is configured to carry out the method steps according to the invention.
[0067] The object according to the invention is achieved by a system of the type mentioned at the outset, wherein the control device is configured to carry out the following method steps:
[0068] R1) Basic model of receiving equipment,
[0069] R2) generating, training and storing a first device model based on the device basic model,
[0070] R3) providing the device model via a first data interface,
[0071] R4) receiving at least one second device model which is not already present in the control device by means of a second data interface,
[0072] R5) applying the at least one second device model received in step R4) to the control device and determining a similarity function with respect to the first device model,
[0073] R6) providing the similarity function via a third data interface,
[0074] R7) Receiving an operating model formed from at least one group model by applying the similarity function by means of a fourth data interface, and controlling the device by means of the operating model as an optimal model.
[0075] In a development of the invention, it is provided that the first data interface and / or the second data interface and / or the third data interface and / or the fourth data interface is formed by a public network and that the installation is connected to the control unit via a local network.
[0076] Good data protection is achieved by the method, since no device data is sent to the server, but only model data. The device data can be protected, for example, by a firewall device. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] The present invention will be further described below based on the embodiments shown in the accompanying drawings. In the accompanying drawings:
[0078] Figure 1 shows the system architecture with industrial equipment controlled with the help of ML models,
[0079] Figure 2 Schematically shows two industrial edge devices, which represent customers,
[0080] Figure 3 An embodiment of the device according to the invention is shown,
[0081] Figure 4 shows an exemplary schematic diagram of the method according to the present invention,
[0082] Figure 5 An embodiment of the present invention is shown in pseudo code form.
[0083] Figure 6 An embodiment of the invention is shown in pseudo code form for forming groups.
[0084] Figure 7 Another embodiment of the present invention is shown in pseudo-code form. DETAILED DESCRIPTION
[0085] Figure 1 A system architecture with an industrial system that is controlled with the aid of an ML (English: Machine Learning) model is shown.
[0086] In order to avoid, for example, damage to assets, such as equipment, machines or facilities during production runs, ML models are trained to identify anomalies based on sensor data recorded in the past. Furthermore, these models are used and called in industrial buildings, for example for industrial control systems, in order to identify and classify anomalies. In this way, operators can be notified to stop the current process or reduce the production speed.
[0087] Furthermore, changes to the production plan can be initiated to avoid calling up the assets involved. In this case, the production process can take into account redundant assets if they are installed and have sufficient capacity. This can be achieved through interaction with the Manufacturing Execution System (MES). Furthermore, spare parts can be ordered automatically when the model classifies the problem and derives a solution with high accuracy.
[0088] However, training ML models based solely on the client’s data, i.e., data from a single facility operator, may result in the identification and classification of anomalies with minimal accuracy, since these models only consider patterns (Muster) derived from past events (Vorfall) for that client.
[0089] In this figure, a customer with an IoT device 10 is shown, such as general technical equipment, such as industrial facilities, robots, CNC milling machines, etc., that is, "assets".
[0090] The device 10 is connected to an edge device 20 , wherein the edge device 20 can access an ML model M0 , which is provided by a computing device with a memory, for example.
[0091] The following process steps are performed by the edge device 20:
[0092] P1 Firstly, the data of the device 10 is detected, pre-processed and made available to the edge device 20 .
[0093] P2 then performs anomaly detection by comparing the data with the ML model M0.
[0094] P3 then controls the device 10 derived from step P2.
[0095] P3a When a production problem is identified, for example, a “stop” of a process step or a “reduction” of the current processing can be triggered, wherein the process step interferes with the ongoing method.
[0096] P3b In addition, an "alarm" can be output to the operating personnel.
[0097] P3c In addition, based on this issue, production can be replanned with the help of the "Manufacturing Execution System" (MES).
[0098] P3d In addition, the ordering of spare parts can be initiated with the aid of an ERP (Enterprise Resource Planning) system.
[0099] Figure 2 Two industrial edge devices 11 and 12 are schematically shown, each representing a customer. The goal is to correctly classify anomalies in the respective production facilities, which are mapped by corresponding ML models P2, P3 and stored in a computing device with corresponding memory, which the edge devices 11, 12 can access.
[0100] Now, the following describes Figure 1 The following process steps P4 to P9 are carried out in a complementary manner.
[0101] The control and data flow can be described by process steps P1 to P3 and correspond to the original anomaly classification process described in the previous paragraph.
[0102] The process is now extended by an additional model update workflow with process steps P4 to P9 , which can be executed in parallel with process steps P1 to P3 .
[0103] P4 uploads the current ML model, such as neural network weights and metadata, such as the size of the dataset and the number of different classes, to the server 30 by modeling and evaluating metrics. Evaluation metrics are also uploaded in subsequent process iterations, where for example the accuracy of a classifier is considered, where the classifier is evaluated on a locally held validation dataset.
[0104] P5 performs a distribution of models and data, in which case the IFL (Industrial Federated Learning) server FLS calls the IFL group manager FLM to create groups based on the distribution of the categories of participating clients.
[0105] Customers with similar class distributions are grouped together. This can be done, for example, by clustering using a distance metric that focuses on similarities and differences in data distribution. In addition, the group is supported by a configurable number of collaborators from other groups.
[0106] These collaborators are identified to contribute knowledge based on the classification of categories that are less present in the corresponding group.
[0107] P6 returns the generated group configuration to the IFL server to aggregate the model parameters of group members and collaborators.
[0108] P7 stores the updated global ML model MG of all groups by means of updates to the cluster (Aggregate).
[0109] P8 forwards the global ML model MG to the client as a model update.
[0110] P9 By retraining using the updated models M1, M2, the client uses the corresponding global model MG to initialize local training on the client.
[0111] Figure 3An exemplary embodiment of the invention is schematically shown in the form of an apparatus for operating each of the technical installations A1 - A3 with a correspondingly optimal ML model.
[0112] The system S includes a first technical facility A1, a second technical facility A2 and a third technical facility A3.
[0113] Each installation A1 - A3 comprises a control device 15 - 17 with a memory and technical devices 25 - 27 .
[0114] The system also has a server FLS1 with a memory MEM.
[0115] The first, second and third facilities A1 - A3 are connected to corresponding control devices 15 - 17 via local networks 5 - 7 , respectively, and are connected to a server FLS1 via a public network 1 , respectively.
[0116] Alternatively, facilities A1-A3 can jointly form an industrial facility with multiple technical devices and be connected to each other via a local network. In this case, the industrial facility can be connected to the server FLS1 via a proxy device with a public network, for example, together with a firewall device.
[0117] In this example, technical facilities A1-A3 are three customers with corresponding edge devices.
[0118] The edge devices 15 - 17 may access the ML model MB, wherein the ML model MB is provided by, for example, a computing device with a memory.
[0119] The following process steps are performed by the system:
[0120] Q1 is based on the technical equipment generation and storage equipment basic model MB.
[0121] Q2 distributes the device base model MB to the first, second and third edge devices 15-17 of the corresponding customers.
[0122] Q3 Based on the basic model MB
[0123] • A first device model MS1 is generated and trained by the first edge device 15,
[0124] • generating and training a second device model MS2 by the second edge device 16, and
[0125] •The third device model MS3 is generated and trained by the third edge device 17.
[0126] Q4 The trained first, second and third device models MS1-MS3 are provided to the IFL server FLS1 and stored in the model memory MEM of the IFL server FLS1.
[0127] Q5
[0128] • loading and providing the second and third similarity models MS2, MS3 to the first edge device 15,
[0129] • loading and providing the first and third similarity models MS1, MS3 to the second edge device 16, and
[0130] • Loading and providing the first and second similarity models MS1 , MS2 to the third edge device 17 .
[0131] Q6
[0132] • applying the second and third similarity models MS2, MS3 to the first edge device 15 and determining corresponding similarity values SF1_2, SF1_3 with respect to the first edge device 15,
[0133] • applying the first and third similarity models MS1, MS3 to the second edge device 16 and determining corresponding similarity values SF2_1, SF2_3 with respect to the second edge device 16,
[0134] • applying the first and second similarity models MS1, MS2 to the third edge device 17 and determining corresponding similarity values SF3_1, SF3_2 with respect to the third edge device 17,
[0135] In this embodiment, the similarity functions are scalars, ie, similarity values.
[0136] Q7 transmits the corresponding similarity values SF1_2, SF1_3, SF2_1, SF2_3, SF3_1, SF3_2 to the IFL server FLS1.
[0137] Q7a Based on the "federated average" method, at least one centroid model MC1-MC3 is calculated using the similarity function (SF1_2, SF1_3, SF2_1, SF2_3, SF3_1, SF3_2).
[0138] The centroid models MC1-MC3 represent the models having the greatest similarity within a set of similarity models (Ähnlichkeitsmodellen-Modellen) MS1-MS3, respectively.
[0139] Q7b stores the similarity values SF1_2, SF1_3, SF2_1, SF2_3, SF3_1, SF3_2 in the similarity matrix SIM.
[0140] Q8 When the similarity values SF1_2, SF1_3, SF2_1, SF2_3, SF3_1, SF3_2 are applied by the IFL server FLS1, at least one customer group is formed and corresponding group models MG1-MG3 are created by performing federated learning within the customer group, wherein at least the centroid models MC1-MC3 are taken into account and at least one weighting function is applied to the corresponding group models MG1-MG3.
[0141] Q9 An operating model M01 is selected and loaded for the installation A1 from at least one group model MG1 - MG3 , and the operating model M01 is provided to the control device 15 of the installation A1 , and the device 25 is actuated with the aid of the operating model M01 selected as the best model.
[0142] The device basic model MB is generated or provided by the server.
[0143] Obviously, the model memory MEM may be in communication with said server and may also be located in the cloud, for example.The model memory MEM is assigned to the IFL server FLS1.
[0144] Group formation can be understood as the grouping or clustering of distributed FL clients based on similar data or data sets.
[0145] The groups can be formed using a similarity model using a group formation algorithm, wherein the similarity model ensures data protection for the used data or data sets. Here, such an algorithm returns similarities from the control device of the client.
[0146] This enables the FL process to facilitate the exchange of knowledge among appropriate groups to thereby improve accuracy.
[0147] Here, the IFL server may perform a known "federated averaging" algorithm for each group.
[0148] In order to determine the corresponding similarity between the data of two customers A and B or, in the above example, between A1 and A2, a similarity model can be introduced, such as an autoencoder, which is trained with the help of the data of model B and the data of model A is applied to the autoencoder in order to calculate the similarity therefrom.
[0149] This results in a reconstruction error which is larger when the data are dissimilar and smaller when the data are similar. In other words, if the similarity of two customers A and B is large, they enter the same group.
[0150] The method should enable optimal group formation, ie grouping FL customers with similar data in order to ensure improved accuracy within groups.
[0151] In the following, a random group algorithm based on a similarity measure or a distance measure, such as a hierarchical clustering algorithm with variants such as “SingleLinkage” or “CompleteLinkage”, is used as a starting point.
[0152] The grouping process can be performed in parallel with the FL process. The resulting groups can be used to trigger the reorganization of the FL process to perform subsequent learning rounds.
[0153] The similarity between two customers A and B can be described by the following relationship:
[0154] Similarity (A, B)
[0155] = Similarity(Similarity Model(A), Dataset(B))
[0156] +Similarity(Similarity Model(B), Dataset(A))
[0157] The first item is calculated for customer B, in which there is a data set of customer B, and the second item is calculated for customer A, in which there is a data set of customer A.
[0158] For example, the reconstruction error can be calculated for an autoencoder similarity model:
[0159] Similarity (A, B)
[0160] = 1⁄Reconstruction Error (Autoencoder (A), Dataset (B))
[0161] + 1⁄Reconstruction error (autoencoder (B), dataset (A))
[0162] For the installation A1-A3 according to the invention for operation using an optimal model, the following method steps are provided for the corresponding control devices 15-17:
[0163] R1) corresponds to step Q2),
[0164] R2) corresponds to step Q3),
[0165] R3) corresponds to step Q4),
[0166] R4) corresponds to step Q5),
[0167] R5) corresponds to step Q6),
[0168] R6) corresponds to step Q7),
[0169] R7) corresponds to step Q9).
[0170] Figure 4 An example is schematically shown of a method in which group-specific data elements are formed and thus the group-specific accuracy can be improved.
[0171] In contrast to the prior art, in which group models M1-M4 are considered in group G1, only group models M1-M3 can be included in group G2, since model M4 differs greatly from the other models M1-M3 and is therefore not similar enough. The common group model MG2 of group G2 is therefore more accurate.
[0172] Optionally, in some applications the M4 model may only be considered with a lower weight than the M1-M3 models.
[0173] In other words, similar customers are combined in one group or cluster, and collaborators (also called “collaborateurs”) are linked with a similar but new category name.
[0174] This enables good performance in classifying anomalies that have been logged frequently in the group, and is ready to classify potential future anomalies that have been logged by other customers.
[0175] Ignoring customers (ie group participants) with very different class distributions, for example group participants with the M4 model, ensures high accuracy in classifying known anomalies without introducing too much distortion.
[0176] While the global model has only moderate accuracy due to the heterogeneous class distribution, the group model shows improved accuracy by aggregating models within the same group and associated collaborators.
[0177] In other words, the global model aggregates all customer models, while the group model selectively aggregates only the models of customers having models M1 - M3 into the group model MG1 .
[0178] The following figures now illustrate embodiments of the present invention in pseudo-code form.
[0179] Here, a group is also referred to as a "cohort". The "centroid" means the center of a group.
[0180] Figure 5 An embodiment of the present invention is shown in pseudo-code form.
[0181] A group-based FL process is shown in this example to improve the classification model in identifying anomalies.
[0182] Set a limited set of customers as input set
[0183] Clients = {c1, c2, …, c n}.
[0184] In line 5, the model determined for a specific group and improved with regard to anomaly classification is sent to the customer by additionally requesting the customer to train the group model for the subsequent round. Thus, after each subsequent step, the customer can benefit from the improved model.
[0185] It can be seen in row 9 that the weights are taken into account when forming the corresponding model (w).
[0186] The quotient in the second term takes into account, for example, the Coh ) so that more similar models are included to a greater extent than less similar models.
[0187] The algorithm shown can be modified in that the iteration process is terminated when a predetermined desired accuracy or an accuracy maximum is reached.
[0188] When adding new customers to the FL process, group formation can also be performed there.
[0189] Figure 6 An embodiment of the present invention is shown in pseudo-code form for forming a group.
[0190] Figure 6 The pseudo code is Figure 5 , in order to facilitate joint learning for multiple groups.
[0191] Set a limited set of customers as input set
[0192] Clients = {c1, c2, …, c n},
[0193] The client participates in the FL process.
[0194] The algorithm provides a finite set of customers as output
[0195] .
[0196] In addition, for all A finite set of centroids (Centroids) of the aggregated similarity models is determined.
[0197] Figure 7 Another embodiment of the present invention is schematically shown as pseudo code.
[0198] We now further elaborate on the behavior of the IFL server that performs global IFL group learning, which leads to a global model for each group C and time step t.
[0199] At the beginning of each time step, the current group configuration is evaluated and new customer clusters with updated training data may be considered.
[0200] For each group, m collaborators are selected in order to introduce knowledge of customers with different class distributions.
[0201] The actual interaction with customers and collaborators takes place in two inner loops to initiate training for groups of customers based on the current model.
[0202] Additionally, metrics such as accuracy, size of the training dataset, and number of classes can be examined.
[0203] These metrics are important in order to aggregate the model parameters for the set of models in the next round t+1.
[0204] Unlike known methods based on federated learning, the averaging of local customer models in line 17 does not only use the dataset size n k as the participation weight.
[0205] Therefore, also about the different categories of customer models c k The number and accuracy of the customer model acc k These data are provided by the customer and therefore make it possible to have a good model with many categories, i.e. a model with correspondingly high accuracy, and make it more likely that these data will be included in the aggregated global group model in row 17. is weighted.
[0206] To take into account the knowledge of the collaborator (also called “collaborateur”), the second added term in row 17 shows the weighted inputs of the m collaborator models.
[0207] List of reference numerals:
[0208] 1 Network
[0209] 5-7 Local Network
[0210] 10-12, 15-17 Control devices, edge devices
[0211] 20-22, 25-27 IoT devices, technical equipment, “assets”
[0212] FLS, FLS1 Model Server
[0213] FLM, FLM1 Group Manager
[0214] G1, G2 Group
[0215] A1-A3 Facilities
[0216] S system
[0217] M0, MB basic model
[0218] M01 Operation model
[0219] MC1-MC3 Centroid Model
[0220] MG, MG1, MG2 group models
[0221] M1-M4, MS1-MS3 Similarity Model, “Similarity” Model
[0222] MES Production System "Manufacturing Execution System"
[0223] ERP Resource Planning System "Enterprise Resource Planning"
[0224] P1-P9, Q1-Q9, R1-R7 process steps
[0225] P1 Preprocessing data
[0226] P2 Detection Anomaly
[0227] P3 Device Control
[0228] P3a Stop / Reduce
[0229] P3b Alert
[0230] P3c Re-planning of production
[0231] P3d Order spare parts
[0232] P4 Modeling and evaluating metrics
[0233] P5 Model and Data Distribution
[0234] P6 Group Configuration
[0235] P7 Cluster Updates
[0236] P8 Model Update
[0237] P9 Retrain with updated model
[0238] Q1 Generate equipment basic model MG
[0239] Q2 Distribute the device basic model MG to edge devices
[0240] Q3 Generate and train device models based on device basic model MB through edge devices
[0241] Q4 Provide device models MS1-MS3 to IFL server FLS1 and storage
[0242] Q5 Loading and serving similarity models to edge devices
[0243] Q6 Apply the similarity model to the edge device and determine the similarity function or similarity value
[0244] Q7 Transmit the similarity function or similarity value to the IFL server
[0245] Q7a Calculate the centroid model
[0246] Q7b Store the similarity function or similarity value in the similarity matrix SIM
[0247] Q8 Formation of model group
[0248] Q9 Use the running model to control the device
[0249] R1 Receiver Basic Model
[0250] R2 Generate, train, and store first device models
[0251] R3 provides equipment models
[0252] R4 Receive Second Device Model
[0253] R5 applies the second device model and determines a similarity function or similarity value
[0254] R6 provides similarity function or similarity value
[0255] R7 receives the running model.
Claims
1. A method for operating a technical installation (A1-A3) using an optimal model, wherein the technical installation (A1-A3) is part of a system (S) having a first technical installation (A1) and at least one second technical installation (A2, A3), each technical installation (A1-A3) comprising a control device (15-17) with a memory and connected technical devices (25-27), and the system also having a server (FLS1) with a model memory (MEM), characterized in that The following method steps Q1)-Q9) are performed: Q1) generating a device basic model (MB) for at least one of the technical devices (25-27), Q2) distributing the device basic model (MB) to a first control device (15) and at least one second control device (16, 17), Q3) generating, training and storing a first system model (MS1) by the first control device (15) and generating, training and storing at least one second system model (MS2, MS3) by the at least one second control device (16, 17) based on the system basic model (MB), respectively; Q4) providing the first device model (MS1) and the at least one second device model (MS2, MS3) to the server (FLS1), and storing the first device model (MS1) and the at least one second device model (MS2, MS3) in a model memory (MEM) of the server (FLS1), Q5) loading at least those second device models (MS2, MS3) that are not already present in the first control device (15) and providing them to the corresponding first control device (15), Q6) applying the second device model (MS2, MS3) provided in step Q5) to the corresponding first control device (15) and determining a corresponding first similarity function (SF1_2, SF1_3) about the first control device (15), and applying the first similarity model to the at least one second control device (16, 17) and determining a corresponding second similarity function (SF2_1, SF2_3, SF3_1, SF3_2) about the at least one second control device (16, 17), Q7) transmitting the first similarity function (SF1_2, SF1_3) and the second similarity function (SF2_1, SF2_3, SF3_1, SF3_2) to the server (FLS1), Q8) when the first similarity function (SF1_2, SF1_3) and the second similarity function (SF2_1, SF2_3, SF3_1, SF3_2) are applied by the server (FLS1), at least one customer group is formed and a corresponding group model (MG1-MG3) is created by performing federated learning within the customer group, Q9) For a first technical facility (A1), an operating model (M01) is selected and loaded from at least one group model (MG1-MG3), and the operating model (M01) is provided to a first control device (15) of the first technical facility (A1), and a first technical device (25) connected to the first control device (15) is controlled by means of the operating model (M01) selected as the optimal model.
2. The method according to claim 1, wherein: After step Q7), a further step is performed: Q7a) using said first similarity function (SF1_2, SF1_3) and said second similarity function (SF2_1, SF2_3, SF3_1, SF3_2), calculating at least one centroid model (MC1-MC3) based on the "federated average" method, And in step Q8), at least one centroid model (MC1-MC3) is taken into account when forming at least one group model (MG1-MG3).
3. The method according to claim 1, wherein: After step Q7) the further steps are performed: Q7b) storing the first similarity function (SF1_2, SF1_3) and the second similarity function (SF2_1, SF2_3, SF3_1, SF3_2) in a similarity matrix (SIM).
4. The method according to claim 1, wherein: In step Q8), at least one weighting function is applied when forming the at least one group model (MG1-MG3).
5. The method according to any one of claims 1 to 4, wherein: Executed by the server: generating the device basic model (MB).
6. The method according to any one of claims 1 to 4, wherein: The first technical installation (A1) is connected to the first control device (15) via a local network (5, 6, 7) and is connected to the server (FLS1) via a public network (1).
7. The method according to any one of claims 1 to 4, wherein: The at least one second technical installation (A2, A3) is connected to the corresponding second control device (16, 17) via a local network (5, 6, 7) and is connected to the server (FLS1) via a public network (1).
8. A system (S) for operating a technical installation (A1-A3) using an optimal model, wherein the system (S) comprises a first technical installation (A1) and at least a second technical installation (A2, A3), each technical installation (A1-A3) comprising a control device (15-17) with a memory and a technical device (25-27), and the system also has a server (FLS1) with a model memory (MEM), characterized in that The system (S) is arranged to perform the steps of the method according to any one of claims 1-7.
9. A technical installation (A1-A3) for operating using an optimal model, the technical installation (A1-A3) comprising a control device (15-17) with a memory and a technical device (25-27), characterized in that: The control device (15-17) is configured to perform the following method steps: R1) receiving equipment basic model (MB), R2) generating, training and storing a first device model (MS1) based on the device basic model (MB), R3) providing said first device model (MS1) via a first data interface, R4) receiving, by means of a second data interface, at least one second device model (MS2, MS3) which is not already present in the control device (15-17), R5) applying the at least one second device model (MS2, MS3) received in step R4) to the control device (15-17) and determining at least one similarity function (SF1_2, SF1_3) with respect to the first device model, R6) providing the at least one similarity function (SF1_2, SF1_3) via a third data interface, R7) receives, by means of a fourth data interface, an operating model (M01) formed by at least one group model (MG1-MG3) when applying the at least one similarity function (SF1_2, SF1_3), and controls corresponding technical devices (25-27) connected to the control device (15-17) by means of the operating model (M01) as the optimal model.
10. The technical installation (A1-A3) according to claim 9, wherein: The first data interface and / or the second data interface and / or the third data interface and / or the fourth data interface are formed by a public network (1), and the technical installation (A1-A3) is connected to the control device (15-17) via a local network (5, 6, 7).