Computer implementation methods for modifying machine learning models for industrial process plants
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-11
- Publication Date
- 2026-08-14
AI Technical Summary
例如，在工厂操作期间，可能会发生错误，例如在一个或多个工业设备内发生错误
Smart Images
Figure CN122572718A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a computer implementation method for modifying a machine learning model to incorporate underrepresented subgroups into training data from operations at an industrial process plant; it also relates to one or more computer program products, a data processing system, and a plant operating system for an industrial process plant. Background Technology
[0002] Plant operating systems for industrial process plants comprise a combination of hardware and software for a wide range of industrial applications. These applications include, but are not limited to, mining, transportation and processing, chemical plants, water and wastewater treatment, power plants, and pharmaceutical processing facilities. The plant operating systems used are typically engineered to monitor and manage industrial equipment and processes specific to the application.
[0003] The operation of these industrial process plants is typically carried out through a plant operating system, which serves as a control interface for one or more plant operators (usually engineers or technicians) who are currently responsible for controlling or supervising the operation of the industrial process plant.
[0004] Currently, factory operating systems are known to employ machine learning models to assist factory operators in the operation of industrial process plants, particularly in controlling and / or supervising their operations. These machine learning models can be used to provide various types of assistance to factory operators. For example, errors may occur during factory operations, such as within one or more industrial machines. Machine learning models can be used to determine the severity of such errors, whether the error has occurred before, to provide the factory operator with explanations on how to resolve the error, to determine whether the error is actually an error or an anomaly, and so on. Another example where machine learning models can be used is to predict the quality of outputs from industrial process plants, such as when processing products like paint in batches. Here, the quality of different batches may be affected by different seasons and due to different materials, and different problems may exist; machine learning models can help factory operators analyze large amounts of data. Summary of the Invention
[0005] A potential problem when building such machine learning models is the use of imbalanced training data, a common scenario in industrial machine learning or artificial intelligence applications (note that the terms "machine learning" and "artificial intelligence" are understood and used interchangeably in this article). In this case, the machine learning model may fail to learn the latent semantics of patterns (or, in other words, scenarios) that are not adequately represented in the imbalanced training data.
[0006] In particular, it has been found that the highly dynamic nature of industrial process plants makes it difficult to capture enough data (used for training machine learning models) to relate to all patterns or scenarios within the plant. For example, the data captured during malfunctions in an industrial process plant is always less than the data captured during normal operation. Therefore, when using this data to build a machine learning model, the model may fail to learn the latent semantics of scenarios that are not adequately represented. Another example is information about different batch runs of white paint production in an industrial process plant. Data from different paint runs may be available, but the type of input material used in each run might be overlooked. This could cause the model to miss certain runs that resulted in poor quality due to inferior input raw materials. However, since this information is not directly available in the data, the model may not perform adequately when it encounters such data.
[0007] Because the model cannot learn the semantics of the underrepresented data, its performance degrades when encountering similar scenarios in industrial process plants. This leads to erroneous outputs when the model is deployed in the production environment of an industrial process plant, resulting in decreased service quality and a loss of customer trust. Therefore, there is a need for machine learning models that assist plant operators in the operation of industrial process plants, models that are robust to underrepresented data, and thus perform well enough in most scenarios of industrial process plant operation.
[0008] The subject matter of the independent claims of this disclosure addresses or alleviates at least part of the aforementioned problems or needs, with further examples incorporated in the dependent claims.
[0009] According to one aspect of this disclosure, a computer-implemented method is provided, comprising:
[0010] Acquire training data from operations at industrial process plants;
[0011] Use training data to train machine learning models to assist plant operators in the operation of industrial process plants, especially but not limited to assisting plant operators in the operation of industrial process plants.
[0012] Multiple subgroups were identified in the training data, and each subgroup was associated with different scenarios of operation (especially similar operation) in an industrial process plant;
[0013] Determine multiple performance metrics for the machine learning model, each of which is associated with a different subgroup;
[0014] Based on the performance of each subgroup within the subgroup, identify one or more underrepresented subgroups that are underrepresented in the training data compared to other subgroups within the plurality of subgroups; and
[0015] Based on one or more underrepresented subgroups, the machine learning model is modified to incorporate consideration of these one or more underrepresented subgroups when assisting plant operations (especially but not limited to plant operators) in industrial process plants.
[0016] Therefore, this method can effectively modify trained machine learning models to be robust to underrepresented data, thus performing well in most operating scenarios of industrial process plants.
[0017] The terms “machine learning model,” “training data,” “subgroup,” “scenario,” and “performance” should be understood broadly in this document. Training data refers to any data from the operations of an industrial process plant that can be used to train a machine learning model in a general manner or in a specific context assisting the plant operator. Examples of such training data are sensor readings or any other form and type of data collected from technical equipment installed in the industrial process plant. For example, training data can be any type of data, such as text, time-series data, images, videos, etc., but is not limited to these. For example, error text issued by the plant operating system, time-series data of sensor readings, and images or videos from the plant operating system's user interface can be used as training data for training a machine learning model. Machine learning models can be trained using any technique, including but not limited to supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and so on.
[0018] Modifications to a trained machine learning model can be made using any technique, such as retraining or fine-tuning. Modifications can be performed, in particular, automatically or semi-automatically. In a semi-automatic manner, the plant operating system may require operator input to modify the machine learning model, such as for confirmation, but is not limited to this.
[0019] In training data derived from operational data of industrial process plants, subgroups refer to distinct groups within the training data that share specific characteristics that may influence the behavior of machine learning models trained with them. These groupings can, in particular, take the form of operational groupings and / or contextual groupings. These distinct subgroups, or in other words, data groupings, can be identified using different techniques, such as focusing on operational and / or contextual groupings. Examples of such techniques, such as statistical techniques, will be discussed further below.
[0020] Subgroups or groups typically share scenarios of operations within an industrial process plant (especially, but not limited to, similar operations within an industrial plant). Typically, a scenario refers to a specific operational or data distribution configuration where subgroup dynamics can significantly impact model performance or reliability. These distinct scenarios exist within the data due to the contextual differences reflecting the real-world operations of an industrial process plant.
[0021] Inadequately represented subgroups in industrial training data typically refer to data clusters or groups, especially, but not limited to, those containing specific operating conditions, equipment states, or process variables that occur less frequently within the dataset compared to the dominant patterns. In the context of an industrial process plant, this might involve rare failure conditions (e.g., a specific mechanical failure with a probability of less than 1% occurring during operating time), extreme environmental parameters (abnormal temperature / pressure combinations), or a limited number of product variations in production batches. These subgroups or clusters of data become underrepresented when data is disproportionately captured during the routine operation of an industrial process plant, while edge cases are undersampled. As discussed in this paper, this can be a critical issue because machine learning models may fail to identify these patterns during deployment, potentially missing key anomalies, for example, in quality control or predictive maintenance scenarios.
[0022] The performance of a machine learning model is typically quantified by its ability to generalize to new industrial data while maintaining operational reliability. It can be measured using task-specific metrics. For example, a classification task can be used, employing a precision-recall tradeoff to balance false positives and false negatives in anomaly identification systems. Another option is a regression model, which can use mean absolute percentage error to quantify bias in production output predictions. Other options include cross-validation techniques and advanced monitoring. This approach can use any of the above or any other available techniques to determine the specific performance of a machine learning model across multiple performance metrics for each distinct subgroup.
[0023] From the performance of different subgroups, it is possible to deduce which subgroups are relatively low-performing and which are relatively high-performing. Typically, the low-performing subgroups are likely those underrepresented in the training data, as the model cannot perform well on these subgroups. Therefore, using the methods described in this paper, it is known which subgroups are underrepresented and need to be addressed in the training data for training the machine learning model. To this end, the machine learning model is modified to take these underrepresented subgroups into account when performing the task (i.e., assisting the factory operator in factory operations). For example, as already mentioned, the machine learning model can be retrained or fine-tuned using the underrepresented subgroups. Other examples of modifications are given below.
[0024] The method of the first aspect can be, in particular, a method implemented at least partially or entirely by a computer. This means that at least one, several, or all steps of the method can be performed by a data processing system, which may include one or more computers or data processing devices, which may include one or more processors and data memory or storage. Different steps can be performed by the same or different computers in the data processing system. Specifically, the method can be performed by a control system or on a control system.
[0025] Typically, the entire method can be executed on one or more computers, which can run or serve as the factory operating system for an industrial process plant. Specifically, the factory operating system may include one or more computers, which may also reside in a distributed computing environment. In addition to hardware, the factory operating system may also include software components that enable it to control the industrial plant, such as the technical equipment or devices controlling the plant, including sensors and actuators. To receive input from operators, such as operator feedback to a machine learning model, the factory operating system can typically include any hardware input device, such as, but not limited to, a keyboard, mouse, touchscreen, etc. Furthermore, to provide operators with any information, data, etc., such as the output from the machine learning model, the factory operating system can typically include any hardware output device, such as one or more screens or displays, speakers, etc.
[0026] Typically, machine learning models can employ any conventional or specific machine learning technique, such as, but not limited to, anomaly detection, classification, clustering, neural networks, and so on. Often, using training data from operations at industrial process plants, machine learning models can be trained and / or used to enhance industrial plant operations by providing intelligent decision support systems for plant operators or plant operations, handling complex data patterns that even exceed human capabilities. An example of a machine learning model task is real-time process optimization. Here, the machine learning model can analyze multivariate sensor data (e.g., temperature, pressure, flow rate) to dynamically adjust process parameters in the process plant operation. Other alternative or complementary examples include using machine learning models for predictive maintenance, anomaly detection, energy and resource management in industrial process plants, and digital twin simulation. Typically, machine learning models can be combined with the examples above and assist plant operators or operations in a variety of ways. For example, a machine learning model can generate outputs for the plant operator at the human-machine interface of a plant operator system. These outputs can contain the results of the machine learning model for a specific task, which may be requested by the plant operator or executed automatically in response to a specific event (e.g., an alarm). This outcome can include, for example but not limited to, interpretations, instructions, or settings provided by machine learning models within a factory operating system to handle specific situations or events, such as resolving problems that trigger alarms. Another example is energy and resource management, where factory operators may want to know certain metrics or analyses of energy and resource usage, and machine learning models can efficiently search for or deduce this information. Therefore, factory operators can receive assistance during factory operations, such as resolving problems that trigger alarms or operating the factory based on energy and resource usage. However, in other examples, factory operations can be assisted directly without the intervention of factory operators, such as for predictive maintenance planning.
[0027] Of course, the method may also include, for example, executing a machine learning model on a factory operating system to assist factory operators in operation or generally in factory operations in an industrial process plant. The execution of the machine learning model can be triggered automatically, such as when certain events occur, such as when abnormal sensor readings are determined; semi-automatically, such as when certain events occur and confirmation of the model's activation is received based on user input from the factory operator; or manually, such as when a user requests a specific output or analysis from the machine learning model.
[0028] Training data may include time-series process data, which indicates sensor readings from sensors in an industrial process plant. However, training data is not limited to this and may additionally or alternatively include other data that can be generated by the plant operating system as mentioned herein, such as in the form of text, images, or video.
[0029] Subgroups within time-series process data can be identified by applying one or more statistical techniques to the data. For example, a two-step approach can be used as a statistical technique to create subgroups, or in other words, subgroup clusters. First, statistical clusters are identified. Second, these statistical clusters are contextualized, for example, using domain information data, which can be part of or included as part of the training data. Typically, domain information refers to specialized operational knowledge and contextual understanding that informs how to interpret, process, and / or model data within a specific industrial process plant. This can include explicit technical parameters and / or tacit expertise regarding equipment, processes, and operational constraints within the industrial process plant. Domain information can be particularly contained within metadata. It can correspond to input data such as different raw materials, operating conditions, batches, categories, alarms, events, and / or other plant-related information. This domain information is typically available in unstructured formats (such as documents) or structured formats (such as databases / SAP / quality systems, etc.). Further specific examples of the two-step approach are provided below.
[0030] Training data may also include domain information data indicating events in the operation of an industrial plant. Additionally or alternatively, the method may also include acquiring domain information data. Training data can be further enriched by domain information, especially when knowledge about other subgroups that might not have been previously available is acquired, against which the model can be tested. In any case, multiple subgroups in the training data can be determined based on the training data and the domain information data. Thus, the domain information data used to determine the subgroups can indicate events such as errors, alarms, etc., in plant operation. Also as mentioned above, using such domain information allows for better determination of the training data, or (in other words) dividing the training data as a whole data group into multiple subgroups. In particular, because domain information involves specialized operational knowledge and contextual understanding as described above, especially (but not limited to) indications of events in the operation of an industrial plant, the method can better segment the overall data group into multiple subgroups for different scenarios. For example, when using events included in the domain information, the domain information can specify various types of events that may occur in plant operation, such as certain error events, alarm events, etc. Each of these events can form a different scenario of industrial plant operation, or multiple similar events (e.g., similar in one or more characteristics of plant operation) can be aggregated into different scenarios of industrial plant operation. Therefore, different subgroups can be segmented in a way that makes them well distinguishable from each other, and their performance can be analyzed to determine their representativeness within the overall population of the training data, particularly whether they are underrepresented.
[0031] Domain information data can be provided, or in other words, fed into one or more Large Language Models (LLMs) for identifying subgroups of industrial process plants. This LLM can be a general-purpose LLM for understanding and analyzing any arbitrary text and / or other modalities (such as images, videos, etc.), and / or it can be specifically trained or fine-tuned to better understand and analyze the information contained within the domain information (e.g., in text form). Therefore, the LLM can allow the extraction of relevant information from the domain information, such as information related to different events or scenarios in plant operations, which can be used to identify subgroups. For example, the LLM can extract from the domain information which features typically characterize different events or scenarios, such as typical or commonly used values of sensor readings in plant operations, and then identify those subgroups that have similar sensor values for each of the different events or scenarios, as specified by the domain information. The LLM can be configured, in particular, as a generative and / or extractive artificial intelligence model. For example, a retrieval-reader architecture without generative capabilities can be used to extract information (e.g., text) from the domain information data. Typically, the domain information data can include one or more documents, such as documentation. Therefore, the LLM can generate and / or extract content from the domain information.
[0032] A machine learning model or another model (especially another machine learning model) can be used to identify multiple subpopulations in the training data. This model overlays information from identified subpopulations determined based on one or more statistical techniques and one or more LLMs, particularly onto the training data. Therefore, the model overlays information from identified subpopulations based on (multiple) statistical techniques and (multiple) LLMs to improve insight into the reasons for potentially poor model performance on underrepresented subpopulations.
[0033] Modifications to a machine learning model can include augmenting the training data with synthetic training data targeting one or more underrepresented subgroups, and then retraining the machine learning model using the augmented training data. Synthetic training data is artificially generated training data, not derived from actual operations in an industrial process plant, but can be simulated or computed, for example, based on underrepresented subgroups, thereby supplementing the training data with more subgroups representing the same operational scenarios in the industrial process plant. By adding these synthetic subgroups, previously underrepresented subgroups will no longer be underrepresented.
[0034] Additionally or alternatively, modifications may include selecting different machine learning models to train on the training data to assist plant operators in plant operations or, more generally, plant operations in industrial process plants. Thus, different machine learning models can be selected, particularly those better suited to better account for underrepresented subgroups.
[0035] Additionally or alternatively, modifications may include adjusting one or more hyperparameters of the machine learning model and retraining the model using the adjusted hyperparameters with the training data. Thus, a robust machine learning model can be obtained in a particularly efficient manner to better account for underrepresented subpopulations.
[0036] Additionally or alternatively, modifications may include selecting different learning algorithms for training the machine learning model and retraining the machine learning model using the different learning algorithms. Thus, instead of using different machine learning models, different training algorithms are used, for example, specifically selecting an algorithm better suited to better consider underrepresented subpopulations to train the machine learning model.
[0037] If the number of samples in one or more underrepresented subgroups is below a sample threshold, modifications to the machine learning model can, in particular, involve augmentation of the training data. Therefore, the sample threshold can be defined as the minimum number of (data) samples required in the underrepresented subgroups, such as time-series data samples. If this sample threshold is not met, augmentation of the training data can be chosen as one of the several examples of modifications to the machine learning model discussed in this paper to increase the amount of data and ultimately retrain the machine learning model with sufficient sample data for use in one or more underrepresented subgroups.
[0038] If the number of samples in one or more underrepresented subgroups exceeds a sample threshold, modifications to the machine learning model can include, in particular, the selection of a different machine learning model, the tuning of one or more hyperparameters, and / or the selection of a different learning algorithm. Specifically, if the sample threshold is met, one or more of the examples described above for modifying the machine learning model can be used.
[0039] Synthetic training data can be generated (especially but not limited to) using generative adversarial networks (GANs) and / or variational autoencoders (VAEs).
[0040] Identifying one or more underrepresented subgroups may include comparing the performance of each subgroup within a subgroup to a performance threshold and selecting subgroups with performance below the performance threshold as one or more underrepresented subgroups. Therefore, in addition to a sample threshold, or alternatively, a performance threshold may be used as a cutoff value to determine which of multiple subgroups are underrepresented and which are not underrepresented but, for example, adequately or excessively representative.
[0041] The method may further include: acquiring input from a factory operator by a machine learning model, the input indicating erroneous output from the machine learning model. Acquiring this input may trigger the determination of multiple subgroups in the training data, the determination of multiple performance parameters of the machine learning model, the determination of one or more underrepresented subgroups, and / or modifications to the machine learning model. Thus, a factory operator in the factory operating system may at least partially trigger this method, for example, when they become aware that they have been consistently receiving erroneous predictions or results from the machine learning model since, for example, last month. The method can then be specifically tailored to the factory operator's input (e.g., for a time range they specify) so that only data from last month, etc., is considered.
[0042] Additionally or alternatively, machine learning can be used in this method to monitor the agent. The agent can analyze outputs, such as predictions from the machine learning model. When the agent sees negative feedback or false positives, it can automatically trigger the identification of multiple subgroups in the training data, the determination of multiple performance parameters of the machine learning model, the identification of one or more underrepresented subgroups, and / or modifications to the machine learning model.
[0043] According to a second aspect of this disclosure, one or more computer program products are provided, which include instructions that, when executed by one or more computers, cause one or more computers to perform the methods of the first aspect of this disclosure.
[0044] (Multiple) computer program products can be one or more computer programs themselves, meaning computer programs consisting of program code executed by a computer.
[0045] Alternatively, the computer program product may be one or more products, such as data storage, particularly computer-readable data storage media, in which the computer program may be temporarily or permanently stored.
[0046] According to a third aspect of this disclosure, a data processing system is provided, which is configured to perform the method according to a first aspect of this disclosure.
[0047] This data processing system typically includes one or more computers.
[0048] According to a fourth aspect of this disclosure, a factory operating system for an industrial process plant is provided, the factory operating system including one or more computer program products of the second aspect of this disclosure or a data processing system of the third aspect of this disclosure.
[0049] Typically, a plant operating system for an industrial process plant can be implemented using hardware and software as explained herein. This plant operating system can perform the steps of the method of the first aspect of this disclosure using one or more computers. In addition to one or more computers containing computer program products and / or data processing systems, the plant operating system may also typically include plant operator or user input devices (e.g., keyboard, mouse, touchscreen, etc.), output devices (e.g., monitor, speaker, etc.), and / or similar devices as explained herein for interacting with the plant operator, such as outputting the results of machine learning model execution.
[0050] It should be noted that the above aspects, examples, and features can be combined with each other, regardless of the specific aspects involved.
[0051] The foregoing and other aspects of this disclosure will become apparent and be illustrated by reference to the examples described below. Attached Figure Description
[0052] Exemplary embodiments will now be described further with reference to the accompanying drawings:
[0053] Figure 1 This shows a perspective view of the plant operating system of an industrial process plant operated by human plant operators;
[0054] Figure 2 The illustration shows training data used to train a machine learning model that assists factory operators in operating industrial process plants.
[0055] Figure 3 It shows Figure 1 The flowchart shows the methods that the factory operating system will execute;
[0056] Figure 4 It shows the method used to determine Figure 2 A schematic diagram illustrating statistical techniques for subgroups in the training data;
[0057] Figure 5 The application is shown. Figure 3 A schematic diagram illustrating an exemplary process of the method.
[0058] The accompanying drawings are schematic only and are not drawn to scale. In principle, the same or similar parts, elements and / or steps in the drawings are provided with the same or similar drawing numbers. Detailed Implementation
[0059] Figure 1A plant operating system 10 for controlling an industrial process plant is shown in perspective. The plant operating system 10 is shown located in a control room, where two operators 100 are shown operating the plant operating system 10 to control the industrial plant during its operation.
[0060] In this example, the factory operating system 10 is shown to include multiple computers 11. Alternatively, the factory operating system 10 may include one computer 11 or multiple spatially distributed computers 11, including cloud computing devices, servers, edge devices, etc., which may be located inside or outside the space of an industrial plant.
[0061] In addition, the factory operating system 10 is shown to have an operator interface 14 (see Figure 2 The interface includes multiple output devices 12 and input devices 13. In this example, the output devices 12 are configured as a screen or display, and the input devices 13 are configured as a keyboard and mouse.
[0062] The factory operating system 10 can typically be connected to an industrial plant via one or more data connections. The industrial plant may include various technical equipment or devices, such as, but not limited to, sensors, actuators, etc., which can collect data, such as sensor measurements. This data is referred to herein as industrial plant data and reflects the operation of the industrial process plant in a given situation or current operation. The industrial plant data can be provided to the factory operating system 10, whereby it can be evaluated to control the industrial process plant and its technical equipment. Furthermore, the industrial plant data can also be at least partially indicated to the operator 100 via output device 12.
[0063] For example, there are situations or scenarios where industrial plants, particularly one or more pieces of technical equipment (such as sensors), return anomalous data (e.g., in the form of measured values). In such cases, there may be errors or anomalies. However, it is difficult to assess whether this is indeed the case.
[0064] For this purpose, or for any other purpose described herein, the factory operating system 10 may utilize machine learning model 1 (see Figure 5 This model can be implemented by one or more computers 11 to assist the factory operator 100 in factory operations, for example by determining or predicting whether abnormal measurements are errors or anomalies.
[0065] Figure 2 A schematic diagram of training data 2 in the form of points or a data cloud is shown. For example... Figure 2As can be seen, training data 2 is divided into multiple subgroups 3 in this paper, where each subgroup 3 of the training data is related to different scenarios of past operations of the industrial process plant. In particular, training data 2 is based on data collected from previous operations of the industrial process plant and can be collected further in the present time period and in the future. For example, such data, or training data in general, can include or take the form of time-series data, particularly from technical equipment such as sensors and their readings.
[0066] Therefore, this training data 2 (e.g., continuously) is used to train machine learning model 1. This is in Figure 3 The illustrated method is shown as step S2, which follows the exemplary step S1 of acquiring training data from the operation of an industrial process plant.
[0067] Based on previous operations, machine learning model 1 can be specifically trained to learn, for example, from previous abnormal measurements and whether they were errors or anomalies, so that when the situation occurs again and the measurements are abnormal, it can assist factory operator 100 in determining or predicting whether these values are abnormal or errors.
[0068] However, as Figure 2 As shown, there may be a subgroup 3 that is not adequately represented and is referred to herein as the underrepresented subgroup 4. Because these subgroups are underrepresented, the method may not be able to be adequately trained for these operational scenarios, and therefore, performance suffers when similar scenarios occur during operation and the plant operator 100 requests the machine learning model 1 trained on it to assist them (e.g., to identify whether outliers are errors or anomalies).
[0069] Therefore, in step S3, Figure 3 The method shown in the figure determines Figure 2 The training data illustrated in the example includes multiple subgroups 3. Then, in step S4, the performance of machine learning model 1 for each of the different subgroups 3 (or in other words, the different scenarios they belong to) is determined. In step S5, these performances can be compared to performance thresholds, wherein subgroups 3 associated with performance below the performance threshold are identified as underrepresented subgroups 4 among all subgroups 3 in the training data 2.
[0070] In other words, the performance of Model 1 can be examined on different clusters (or, in other words, subgroup 3). First, a performance threshold can be selected. For example, the performance threshold could be chosen as k times the acceptable performance, where the acceptable performance is given / known in advance, and k is a hyperparameter between (0, 1), which can be chosen based on the task. For example, if the task requires a low false positive rate, k can be high, such as 0.7. Otherwise, it can be low. Clusters with performance below this threshold can be classified as underrepresented.
[0071] Therefore, in step S6, the machine learning model 1 can be modified in a way that it considers one or more identified, underrepresented subgroups 4 in an incremental manner. For example, as... Figure 5 As shown, data augmentation techniques can be used to modify machine learning model 1 by adding the synthetic data described herein to training data 2 to retrain machine learning model 1.
[0072] exist Figure 4 The illustration shows a detailed statistical technique for processing time-series data (as an example of training data 2) to determine, for example... Figure 2 The training data 2 shown here contains multiple subgroups 3, or in other words, they are identified as clusters or groups of data related to the same scenario and / or similar operations in an industrial process plant.
[0073] Specifically, a two-step approach is used here to create subgroups or clusters: first, statistical clusters are identified, and then domain information is used to contextualize these statistical clusters, such as... Figure 5 The domain information data is shown in the figure.
[0074] Because time-series data are dependent, they can be used as follows: Figure 4 The standard-based method shown clusters subgroup 3. The first standard can depend on the availability of data and the model. If only data is available, a dip test for unimodality can be run on the data. If the data is unimodal, any window-based similarity technique can be used. If the data is multimodal, different patterns or scenarios can be found, where each pattern can be called a cluster.
[0075] If both the data and the model are available, you can proceed based on whether the data is univariate or multivariate. For univariate data, you can examine the mean variation and seasonality. Different seasons can be grouped into different clusters. After removing seasonality and the mean variation (i.e., making the data stationary), you can select different data windows and compare their correlation structures. If they are significantly different, they can be grouped into different clusters. If the data is multivariate, you can use any technique based on dimensionality reduction and clustering.
[0076] like Figure 5 As illustrated in the exemplary use case, plant operator 100 may use Model 1 for certain applications related to the operation of an industrial process plant, such as to identify whether a current value is abnormal or erroneous. Users and / or software monitoring agents may notice that the model performs poorly for such requests to Model 1. Plant operator 100 or the agent may provide input to plant operating system 10, essentially asking why this is the case, which serves as a trigger for executing the methods described herein.
[0077] Therefore, as well as Figure 5 As shown, training data can be searched for similar scenarios, meaning for subgroups 3 with similar operational scenarios. Figure 5 The diagram illustrates an example where an LLM is used to supply domain information data 5 to the LLM to determine subgroups 3. Domain information data 5 can indicate events in the operation of an industrial process plant, where multiple subgroups 3 in training data 2 are determined using an LLM based on training data 2 and domain information data 5. For this purpose, domain information data 5 can be fed into the LLM to identify subgroups 3 of the industrial process plant. Therefore, as also... Figure 5 As shown in the diagram, as described herein, it is possible to identify whether the subgroup 3 thus identified is underrepresented, or underrepresented but in sufficient numbers, and then any of the techniques described herein (including, for example) can be used. Figure 5 The data augmentation shown is used to modify Model 1 for retraining Model 1.
[0078] Typically, domain information data 5 can include metadata related to different raw materials, operating conditions, batches, categories, alarms, and events that may occur or be used in an industrial process plant. This information is usually available in unstructured formats (such as documents) or structured formats (such as databases / SAP / quality systems). Clusters or subgroups 3 can be contextualized by, for example, providing the documented and structured information of domain information data 5 to an LLM (especially an LLM-based generative artificial intelligence (AI) model) to obtain relevant timestamps for different events from these information sources. Once the timestamps are retrieved, the LLM or generative AI model can be asked to generate relevant Python code to extract data samples for those timestamps, or alternatively, to create a graph of data samples with relevant timestamped events, which can be referred to as a domain cluster. The extracted datasets with different events can be overlaid with statistics from the clusters obtained in the previous step. Graph generation can help to overlay information from previous clusters in a visual and analytical form.
[0079] Once all clusters or subgroups 3 are acquired, they can be used to visualize the different subgroups 3 and their events. One approach is to use timestamps from the clusters containing events and plot them as a graph, for example, as shown below. Figure 2 The line chart or scatter plot shown can be overlaid on the graph to correlate these points with cluster timestamps, allowing you to see data from underrepresented subgroup 3 or points where Model 1 performs poorly. Statistical clusters can also be overlaid on this graph for a deeper understanding of the reasons for poor model performance.
[0080] Typically, if subgroup 3 is not adequately represented, meaning the model performs poorly for that cluster, the number of samples can be examined. If the number of samples is high, it indicates that model 1 cannot effectively capture subgroup 3. In this case, different models / hyperparameters / strategies (such as ensemble learning) can be chosen to modify model 1. If the number of samples is low, GANs or VAEs can be used to augment the training data 2 with synthetic temporal data.
[0081] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions should be considered illustrative rather than restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and implemented by those skilled in the art and those who practice the invention after studying the drawings, this disclosure, and the claims.
[0082] As used herein, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The fact that certain measures are recited in mutually different dependent claims does not imply that combinations of these measures cannot be used advantageously. Furthermore, as used herein, the phrase “at least one” or similar phrases (e.g., “one or more”) referring to a list of one or more entities should be understood to mean at least one entity selected from any one or more entities in that list, but not necessarily including at least one of every entity specifically listed in that list, nor excluding any combination of entities in that list. This definition also allows for the optional presence of entities other than those specifically identified in the list of entities referred to by the phrase “at least one” or similar phrases, regardless of whether such entities are related to the specifically identified entities. Therefore, as a non-restrictive example, "at least one of A and B" (or equivalently "at least one of A or B", or equivalently "at least one of A and / or B", or equivalently "one or more of A and B", "one or more of A or B", or "one or more of A and / or B") in one example can refer to at least one (and optionally more than one) A, without B (and optionally other entities besides B); in another example, it can refer to at least one (and optionally more than one) B, without A (and optionally other entities besides A). In yet another example, it can refer to at least one (and optionally more than one) A, and at least one (and optionally more than one) B (and optionally other entities). In other words, the phrases "at least one", "one or more", and "and / or" are open-ended expressions that are operationally both conjunction and disjunctive. For example, each of the expressions “at least one of A, B and C”, “at least one of A, B or C”, “one or more of A, B and C”, “one or more of A, B or C”, and “A, B and / or C” can mean a single A, a single B, a single C, a combination of A and B, a combination of A and C, a combination of B and C, a combination of A, B and C, and optionally, any of the above combinations with at least one other entity.
[0083] As used herein, the phrase “indicate” can mean, for example, “reflect” and / or “include”. Therefore, the entities, elements, and / or steps referred to herein as “indicate [...]” can be used synonymously or interchangeably with one, two, or all of “include [...]” and “reflect [...]”. Furthermore, as used herein, unless otherwise stated, phrases such as “based on,” “related to,” “associated with,” or “associated with” should not be limited to the entities, elements, and / or steps they refer to. Rather, unless otherwise stated, these phrases should be understood to be inclusive; for example, when an entity, element, or step refers to another entity, element, or step by these phrases or similar phrases (e.g., “based on”), this does not preclude that the entity, element, or step may further or also be “based on” any other entity, element, or step besides those mentioned.
[0084] The designation of methods and steps as first, second, etc., herein is merely intended to make these methods and their steps referable and distinguishable from one another. Such designation of methods and steps does not constitute a limitation on the scope of this disclosure. For example, when this disclosure describes a third step of a method, the first or second step of the method need not exist, much less be performed before the third step, unless expressly stated that they are essential or must precede the third step. Furthermore, the presentation of methods or steps in a particular order is merely an example intended to facilitate this disclosure and does not constitute a limitation on the scope of this disclosure. Generally, methods and steps may be performed in any feasible order unless expressly stated otherwise. In particular, the terms “first,” “second,” “third,” or “(a),” “(b),” “(c)” as used in the specification and claims are used to distinguish similar elements and are not necessarily used to describe an order or chronological order. It should be understood that these terms may be used interchangeably where appropriate, and embodiments of the invention described herein may operate in an order other than that described or illustrated herein.
[0085] No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A computer-implemented method, comprising: Acquire training data from operations at industrial process plants (2); The training data (2) is used to train a machine learning model (1) to assist the operation of the industrial process plant. Multiple subgroups (3) are identified in the training data (2), each of the subgroups (3) being associated with different scenarios of operation of the industrial process plant; Determine multiple performances of the machine learning model (1), each of which is associated with a different subgroup (3); Based on the performance of each subgroup in the subgroup (3), one or more underrepresented subgroups (4) are identified, which are underrepresented in the training data (2) compared with the other subgroups (3) in the plurality of subgroups (3); as well as Based on the one or more underrepresented subgroups (4), the machine learning model (1) is modified to incorporate consideration of the one or more underrepresented subgroups (3) when assisting the operation of the industrial process plant.
2. The method of claim 1, wherein the training data (2) comprises time-series process data indicating sensor readings from sensors in the industrial process plant.
3. The method according to claim 2, wherein the subgroups (3) in the time series process data are identified by applying one or more statistical techniques to the time series process data.
4. The method according to claim 2 or 3, wherein the training data (2) further comprises domain information data (5), the domain information data (5) indicating events in the operation of the industrial process plant, and / or the method comprises: Acquire domain information data (5), wherein the plurality of subgroups (3) in the training data (2) are determined based on the training data (2) and the domain information data (5).
5. The method according to claim 4, wherein the domain information data (5) is fed into one or more large language models (LLMs) for identifying subgroups (3) of the industrial process plant.
6. The method of claim 5, wherein the plurality of subgroups (3) in the training data (2) are determined using the machine learning model (1) or another model, the machine learning model (1) or the other model superimposing information from the identified subgroups (3) determined based on the one or more statistical techniques and based on the one or more LLMs.
7. The method according to any one of the preceding claims, wherein the modification of the machine learning model (1) comprises: The training data (2) is augmented using synthetic training data for the one or more underrepresented subgroups (4), and the machine learning model (1) is retrained using the augmented training data (2). Different machine learning models are selected to train the different machine learning models using the training data (2) to assist the operation of the industrial process plant. Adjust one or more hyperparameters of the machine learning model (1), and retrain the machine learning model (1) using the adjusted one or more hyperparameters with the training data (2); and / or Different learning algorithms are selected for training the machine learning model (1) and the different learning algorithms are used to retrain the machine learning model (1).
8. The method of claim 7, wherein if the number of samples in the one or more underrepresented subgroups (3) is less than a sample threshold, the modification to the machine learning model (1) comprises: The enhancement of the training data (2).
9. The method of claim 8, wherein if the number of samples in the one or more underrepresented subgroups (4) is higher than a sample threshold, the modification to the machine learning model (1) comprises: The selection of different machine learning models, the tuning of one or more hyperparameters, and / or the selection of different learning algorithms.
10. The method according to any one of claims 7 to 9, wherein the synthetic training data is generated using a generative adversarial network (GAN) and / or a variational autoencoder (VAE).
11. The method according to any one of the preceding claims, wherein the determination of the one or more insufficiently represented subgroups (4) comprises: The performance of each subgroup in the subgroup (3) is compared with a performance threshold, and the subgroup (3) whose performance is lower than the performance threshold is selected as the one or more underrepresented subgroups (4).
12. The method according to any one of the preceding claims, wherein the method further comprises: The machine learning model (1) obtains input from the factory operator (100), the input indicating the error output from the machine learning model (1); The acquisition of the input triggers the determination of the plurality of subgroups (3) in the training data (2), the determination of the plurality of performances of the machine learning model (1), the determination of the one or more underrepresented subgroups (4), and / or the modification of the machine learning model (1).
13. One or more computer program products, including instructions that, when executed by one or more computers (11), cause the one or more computers (11) to perform the method of any one of the preceding claims.
14. A data processing system configured to perform the method according to any one of claims 1 to 12.
15. A factory operating system (10) for an industrial process plant, the factory operating system (10) comprising one or more computer program products according to claim 13 or a data processing system according to claim 14.