Method for providing training data for a machine learning (ML) model to predict the behavior of a technical system

Abstract

The present invention comprises a method (100) for providing training data (10) for a machine learning (ML) model (30) for predicting the behavior of a technical system (50), comprising the following steps: - Generating (102) a first data set (12), wherein this first data set (12) contains simulation data (13) for the technical system (50), - Generating (104) a second data set (14), wherein this second data set (14) contains sample data (16) of the technical system (50), - Generating (106) a third data set (18) by combining the first data set (12) with the second data set (14), - Training (107) the ML model (30) based on the third data set (18), - Generating (110) a fourth data set (20) as first input data (41) based on the third data set (18) by maximizing an information function (42) to obtain a first feature combination (45) as input data (43) for the technical system (50), - Measuring (112) the first feature combination (45) for the sample data (16) of the technical system (50) to obtain a fifth data set (22); and - Adding (114) the fifth data set (22) as output data (46) and the generated fourth data set (20) as first input data (41) to the third data set (18) as training data for the ML model (30).

Description

[0001] The present invention relates to a method for providing training data for a machine learning (ML) model for predicting the behavior of a technical system. State of the art

[0002] Active learning is a well-established approach for efficiently training machine learning (ML) models, such as neural networks, using provided data. US 2023 177 118 A1 describes an example of such an approach for training an ML model.

[0003] The published article by Cen-You Li et al., “Safe Active Learning for Multi-Output Gaussian Processes”, Bosch Center for Artificial Intelligence, Robert-Bosch-Campus 1, 71272 Renningen, Germany, also describes in detail the approach of active learning for multi-output processes.

[0004] For training a ML model based on the known active learning approaches described at the beginning, simulation data is currently used in the prototype phase of a product development project. This data is easy to generate without having to make large investments (e.g., prototype parts).

[0005] This simulation data serves to support product development, but is not always accurate enough to replace, for example, a subsequent end-of-line test of the finished product. More precise machine learning models are generally required for this, but their training requires a very large amount of data to bring these models to the required level of accuracy or quality.

[0006] A major problem is that during the prototype phase of product development, there is insufficient data to train a suitable machine learning model, and generating additional data is very expensive.

[0007] The object of the present invention is therefore to provide a solution by which efficient and cost-effective training data for a machine learning (ML) model can be generated in order to efficiently predict the behavior of a technical system. Disclosure of the invention

[0008] This task is solved by a method for providing training data for a machine learning (ML) model to predict the behavior of a technical system with the features of an independent claim.

[0009] According to one aspect, the disclosure relates to a method for providing training data for a machine learning (ML) model to predict the behavior of a technical system.

[0010] The first step involves generating an initial data set, which contains simulation data for the technical system.

[0011] In a second step, a second data set is generated, whereby this second data set contains sample data of the technical system.

[0012] In a third step, a third data set is generated by combining the first data set with the second data set.

[0013] In a fourth step, the ML model is trained based on the third data set.

[0014] In a fifth step, a fourth data set is generated as the first input data based on the third data set by maximizing an information function in order to obtain a first combination of features as input data for the technical system.

[0015] In a sixth step, the first combination of features for the sample data of the technical system is measured in order to obtain a fifth data set.

[0016] In a seventh step, the fifth dataset is added as output data, and the generated fourth dataset is added as the first input data to the third dataset. This third dataset is the training data for the machine learning model, which is provided as part of the process.

[0017] Optionally, an eighth step can involve repeating steps four to seven.

[0018] A fundamental concept of the present invention is that, based on the active learning approach, information or data from the sample phase are correlated with data from the pattern phase. In this way, less data from the pattern phase is required to generate the necessary training data for training a machine learning model to a defined level or to achieve a defined model performance level.

[0019] Furthermore, the present invention is based on the following essential aspects.

[0020] As a starting point for generating training data to train a machine learning model, datasets or measurements from the sample phase are used. Furthermore, one or more test systems from the prototype phase are available, but with few or no corresponding measurements or datasets.

[0021] In accordance with the present invention, an initial multi-output model, e.g., a multi-output Gaussian process, is trained which, based on input data x, predicts the outputs y_1 of the technical system from the sample phase and y_2 from the subsequent phase (production phase). For a given x, a measurement y_1, y_2, or both may be available.

[0022] One possible implementation of the process involves checking, after the fourth step of training the machine learning (ML) model, whether a termination criterion has already been met. If this criterion is met, a further step is performed to terminate the process. This allows for efficient training of the ML model and reduces the data processing effort required to provide the necessary training data, or allows it to be flexibly adapted to the required performance level of the ML model being trained.

[0023] One possible embodiment of the method provides that the defined termination criterion includes at least one of the following criteria for whether the ML model is trained further: maximum number of measurements performed, a defined model quality of the ML model, number of iterations performed to improve the ML model, or a predetermined training time t. This achieves the advantage that, depending on the requirements for the ML model to be trained, the method according to the invention can be carried out efficiently and cost-effectively.

[0024] One possible embodiment of the method involves performing the testing step using a validation dataset. This validation dataset generates verifiable performance information for the trained ML model, indicating how accurately the trained ML model represents the validation dataset. This offers the advantage that, depending on the requirements for the ML model to be trained, the method according to the invention can be carried out efficiently and cost-effectively.

[0025] One possible embodiment of the method provides that the performance information of the ML model indicates an error value that the trained ML model makes when applying the validation data set. If this output error value of the ML model is below a defined error tolerance limit, the training of the ML model is terminated. This achieves the advantage that, depending on the application-specific requirements for the ML model to be trained, the method according to the invention can be carried out efficiently and cost-effectively.

[0026] According to a second aspect, the disclosure relates to a computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer(s) or computer instances to execute the method according to the invention.

[0027] According to a third aspect, the disclosure concerns a machine-readable data carrier and / or download product containing the computer program.

[0028] According to a fourth aspect, the disclosure relates to one or more computers and / or computer instances with the computer program, and / or with the machine-readable data carrier and / or the download product.

[0029] Further measures improving the invention are described in more detail below, together with a description of preferred embodiments of the invention, with reference to figures. Examples of implementation

[0030] It shows: Fig. 1 Schematic flowchart of the procedure 100 to provide training data 10 for a machine learning (ML) model 30 to predict the behavior of a technical system 50.

[0031] The Fig. Figure 1 shows a schematic flowchart of the procedure 100 for providing training data 10 for a machine learning (ML) model 30 to predict the behavior of a technical system 50. A technical system 50 could, for example, be an injector, and the behavior would be the injection quantity. It could also be an EAC (= Electrical Air Compressor), and the behavior to be modeled would, for example, be a pressure ratio in the compressor / turbine section.

[0032] The training data 10 can be derived from data sets 12+14 or from data sets 12+14+22 with a corresponding number of additional iterations.

[0033] In step 102, a first data set 12 is generated, whereby this first data set 12 contains simulation data 13 for the technical system 50.

[0034] In step 104, a second data set 14 is generated, wherein this second data set 14 contains sample data 16 of the technical system 50.

[0035] In step 106, a third data record 18 is generated by combining the first data record 12 with the second data record 14.

[0036] In step 107, the ML model 30 is trained based on the third data set 18.

[0037] In step 110, a fourth data set 20 is generated as first input data 41 based on the third data set 18 by maximizing an information function 42 in order to obtain a first feature combination 45 as input data 43 for the technical system 50.

[0038] A possible example of an information function 42 could be a so-called "uncertainty" function. According to one aspect of the present invention, several possible combinations of input parameters (e.g., 3 possibilities) are presented to the model. The model then makes a prediction for each of these 3 combinations and outputs the information on the uncertainty prevailing for each of these combinations. The combination with the highest uncertainty is selected and then physically measured and added to the training set.

[0039] The following example of a parameter combination, pressure and actuation duration, is given using an injector: • Combination 1: P=1000 bar, AD=1000 µs. Model prediction: Injection quantity=200 mm³ 3 / E Uncertainty: ±0.3mm 3 / E • Combination 2: P=1500 bar, AD=1200 µs. Model prediction: Injection quantity=250 mm³ 3 / E Uncertainty: ±0.5mm 3 / E • Combination 3: P=2000 bar, AD=800 µs Model prediction: Injection quantity=270 mm 3 / E Uncertainty: ±1.3mm 3 / E. This combination is being tested (e.g., measured injection quantity = 272.4 mm³). 3 / E) and the result is added to the training data.

[0040] Alternatively, the uncertainty can also be expressed as an absolute value.

[0041] In step 112, a measurement 112 of the first feature combination 45 is taken for the sample data 16 of the technical system 50 in order to obtain a fifth data set 22.

[0042] In step 114, the fifth data record 22 is added as output data 46 and the generated fourth data record 20 is added as the first input data 41 to the third data record 18. After step 114, it is possible to return to step 107.

[0043] Step 116 involves repeating steps 107 to 112. This step 116 is optional.

[0044] As further in the Fig. As shown in Figure 1, optionally after step 107 of training the ML model 30, a check 108 can be performed to determine whether a termination criterion 40 has already been met. If this termination criterion 40 is met, step 118 is performed, which terminates or ends the procedure 100.

[0045] The defined termination criterion 40 can include at least one of the following criteria to decide whether the current ML model 30 is to be trained further: maximum number of measurements performed, a defined model quality of the ML model 30, number of iterations performed to improve the ML model 30, a predetermined time period t for training the ML model 30. Further criteria would be conceivable and are not limited to the examples mentioned.

[0046] The verification step 108 can preferably and optionally be performed using a validation data set 48. By applying or using this validation data set 48, verifiable quality information 49 is generated for the trained ML model 30. This information indicates how accurately the trained ML model 30 represents the validation data set 48. Specifically, it indicates whether the outputs of the trained ML model 30 generated by the validation data set 48 are identical to the outputs generated by a simulation or measured outputs of this validation data set 48, or whether there are deviations. These deviations (actual state of the ML model 30) from a defined target state can preferably be treated as error information.

[0047] The performance information of the ML model 30 indicates an error value of the ML model 30 that the trained ML model 30 makes or generates when applying the validation data set 48. If this detected error value of the ML model 30 is below a defined error tolerance limit, the training of the ML model 30 or the further generation of training data 10 for the ML model 30 is terminated.

[0048] The corresponding sequence of the inventive method 100 can also be mathematically described in simplified form as an algorithm as follows: For i=1:N 1) Select a new x_i = argmax Info(x) based on an information measure for the multi-output GP. 2) Measure the corresponding y_2 (e.g., with serial parts). 3) Extend the data set by (x,y_2). 4) Update the multi-output model. END

[0049] Under 1), which corresponds to process step 110, the maximization of an information function is to be understood as the information measure.

[0050] The 2) corresponds to process step 112.

[0051] The 3) corresponds to process step 114.

[0052] Step 4) corresponds to process step 107, except during the first iteration of step 107.

[0053] The multi-output GP model can correlate data from sample phases y_1 and y_2 and thereby transfer the knowledge gained from the sample phase. This advantageously allows for the correction of simulation results available for the entire characteristic curve of a product and the achievement of good accuracy in the machine learning (ML) model being trained, without requiring a large amount of data for training the ML model. Consequently, training the ML model to a defined level of accuracy based on the training data generated by the inventive method can be carried out cost-effectively and efficiently. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2023 177 118 A1

[0002]

Claims

[1] Method (100) for providing training data (10) for a machine learning (ML) model (30) for predicting a behavior of a technical system (50), comprising the following steps: - generating (102) a first data set (12), wherein this first data set (12) contains simulation data (13) for the technical system (50), - generating (104) a second data set (14), wherein this second data set (14) contains sample data (16) of the technical system (50), - generating (106) a third data set (18) by combining the first data set (12) with the second data set (14), - Training (107) the ML model (30) based on the third data set (18), - generating (110) a fourth data set (20) as first input data (41) based on the third data set (18) by maximizing an information function (42) in order to obtain a first feature combination (45) as input data (43) for the technical system (50), - measuring (112) the first feature combination (45) for the sample data (16) of the technical system (50) to obtain a fifth data set (22); and - adding (114) the fifth data set (22) as output data (46) and the generated fourth data set (20) as first input data (41) to the third data set (18) as training data (10) for the ML model (30). [2] The method (100) of claim 1, wherein steps (107) to (112) are repeated (116). [3] Method (100) according to one of claims 1 to 2, wherein after the step (107) of training the ML model (30), a check (108) is carried out as to whether a termination criterion (40) is already met, and if this termination criterion (40) is met, a step (118) is carried out which terminates the method (100). [4] Method (100) according to claim 3, wherein the defined termination criterion (40) comprises at least one of the following criteria as to whether the ML model (30) is further trained: maximum number of measurements performed, a defined model quality of the ML model (30), number of iterations performed to improve the ML model (30), a predetermined time period t for training the ML model (30). [5] Method (100) according to claim 3 or 4, wherein the step of checking (108) is carried out using a validation data set (48), wherein the validation data set (48) is used to generate verifiable quality information (49) for the trained ML model (30), which indicates how accurately the trained ML model (30) maps the validation data set (48). [6] Method (100) according to claim 5, wherein the quality information of the ML model (30) indicates an error value of the ML model (30) that the trained ML model (30) makes when applying the validation data set (48), and if this output error value of the ML model (30) is below a defined error tolerance limit, the training of the ML model (30) is aborted. [7] A computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer or computer instances to carry out the method according to any one of claims 1 to 6. [8] Machine-readable data carrier and / or download product with the computer program according to claim 7. [9] One or more computers and / or computer instances with the computer program according to claim 7 and / or with the machine-readable data carrier and / or download product according to claim 8.

Images