Method for checking the degree of realism of synthetic training data for a machine learning model

Abstract

The invention relates to a method (100) for checking a degree of realism of synthetic training data for a machine learning model, comprising the following steps: - providing (101) the synthetic training data, wherein the synthetic training data are described by a statistical variable, wherein the synthetic training data simulate sensor data, - determining (102) an upper limit of a confidence interval for the statistical variable based on the synthetic training data as part of training the machine learning model, - providing (103) real data, wherein the real data are also described by the statistical variable, wherein the real data comprise sensor data, wherein the sensor data result from a detection of at least one sensor, - determining (104) a lower limit of the confidence interval for the statistical variable based on the real data as part of an inference of the machine learning model,wherein the lower limit is determined continuously from the beginning of the inference, - checking (105) the degree of realism of the synthetic training data based on a comparison of the continuously determined lower limit with the determined upper limit, wherein a systematic deviation of the synthetic training data from the real data is detected. Furthermore, the invention relates to a computer program, a device and a storage medium for this purpose.

Claims

[1] A method (100) for checking a degree of realism of synthetic training data for a machine learning model, comprising the following steps: - providing (101) the synthetic training data, wherein the synthetic training data are described by a statistical quantity, wherein the synthetic training data simulate sensor data, - determining (102) an upper limit of a confidence interval for the statistical value on the basis of the synthetic training data as part of training the machine learning model, - providing (103) real data, wherein the real data are also described by the statistical quantity, wherein the real data comprise sensor data, wherein the sensor data result from a detection of at least one sensor, - determining (104) a lower limit of the confidence interval for the statistical value on the basis of the real data as part of an inference of the machine learning model, wherein the lower limit is determined continuously from the beginning of the inference, - checking (105) the degree of realism of the synthetic training data on the basis of a comparison of the continuously determined lower limit with the determined upper limit, wherein a systematic deviation of the synthetic training data from the real data is detected. [2] Method (100) according to claim 1, characterized by that an order is specified in the real data in order to determine the lower limit of the confidence interval with the real data with the specified order. [3] Method (100) according to one of the preceding claims, characterized by that the method further comprises the following steps: - Performing the inference based on a portion of the synthetic training data, continuously determining the lower bound, - Performing the inference based on the real data, continuing to determine the lower limit, - Checking the synthetic training data based on an analysis of an increase in the continuously determined lower limit when transitioning from the part of the synthetic training data to the real data. [4] Method (100) according to one of the preceding claims, characterized by that when checking the synthetic training data, the systematic deviation of the synthetic data from the real data exists if the continuously determined lower limit exceeds the determined upper limit. [5] Method (100) according to one of the preceding claims, characterized by that the method further comprises the following step: - performing an action depending on a result of checking the synthetic training data, wherein the action comprises at least initiating an output of a warning message. [6] Method (100) according to one of the preceding claims, characterized by that the method further comprises the following step: - Define a threshold for a false alarm rate. [7] Method (100) according to one of the preceding claims, characterized by that the sensor data include measurement data, in particular image data, of a production process and the statistical value represents an error of a respective component. [8] Computer program (20) comprising instructions which, when the computer program (20) is executed by a computer (10), cause the computer (10) to carry out the method (100) according to one of the preceding claims. [9] Device (10) for data processing, which is arranged to carry out the method (100) according to one of claims 1 to 7. [10] A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to carry out the steps of the method (100) according to any one of claims 1 to 7.

Images