Evaluating results of a computer-based machine learning system

Abstract

One aspect of the present disclosure relates to a computer-implemented method for evaluating results of a computer-based machine learning system. The method comprises receiving a dataset of real-world elements, wherein each real-world element of the dataset of real-world elements corresponds to a real-world observation. The method further comprises receiving a first result calculated by the computer-based machine learning system using the dataset of real-world elements. In the next step, the method of the first aspect comprises generating a dataset of synthetic elements with respect to the dataset of real-world elements. Furthermore, the method comprises transmitting the dataset of synthetic elements.The method further comprises receiving a second result calculated by the computer-based machine learning system using the data set of synthetic elements. To this end, the method of the first aspect comprises comparing the first result with the second result. Finally, the method comprises determining, based on the first result and / or the comparison of the first result with the second result, one or more validity values that characterize a confidence measure of the calculated first result.

Description

Technical area

[0001] The present invention relates to techniques for evaluating results of a computer-based machine learning system. Related aspects include a computer program, a monitoring module, an apparatus, and a distributed system. background

[0002] Computer-based machine learning systems are increasingly being used in various technical devices. In many cases, computer-based machine learning systems (e.g., those using artificial neural networks) must be extensively trained and / or validated using known data sets in order to provide plausible output data (i.e., unknown output data) for any given input data of a device. However, the use of such models in safety-critical applications (e.g., in the context of autonomous or assisted driving) is limited because if the machine learning system operates on data sets other than those on which it was trained and / or validated, some of the existing state-of-the-art methods can produce unexpected results (e.g., false positives and / or false negatives). In some cases, e.g.,In image processing, an artificial neural network can misclassify an originally correctly classified object in an image if the same image is mixed with noise (even if such noise would not be detectable by the human eye).

[0003] Therefore, some existing state-of-the-art computer-based machine learning systems generate uncertainty values ​​(directly or using a separate probabilistic model) along with the output data, quantifying the extent to which the computed output data is reliable. However, these uncertainty values, provided by state-of-the-art computer-based (e.g., in-vehicle) machine learning systems, which can be generated in real time in a moving vehicle, do not refer to a current environment / scene and, what may be even more problematic, to its temporal evolution. Furthermore, some state-of-the-art methods do not consider how the uncertainty values ​​change over time. This can lead to an overestimation of the reliability of the results generated by the computer-based machine learning systems, which may be unacceptable for safety-critical applications.

[0004] Therefore, there is a need to develop new techniques for computer-based machine learning systems that can solve some or all of the above-mentioned problems. Summary of the invention

[0005] A first general aspect of the present disclosure relates to a computer-implemented method for evaluating results of a computer-based machine learning system. The method comprises receiving a dataset of real-world elements, wherein each real-world element of the dataset of real-world elements corresponds to a real-world observation. The method further comprises receiving a first result calculated by the computer-based machine learning system using the dataset of real-world elements. In the next step, the method of the first aspect comprises generating a dataset of synthetic elements with respect to the dataset of real-world elements. Furthermore, the method comprises transmitting the dataset of synthetic elements.The method further comprises receiving a second result calculated by the computer-based machine learning system using the data set of synthetic elements. To this end, the method of the first aspect comprises comparing the first result with the second result. Finally, the method comprises determining, based on the first result and / or the comparison of the first result with the second result, one or more validity values ​​that characterize a confidence measure of the calculated first result.

[0006] A second general aspect of the present disclosure relates to a computer program designed to carry out the method according to the first general aspect of the present disclosure.

[0007] A third general aspect of the present disclosure relates to a monitoring module configured to execute the computer-implemented method according to the first general aspect and / or to execute the computer program according to the second general aspect.

[0008] A fourth general aspect of the present disclosure relates to an apparatus comprising a monitoring module according to the third aspect. Furthermore, the apparatus comprises a module comprising a computer-based machine learning system, wherein the module is configured to receive a data set of real-world elements received from the monitoring module of the third aspect. Furthermore, the module is configured to calculate the first result by the computer-based machine learning system using the data set of real-world elements and to send the first result to the monitoring module.

[0009] A fifth general aspect of the present disclosure relates to a distributed system comprising a device according to the fourth aspect and an add-on module comprising a computer-based machine learning system. The add-on module is configured to receive a data set of synthetic elements sent from the monitoring module according to the third aspect. Furthermore, the add-on module is configured to generate a second result by the computer-based machine learning system using the data set of synthetic elements and to send the second result to the monitoring module.

[0010] The techniques of the first to sixth general aspects may have one or more of the following benefits.

[0011] First, the present techniques can enable efficient determination of a confidence measure for the results generated in real time by a computer-based machine learning system with respect to real-world elements associated with real-world observations (e.g., when these real-world elements, such as captured image data, are processed in real time by the computer-based machine learning system). In particular, based on these generated results, which in some cases, together with the output data of the computer-based machine learning system, include uncertainty values, it is possible to conclude in real time whether the output data is trustworthy enough (or not) for a particular application.

[0012] Second, the present techniques offer the possibility of generating synthetic elements for the real elements analyzed at a current time, which can emulate the temporal evolution of the current scene / environment (including, for example, synthetic elements that emulate future, not yet captured real elements), and which are also processed by the computer-based machine learning system. It is possible to compare the results generated by the computer-based machine learning system in relation to the synthetic and real elements to further specify a confidence measure of the results generated with respect to the real-world elements (e.g., in real time).This makes it possible with the present techniques to assess the extent to which the generated results are trustworthy with respect to the real elements, using relatively manageable hardware resources and more accurately than with some state-of-the-art techniques.

[0013] Some terms are used in this disclosure as follows: The term “computer-based machine learning system” in the present disclosure refers to any device that can be or has been trained for one or more tasks using machine learning. During training, training data sets can be made available to the computer-based machine learning system as input data, and the properties (e.g. corresponding parameters) of the computer-based machine learning system can be adapted in response to the processing of the training data sets (e.g. by analyzing the output data) in order to solve the one or more tasks in a defined manner (e.g. with a specific accuracy). A computer-based machine learning system can contain a model that can be parameterized. The adaptation of the properties of the machine learning system during learning can, for example, be carried out by an optimization method with respect to (unknown) parameters of the machine learning system (e.g.of a corresponding model), which can be represented as the minimization of a loss function (within a predetermined numerical accuracy and / or until a predetermined stopping criterion is reached). The adapted computational machine learning system can then be used to predict the responses for the observations in another dataset, called the validation dataset. Thus, the validation dataset can provide an evaluation of the machine learning system's fit to the training datasets. Finally, in some cases, test datasets can be used, which are typically used to evaluate the final machine learning system selected during the validation process.

[0014] In some cases, "a computer-based machine learning system" may perform a classification task or a regression task. As a non-limiting example, the computer-based machine learning system (e.g., its model) may comprise an artificial neural network having a specific topology and a number of neurons with corresponding connections. According to some embodiments, the neural network may be a convolutional neural network (CNN), which is defined, for example, by the number of filters, filter sizes, step sizes, etc. A convolutional neural network may, for example, be used for the purpose of image classification and perform one or more transformations on digital images based, for example, on convolution, nonlinearity (ReLU), pooling, or classification operations (e.g., using fully connected layers).The neural network can also be designed as a multi-layer feedforward or recurrent network, as a neural network with direct or indirect feedback, or as a multi-layer perceptron. Machine learning systems based on neural networks can be used in a vehicle computer or another vehicle component, or in an at least partially autonomous robot (e.g., to evaluate an operating state of the vehicle or robot and / or to control a function of the vehicle or robot based on state and / or environmental data of the vehicle or robot as input data). For example, computer-based machine learning systems can be implemented in any suitable form, i.e., in software, in dedicated hardware, or in a hybrid of software and dedicated hardware.Therefore, computer-based machine learning systems can be a software module (also integrated into a higher-level software system) that can be executed on a general-purpose processor. In other cases, a computer-based machine learning system can be implemented (at least partially) in circuitry.

[0015] For the purposes of this disclosure, the term "real-world item dataset" may refer to a collection of real-world items (or, in some cases, a real-world item) associated with real-world observations (in other words, real-world observations, e.g., measurements). Depending on the nature of the task for which a computer-based machine learning system is used, the items of a "real-world item dataset" may contain different information. For example, each real-world item of a "real-world item dataset" may contain a single image (or a portion thereof). For example, an image item may contain a number of image pixels (e.g., 1024x2048 pixels), where each image pixel has a number of color values ​​(e.g., three color values ​​with 16-bit color depth). In other examples, items of a "real-world item dataset" may contain data series (e.g., time series).The image data and / or data series can be generated using various sensors (e.g., cameras, radar, lidar, ultrasonic, GPS, or thermal sensors, sensors for a vehicle's engine control). In some examples, a vehicle sensor can be used in the context of semi-autonomous, autonomous, or assisted driving of a vehicle to provide functionality for the vehicle (e.g., with regard to safety functions, the implementation of driver information, and comfort functions such as intelligent headlight control and traffic sign information).

[0016] A "feature" of an element of a "data set of real elements" in the present disclosure may refer to a part of the element that is recognizable in this element (e.g., by appropriate data processing) and that can be characterized by a specific property or assigned to certain categories (e.g., by means of an image classifier as a computer-based machine learning system). If an element is an image, a feature may refer to a spatially limited part of a scene depicted in an image and / or a plurality of pixels that correspond to a corresponding part of an image. In the example of a vehicle-integrated sensor system, a "feature" may be an object in the images in connection with road scenes recorded thereon (in other words: captured) (e.g., in real time while a vehicle is moving). A non-exhaustive list of such objects (i.e.,Features) can include traffic signs, lanes or other road or pedestrian zone markings, trees, buildings, road users such as pedestrians, cyclists or other vehicles. Within the scope of the present disclosure, it is also conceivable that a “feature” can be a part of an object: in some cases where a vehicle is recognized in an image, a “vehicle color” and “vehicle tires” are examples of such features. Furthermore, “features” such as “weather” or “time of day” can also be extracted from an image. An “element” that contains a (measured) data series (e.g. time series) from a sensor can also contain corresponding “features”. For example, a steering wheel angle while driving as a function of time or injection times in a vehicle’s engine over the pressure curve are two non-exhaustive examples of such features. In some cases, features can be extracted from the element’s data series (e.g.calculated): For example, the minimum and / or maximum of a time series or the maximum and / or minimum derivative (ie the slope) of a time series or any combination thereof can be calculated and assigned to a corresponding feature.

[0017] Accordingly, the term "dataset of synthetic elements" can refer to a collection of synthetic elements (or, in some cases, a synthetic element) that, unlike the elements of a "dataset of real elements," are not linked to real observations but have an artificial origin. For example, a "synthetic element" is not a captured image and therefore does not require any data processing, such as image processing. A synthetic element can be generated from one or more real elements (e.g., captured images) that are varied (in other words, perturbed) according to their nature to generate this synthetic element.The synthetic element may be similar to the original real elements (or real element) from which it was created after their variation, in the sense that it also contains one or more 'features' present on those original real elements (or real element). In the example of captured images (e.g., by a camera), this variation may include one or more variations from the following, non-exhaustive list: i) pixel-level variation to model sensor noise; ii) optical flow-based variation to model object motion at the pixel level; iii) variation based on object motion models; iv) variation using synthetically rendered images or other variations.

[0018] Within the scope of the present disclosure, it is conceivable that a "real" or "synthetic element" is fed as input data into a computer-based machine learning system, which can provide a corresponding "result." This "result" can be a processing result for this element by the machine learning system, which is generated by the machine learning system in the form of output data. Furthermore, the "result" can include an "uncertainty value" with respect to the generated output data, which can quantify the extent to which this output data is trustworthy (more on this below). As already mentioned above, a "computer-based machine learning system" can be a classifier, which can be configured to receive "elements" as input data and to classify "features" of these elements into corresponding classes.In some cases, a classifier can provide probabilities with which a feature was classified into different classes (e.g., after detecting these features on the elements). In some examples, a classifier can be an image classifier that semantically segments image data pixel by pixel or region by region. The image classifier can be configured to receive input data (e.g., an "element") in the form of image data and classify it into multiple classes. In some examples, this can involve mapping input data in the form of an input vector of one dimension (R). n ), which contains image data, to output data in the form of an output vector of a second dimension (R m) representing a classification result. For example, components of the input vector may represent image data associated with an image. Each component of the output vector may represent an image classification result that may be computed using the computer-based machine learning system of the present disclosure.

[0019] The term "vehicle" encompasses any device designed to transport passengers and / or cargo. The vehicle may be a motor vehicle (for example, an at least partially autonomously operating / assisted motor vehicle, in particular a car or a truck). However, the vehicle may also be a ship, a train, an aircraft, or a spacecraft. Other devices are also conceivable within the scope of the present disclosure; more on this below. Short description of the characters Fig. 1a is a flowchart illustrating an example of a method for evaluating results of a computer-based machine learning system according to the first aspect. Fig. 1b and Fig. 1c are flow diagrams showing further possible method steps according to the first aspect. Fig. 2a and Fig. Figures 2b schematically show examples of real elements 1; 2 from the dataset of real elements, representing two captured images at different times, in which six features, namely "vehicles 10, 14," "pedestrians 11, 13," and a "cyclist 15," are visible. These figures also show examples when parts of these images (areas in dashed circles) are considered real elements 1a; 2a with one feature, the "vehicle 10." Fig. Figure 3 schematically shows an exemplary monitoring module 20 communicating with the following modules: i) a module 30 comprising a computer-based machine learning system 70; ii) an additional module 40 comprising the same computer-based machine learning system 70; and iii) a planner 50. The incoming and outgoing arrows, respectively, illustrate exemplary communication between the modules. 60: a dataset of real elements; 61 / 62: output data / uncertainty values ​​for real elements ("first result"); 63: generated dataset of synthetic elements; 64 / 65: output data / uncertainty values ​​for synthetic elements ("second result"). Detailed description

[0020] First, based on Fig. 1a to 1c describe techniques for evaluating the results of a computer-based machine learning system. Then, exemplary “real elements” of the present techniques with their “features” are described using Fig. 2a and Fig. 2b. Finally, with reference to Fig. 3 describes an exemplary monitoring module 20 for an implementation of the computer-implemented method of the first aspect.

[0021] As in the Fig. 1a to 1c, a first general aspect relates to a computer-implemented method for evaluating results of a computer-based machine learning system. As already described above, the computer-based machine learning system may, for example, comprise an artificial neural network (e.g., a CNN). The method steps of the corresponding independent claim are shown in the boxes drawn by solid lines in Fig. 1a to 1c, while the method steps of some dependent claims are shown in the boxes represented by dashed lines.

[0022] The first step of the method comprises receiving 100 a data set 60 of real elements, wherein each real element 1; 1a; 2; 2a of the data set of real elements corresponds to a real observation (in the sense defined above). The data set 60 of real elements may, for example, be received by the monitoring module 20, which may be used to evaluate results of a computer-based machine learning system (see discussions below and Fig. 3, in which the "data set 60 of real elements" is fed to the monitoring module 20). In some cases, the monitoring module 20 may be a vehicle component. As already described above, a real element may include a captured image (e.g., a digital image) or a portion thereof, captured, for example, by a sensor such as a camera-based system (e.g., a camera-based system of a vehicle). In this case, the data set of real elements may include a sequence of images (or portions thereof) (e.g., these images may be fed to the monitoring module 20 directly after they are captured while a vehicle is moving). In some cases, the "data set of real elements" may include images captured by the camera-based system at predefined times (e.g., the "data set of real elements" includes the successive images separated by a predetermined time interval).In some cases, the dataset of real-world elements may include a sequence of images processed by the camera-based system, e.g., in real time during image acquisition. It is also conceivable that the "dataset of real-world elements" may include video captured by the camera-based system (e.g., the video may be initially stored upon capture and then transmitted to the monitoring module 20 at a later time). In the example of . Fig. 2a and Fig. 2b, two real elements 1; 2 of the "dataset of real elements" are depicted as two images taken at different times, in which two recorded street scenes can be recognized. Alternatively, two sections of these images 1a; 2a can be used as two real elements of the "dataset of real elements" (more on this below). The dataset 60 of real elements can comprise one or more, five or more, ten or more, one hundred or more, one thousand or more, ten thousand or more images.

[0023] Next, the present techniques include receiving 200 a first result 61; 62 (e.g., by the monitoring module 20 introduced above) calculated by the computer-based machine learning system 70 using the data set of real elements. The computer-based machine learning system 70 may be implemented in a module 30 (in other words, the module 30 may include the computer-based machine learning system 70 as shown in Fig. 3), or the module 30 may be the machine learning system 70. In the example of Fig. 3, the data set 60 of real elements (see examples in the previous paragraph for such a data set) supplied to the monitoring module 20 is also fed to the module 30. In some cases, the module 30 may be a vehicle component. In this context, the computer-based machine learning system 70 may receive the images captured by the camera-based system for further processing of these images, e.g., in real time while a vehicle is moving.

[0024] In the techniques of the present disclosure, a number of real elements of the dataset of real elements may be used as input data for the computer-based machine learning system 70, which generates corresponding output data and an uncertainty value for each real element of the number of real elements. Returning to the example with images as elements: The number of real elements of the dataset of real elements may be a number of images (or sections thereof) captured by the camera-based system at predefined times (e.g., within a predetermined time interval). For example, one image per "N" captured images may belong to the number of real elements (e.g., "N" equals one or more, two or more, five or more, ten or more, fifty or more).Furthermore, the uncertainty value of a real element can characterize a confidence measure of the output data generated by the computer-based machine learning system with respect to the real element of the number of real elements (see also definitions above). The uncertainty value of the real element can, for example, take one value (e.g., the value "one") to reflect the situation when the generated output data is most trustworthy with respect to the real element, and another value (e.g., the value "zero") when the generated output data is not trustworthy. The uncertainty value of the real element can lie between the first and second values ​​(e.g., between the values ​​"zero" and "one"); other definitions for the uncertainty value of the real element are also conceivable.Here, the first result 61; 62 (introduced above) may comprise a number of output data 61 and a number of uncertainty values ​​62 for this number of real elements.

[0025] The next step of the method involves generating 300 a data set 63 of synthetic elements with respect to the data set of real elements. The data set 63 of synthetic elements can be generated, for example, by the monitoring module 20. As already explained in detail above, a "synthetic element" has an artificial origin, which distinguishes it from a "real element."

[0026] In the present techniques, one or more real elements of the dataset of real elements may comprise one or more features 10-15 (see definitions above) present in one or more synthetic elements of the dataset of synthetic elements generated with respect to the corresponding one or more real elements of the dataset of real elements (in other words, starting from the corresponding one or more real elements). In the example of Fig. 2a and Fig. Figure 2b shows two real images taken at different times, in which six features, namely "vehicles 10, 14", "pedestrians 11, 13", and a "cyclist 15" can be seen (e.g., in the correspondingly processed images). The two images 1; 2 can be two images taken consecutively (or images taken at relatively close times), so that they have the same features, although the corresponding features (e.g., the vehicles 10; 12 in images 1; 2) are spatially offset from each other. (E.g., images 1; 2 were taken with a vehicle-integrated sensor system while a vehicle was moving.) These figures also show examples when parts of these images (areas in dashed circles) can be considered real elements 1a; 2a with one feature, the "vehicle 10".

[0027] In one example, generating the dataset of synthetic elements with respect to the dataset of real elements may comprise generating 320 a synthetic element of the dataset of synthetic elements using a real element of the dataset of real elements. This procedure can be illustrated using the example of images as follows: If a real image is an element belonging to the dataset of real elements, this image (i.e., a real element in this example) can be used to generate a synthetic image (i.e., a synthetic element in this example). The synthetic image may, for example, be generated using one of the methods mentioned above. Image 1 comprises six features, namely "vehicles 10, 14," "pedestrians 11, 13," and a "cyclist 15."A synthetic image generated using image 1 may have the same features (i.e., these images can be said to be similar in this sense). Another synthetic image generated using image 2 may also have six features. In another example, one synthetic image may be generated using a portion 1a of image 1. Another synthetic image may be generated using a portion 2a of image 2. In this way, other (e.g., all) synthetic elements of the data set 63 of synthetic elements may be generated (i.e., one synthetic element per corresponding real element).

[0028] In the techniques of the present disclosure, the dataset of real elements may comprise two or more real elements that are correlated with each other, wherein a correlation between the two or more real elements is determined based on an overlap of the one or more features present on these two or more real elements. Furthermore, generating the dataset of synthetic elements with respect to the dataset of real elements may comprise generating 340 a synthetic element of the dataset of synthetic elements using two or more correlated real elements of the dataset of real elements. Such overlap between a feature present on different elements (e.g., images) may, in some cases, be defined by a distance metric that characterizes the spatial offset of the same feature at the pixel level on these elements. For example, ifIf the distance metric between the feature on two considered elements exceeds a predefined value, this may mean that the overlap falls below a predefined overlap value, so these two elements are considered uncorrelated. (If a feature is present on one element while absent on another image, this may also mean that the overlap is below the predefined overlap value, and these two elements are uncorrelated.) For example, two elements in the . Fig. 2a and Fig. 2b shown real elements (ie two images 1; 2) are declared as two correlated elements (in the sense defined above), e.g. because two or more (e.g. all six) overlaps for the respective (same) features in the two images exceed the predefined overlap value.

[0029] In some cases, when a spatially limited part of the image is considered as an element, a predefined value of the distance metric can be used to select a respective spatially limited part on different images. In this case, two elements can have the same feature, as in the Fig. 2a and Fig. 2b: These images show the feature “vehicle 10” on the respective selected parts 1a; 2a (ie elements in this example). In the example of Fig. 2a and Fig. 2b, the elements 1a; 2a can be classified as correlated (in the sense defined above) because they contain the feature whose overlap value on the two images exceeds the overlap value. In other cases, the spatially limited parts of the images (as elements) can have more than one feature. In this way, other (e.g., all) synthetic elements of the data set 63 of synthetic elements can be generated (i.e., one synthetic element for two or more corresponding real images). The correlated two or more real elements of the data set of real elements can represent a sequence of two or more images acquired at different times by a sensor system (e.g., a vehicle-integrated sensor system) (see also the discussions above).In some cases, a generated synthetic element using correlated real elements can emulate a future real element that has not yet been captured.

[0030] Within the scope of the present disclosure, it is conceivable that the "overlap" of one or more features is determined by module 30. Furthermore, the information regarding the correlated elements can be forwarded, for example, to monitoring module 20 (in some cases, "the first result" may contain this information). Monitoring module 20 can, for example, use this information to generate one or more synthetic elements using correlated real elements (as already described above).

[0031] The dataset 63 of synthetic elements may include the synthetic elements generated using the two techniques mentioned above. The dataset 63 of synthetic elements may include one or more, five or more, ten or more, one hundred or more, one thousand or more, ten thousand or more images.

[0032] The next step of the method comprises sending 400 the data set of synthetic elements. The data set 63 of synthetic elements can be sent, for example, from the monitoring module 20, e.g., to an additional module 40. The additional module 40 can comprise the computer-based machine learning system 70, ie, the same computer-based machine learning system that can be associated with the module 30, see Fig. 3. In another case, the data set 63 of synthetic elements may be sent, for example, from the monitoring module 20 to the module 30 comprising the computer-based machine learning system 70 (this case is not shown in the figures).

[0033] Next, the present techniques include receiving 500 a second result 64; 65 calculated by the computer-based machine learning system using the data set of synthetic elements. The second result 64; 65 may, for example, be received by the monitoring module 20, as shown in Fig. 3 shown.

[0034] In the techniques of the present disclosure, a number of synthetic elements of the dataset of synthetic elements may be used as input data for the computer-based machine learning system 70, which generates corresponding output data and an uncertainty value for each synthetic element of the number of synthetic elements. Furthermore, the uncertainty value of a synthetic element may characterize a confidence measure of the output data generated by the computer-based machine learning system with respect to the synthetic element of the number of synthetic elements. The uncertainty value of the synthetic element may, for example, take a first value (e.g., the value "one") to reflect the situation when the generated output data is most trustworthy with respect to the synthetic element, and a second value (e.g., the value "zero") when the generated output data is not trustworthy.The uncertainty value of the synthetic element can be between the first and second values ​​(e.g., between the value "zero" and "one"); other definitions for the uncertainty value of the synthetic element are also conceivable. The second result 64; 65 can comprise a number of output data 64 and / or a number of uncertainty values ​​65 for the number of synthetic elements. The number of synthetic elements can be generated using one or both of the methods described in detail above.

[0035] The next step of the method comprises comparing 600 the first result with the second result (e.g., the “compare” method step can be carried out by the monitoring module 20). Finally, the present techniques comprise determining 700, based on the first result, one or more validity values ​​66 that characterize a confidence measure of the calculated first result. Alternatively or additionally, the present techniques comprise determining 700, based on the comparison of the first result with the second result, one or more validity values ​​66 that characterize a confidence measure of the calculated first result. The “determine” method step can be carried out by the monitoring module 20. A validity value (English “validity bit”) can, for example,have two values: One value can classify the first result as trustworthy (in other words, the validity value can be set to "true"), while the other value can classify the first result as untrustworthy (in other words, the validity value can be set to "false").

[0036] The method step of "determining" the one or more validity values ​​may include setting 710 a first validity value to a value (e.g., to "false") that classifies the first result as untrustworthy if a deviation between an uncertainty value corresponding to a real element observed at a first time and an uncertainty value corresponding to a real element observed at a second time meets a predetermined criterion, wherein the first time is the subsequent time with respect to the second time. Otherwise, if the deviation does not meet the predetermined criterion, the method step of "determining" the one or more validity values ​​may include setting 720 the first validity value to a value (e.g., to "true") that classifies the first result as trustworthy.

[0037] In another case, the method step of "determining" the one or more validity values ​​may include setting 730 a first validity value to a value (e.g., to "false") that classifies the first result as untrustworthy if a deviation between an uncertainty value corresponding to a real element observed at a first time point and a characteristic uncertainty value meets a predetermined criterion. Furthermore, the data set of real elements may comprise two or more real elements. The characteristic uncertainty value may be a weighted sum of two or more uncertainty values ​​for the two or more real elements (e.g., an average of the two or more real elements) observed at respective time points that are the preceding time points with respect to the first time point.Otherwise, if the deviation does not meet the predetermined criterion, the method step “determining” the one or more validity values ​​may include setting 740 the first validity value to a value (e.g., to “true”) that classifies the first result as trustworthy.

[0038] The predetermined criterion may include (for the two variants described above for the method step “Determine”) that the deviation exceeds a predefined threshold value.

[0039] In yet another case, the method step of "determining" the one or more validity values ​​may include setting 750 a first validity value to a value (e.g., to "false") that classifies the first result as untrustworthy if a derivative of a function of uncertainty values ​​at a point in time satisfies a predetermined continuity criterion. The function of uncertainty values ​​may describe the uncertainty values ​​as a function of time, which is determined from the number of uncertainty values ​​for the number of real elements observed at different points in time. Otherwise, if the derivative does not satisfy the predetermined continuity criterion, the method step of "determining" the one or more validity values ​​may include setting 760 the first validity value to a value (e.g., to "true") that classifies the first result as trustworthy.The predetermined continuity criterion may include the derivative exceeding a predefined threshold.

[0040] In the techniques of the present disclosure, comparing the first result with the second result may comprise comparing 620 one or more uncertainty values ​​of the number of uncertainty values ​​for one or more real elements from the number of real elements with one or more uncertainty values ​​of the number of uncertainty values ​​for one or more synthetic elements from the number of synthetic elements generated with respect to the corresponding one or more real elements (as already described above). Alternatively or additionally, comparing the first result with the second result may comprise comparing 640 output data for one or more real elements from the number of real elements with output data for one or more synthetic elements from the number of synthetic elements generated with respect to the corresponding one or more real elements.

[0041] For example, comparing the one or more uncertainty values ​​for the real elements with the corresponding uncertainty values ​​for the synthetic elements may include calculating one or more deviations between the one or more uncertainty values ​​for the real elements and the corresponding uncertainty values ​​for the synthetic elements. Alternatively or additionally, comparing output data for the real elements with the corresponding output data for the synthetic elements may include determining one or more discrepancies between the output data for the one or more real elements and the corresponding output data for the synthetic elements (e.g., discrepancies regarding the classification results of features on synthetic and corresponding real elements, as mentioned above).

[0042] Furthermore, determining the one or more validity values ​​may include setting 770 a second validity value to a value (e.g., to "false") that classifies the first result as untrustworthy if a deviation of the calculated one or more deviations between the uncertainty values ​​for the real and corresponding synthetic elements satisfies a predetermined deviation criterion. Otherwise, if the deviation does not satisfy the predetermined deviation criterion, determining the one or more validity values ​​may include setting 780 the second validity value to a value (e.g., to "true") that classifies the first result as trustworthy. The predetermined deviation criterion may include the deviation exceeding a predefined threshold.

[0043] Alternatively or additionally (to the set second validity value), determining the one or more validity values ​​may include setting 790 a third validity value to a value (e.g., to "false") that classifies the first result as untrustworthy if a number of determined discrepancies between the output data for the real and corresponding synthetic elements meets a predetermined discrepancy criterion. Otherwise, if the number of determined discrepancies does not meet the predetermined discrepancy criterion, determining the one or more validity values ​​may include setting 795 the third validity value to a value (e.g., to "true") that classifies the first result as trustworthy. The predetermined discrepancy criterion may include the number of determined discrepancies exceeding a predefined threshold.

[0044] The method of the first aspect may further include sending 800 one or more validity values ​​(e.g., one or more of the first, second, and third validity values ​​defined above). For example, the monitoring module 20 may send the one or more validity values ​​(e.g., to a planner 50, as described further below).

[0045] In the techniques of the present disclosure, the computer-based machine learning system may be configured for image processing (as explained above), wherein the dataset of real-world elements includes image data. Furthermore, the computer-based machine learning module may be an image classifier (e.g., an image classifier that semantically segments image data pixel-by-pixel or region-by-region, see also the definitions above). The image classifier may be configured to receive input data (e.g., real and / or synthetic elements, as described in detail above) in the form of image data and classify them into multiple classes (e.g., categorize the "features" of these elements into corresponding classes, which may then form the generated output data defined above).

[0046] In some examples, image classification may include semantic segmentation of an image (e.g., region-by-region and / or pixel-by-pixel classification of the image). Image classification may, for example, be object classification. For example, the presence of one or more objects (in other words, "features" as defined above) may be detected in the image data (e.g., road users such as pedestrians, cyclists, or other vehicles in the context of autonomous driving, or traffic signs or lanes in the context of assisted driving). In this case, the computer-based machine learning systems may be integrated into the system for vehicles (e.g., into the systems for assisted or autonomous driving) to provide functionality for the vehicle.

[0047] In other examples, the computer-based machine learning system may be used for a monitoring task (e.g., a manufacturing process and / or for quality assurance). For example, the computer-based machine learning system may be designed or deployed to monitor the operating state and / or the environment of an at least partially autonomous robot. In some examples, the at least partially autonomous robot may be an industrial robot. In other examples, the computer-based machine learning system may be designed or deployed to monitor the operating state and / or the environment of a machine (e.g., a machine tool) or a group of machines (e.g., an industrial plant).In these examples, the input data may contain state data of the at least partially autonomous robot, the machine or group of machines and / or their environment and the output data may contain information regarding the operating state and / or the environment of the respective device.

[0048] In the present disclosure, a device (e.g., a vehicle, a robot, or an industrial plant) may be monitored and / or controlled based on the classification result.

[0049] In further examples, the computer-based machine learning system can be designed or used to control (or regulate) a device. The device can in turn be one of the devices discussed above (e.g., a vehicle, an at least partially autonomous robot, or a machine). In these examples, the input data (as a "real" or "synthetic element") can include state data of the device regarding an internal state of the device (e.g., at least partially sensor data). Additionally or alternatively, the input data can include state data regarding the environment of the device (e.g., at least partially sensor data). The output data of the computer-based machine learning system can characterize an operating state or other internal state of the device (e.g., whether or not an error, an anomaly, or a critical operating condition exists).

[0050] A second general aspect of the present disclosure relates to a computer program configured to carry out the method according to the first general aspect of the present disclosure. The present disclosure also relates to a computer-readable medium (e.g., a machine-readable storage medium such as an optical storage medium or read-only memory, e.g., FLASH memory) and signals storing or encoding the computer program of the present disclosure.

[0051] A third general aspect of the present disclosure relates to a monitoring module 20 configured to execute the computer-implemented method according to the first general aspect and / or to execute the computer program according to the second general aspect. The monitoring module 20 of the third aspect may be configured to receive the data set 60 of real elements. Furthermore, the monitoring module 20 may be configured to receive the first result 61; 62. Furthermore, the monitoring module 20 of the third aspect may be configured to generate the data set 63 of synthetic elements. Furthermore, the monitoring module 20 may be configured to send the data set 63 of synthetic elements and receive the second result 64; 65.Furthermore, the monitoring module 20 may be configured to compare the first result with the second result and to determine one or more validity values ​​66 based on the first result and / or the comparison of the first result with the second result. Furthermore, the monitoring module 20 may be configured to transmit the one or more validity values.

[0052] The monitoring module 20 of the third aspect can be integrated into a vehicle computer or designed as a standalone in-vehicle component (which, for example, communicates with the vehicle computer). The monitoring module 20 can be designed in the same way for the other devices mentioned above. The monitoring module 20 of the third aspect can communicate externally via appropriate communication protocols.

[0053] The monitoring module 20 of the third aspect can comprise one or more processors, controllers, memories, (communication) interfaces, network components, software applications, software architectures, all other software and / or hardware components, parts of the aforementioned, or any combination thereof.

[0054] A fourth general aspect of the present disclosure relates to a device (e.g., a vehicle) comprising a monitoring module 20 according to the third aspect. Furthermore, the device comprises a module 30 comprising a computer-based machine learning system 70 (for various embodiments of the computer-based machine learning system, see the discussions above), wherein the module is configured to receive a data set of real-world elements received from the monitoring module 20 of the third aspect. Furthermore, the module 30 is configured to calculate the first result by the computer-based machine learning system using the data set of real-world elements and to send the first result to the monitoring module 20. The device can communicate externally via appropriate communication protocols.

[0055] The module 30 may comprise one or more processors, controllers, memories, (communication) interfaces, network components, software applications, software architectures, any other software and / or hardware components, parts of the aforementioned, or any combination thereof.

[0056] Furthermore, the device of the fourth aspect may include a planner 50, which may be configured to receive one or more validity values ​​from a monitoring module 20 according to the third aspect. Furthermore, the planner 50 may be configured to control operation of a device (e.g., operation of a vehicle or other device discussed above) based on the received validity values.

[0057] In the example of an autonomous or semi-autonomous driving vehicle, if the first validity value classifies the first result as untrustworthy (e.g., if the first validity value is set to "false"), a command can be sent from the planner 50 (e.g., to a vehicle computer or to a control unit of the vehicle) that triggers a hard maneuver of the vehicle (the hard maneuver can include, for example, emergency braking, stopping the vehicle at a nearby location, deactivating the autonomous driving mode, or any combination thereof). On the other hand, if the second and / or third validity value classifies the first result as untrustworthy (e.g., if the second and / or third validity value is set to "false") and the first validity value classifies the first result as trustworthy (e.g., if the first validity value is set to "true"), a command can be sent from the planner 50 (e.g.,to the vehicle computer or to the vehicle control unit), which triggers a gentle maneuver of the vehicle (the gentle maneuver may, for example, include a reduction in the vehicle speed).

[0058] The Planner 50 may comprise one or more processors, controllers, memories, (communication) interfaces, network components, software applications, software architectures, any other software and / or hardware components, parts of the aforementioned, or any combination thereof.

[0059] A fifth general aspect of the present disclosure relates to a distributed system (e.g., in the form of a cloud system) comprising a device according to the fourth aspect and an additional module 40. The additional module 40 can be designed as a device-external component (e.g., a vehicle-external component) (which, for example, communicates with a vehicle computer). The additional module 40 comprises the computer-based machine learning system 70. The computer-based machine learning system of the additional module 40 can be the same computer-based machine learning system that belongs to the module 30 of the device (e.g., both machine learning systems can comprise a model described by the same parameters). The additional module 40 can be configured to receive a data set of synthetic elements sent by the monitoring module 20 according to the third aspect.Furthermore, the additional module 40 may be configured to generate a second result by the computer-based machine learning system using the data set of synthetic elements and to send the second result to the monitoring module 20.

[0060] The add-on module 40 can comprise one or more processors, controllers, memories, (communication) interfaces, network components, software applications, software architectures, any other software and / or hardware components, parts of the aforementioned, or any combination thereof. The add-on module 40 can communicate externally via appropriate communication protocols.

Claims

[1] A computer-implemented method for evaluating results of a computer-based machine learning system, the method comprising the following steps: Receiving (100) a data set (60) of real elements, wherein each real element (1; 1a; 2; 2a) of the data set of real elements corresponds to a real observation; Receiving (200) a first result (61; 62) calculated by the computer-based machine learning system (70) using the data set of real elements; generating (300) a data set (63) of synthetic elements with respect to the data set of real elements; Sending (400) the data set of synthetic elements; Receiving (500) a second result (64; 65) calculated by the computer-based machine learning system using the data set of synthetic elements; comparing (600) the first result with the second result; Determining (700), based on the first result and / or comparing the first result with the second result, one or more validity values ​​(66) that characterize a confidence measure of the calculated first result. [2] A computer-implemented method according to claim 1, wherein one or more real elements of the dataset of real elements comprise one or more features (10-15) present in one or more synthetic elements of the dataset of synthetic elements generated with respect to the corresponding one or more real elements of the dataset of real elements. [3] The computer-implemented method of claim 2, wherein generating the data set of synthetic elements with respect to the data set of real elements comprises generating (320) a synthetic element of the data set of synthetic elements using a real element of the data set of real elements. [4] The computer-implemented method of any one of the preceding claims 1 to 3, wherein the data set of real elements comprises two or more real elements that are correlated with each other, wherein a correlation between the two or more real elements is determined based on an overlap of the one or more features present on these two or more real elements, and wherein generating the data set of synthetic elements with respect to the data set of real elements comprises generating (340) a synthetic element of the data set of synthetic elements using two or more correlated real elements of the data set of real elements. [5] A computer-implemented method according to any one of the preceding claims 1 to 4, wherein a number of real elements of the data set of real elements are used as input data for the computer-based machine learning system 70, which generates corresponding output data and an uncertainty value for each real element of the number of real elements, where the uncertainty value of a real element characterizes a confidence measure of the output data generated by the computer-based machine learning system with respect to the real element of the number of real elements, wherein the first result (61; 62) comprises a number of output data (61) and a number of uncertainty values ​​(62) for this number of real elements. [6] The computer-implemented method of claim 5, wherein determining the one or more validity values ​​comprises setting (710) a first validity value to a value that classifies the first result as untrustworthy when a deviation between an uncertainty value corresponding to a real element observed at a first time and an uncertainty value corresponding to a real element observed at a second time satisfies a predetermined criterion, wherein the first time is the subsequent time with respect to the second time, and otherwise, if the deviation does not meet the predetermined criterion, setting (720) the first validity value to a value that classifies the first result as trustworthy; or determining the one or more validity values, setting (730) a first validity value to a value that classifies the first result as untrustworthy if a deviation between an uncertainty value corresponding to a real element observed at a first time and a characteristic uncertainty value meets a predetermined criterion, where the data set of real elements comprises two or more real elements, where the characteristic uncertainty value is a weighted sum of the two or more uncertainty values ​​for the two or more real elements observed at respective times that are the preceding times with respect to the first time, and otherwise, if the deviation does not meet the predetermined criterion, setting (740) the first validity value to a value that classifies the first result as trustworthy, optionally wherein the predetermined criterion comprises that the deviation exceeds a predefined threshold. [7] The computer-implemented method of claim 5, wherein determining the one or more validity values ​​comprises setting (750) a first validity value to a value that classifies the first result as untrustworthy if a derivative of a function of uncertainty values ​​at a time satisfies a predetermined continuity criterion, where the function of uncertainty values ​​describes the uncertainty values ​​as a function of time, which is determined from the number of uncertainty values ​​for the number of real elements observed at different times, and otherwise, if the derivative does not meet the predetermined continuity criterion, setting (760) the first validity value to a value that classifies the first result as trustworthy, optionally wherein the predetermined continuity criterion comprises that the derivative exceeds a predefined threshold. [8] A computer-implemented method according to any one of the preceding claims 5 to 7, wherein a number of synthetic elements of the data set of synthetic elements are used as input data for the computer-based machine learning system 70, which generates corresponding output data and an uncertainty value for each synthetic element of the number of synthetic elements, wherein the uncertainty value of a synthetic element characterizes a confidence measure of the output data generated by the computer-based machine learning system with respect to the synthetic element of the number of synthetic elements, wherein the second result (64; 65) comprises a number of output data (64) and / or a number of uncertainty values ​​(65) for the number of synthetic elements. [9] The computer-implemented method of claim 8, wherein comparing the first result with the second result comprises: comparing (620) one or more uncertainty values ​​of the number of uncertainty values ​​for one or more real elements from the number of real elements with one or more uncertainty values ​​from the number of uncertainty values ​​for one or more synthetic elements from the number of synthetic elements generated with respect to the corresponding one or more real elements; and / or comparing (640) output data for one or more real elements from the number of real elements with output data for one or a plurality of synthetic elements from the number of synthetic elements created with respect to the corresponding one or more real elements. [10] The computer-implemented method of claim 9, wherein comparing the one or more uncertainty values ​​for the real elements with the corresponding uncertainty values ​​for the synthetic elements comprises calculating one or more deviations between the one or more uncertainty values ​​for the real elements and the corresponding uncertainty values ​​for the synthetic elements. [11] The computer-implemented method of claim 10, wherein determining the one or more validity values ​​comprises setting (770) a second validity value to a value that classifies the first result as untrustworthy if a deviation of the calculated one or more deviations between the uncertainty values ​​for the real and corresponding synthetic elements satisfies a predetermined deviation criterion, and otherwise, if the deviation does not meet the predetermined deviation criterion, setting (780) the second validity value to a value that classifies the first result as trustworthy, optionally wherein the predetermined deviation criterion comprises that the deviation exceeds a predefined threshold. [12] A computer-implemented method according to any one of the preceding claims 9 to 11, wherein comparing output data for the real elements with the corresponding output data for the synthetic elements comprises determining one or more discrepancies between the output data for the one or more real elements and the corresponding output data for the synthetic elements. [13] The computer-implemented method of claim 12, wherein determining the one or more validity values ​​comprises setting (790) a third validity value to a value that classifies the first result as untrustworthy if a number of determined discrepancies between the output data for the real and corresponding synthetic elements satisfy a predetermined discrepancy criterion, and otherwise, if the number of discrepancies determined does not meet the predetermined discrepancy criterion, setting (795) the third validity value to a value that classifies the first result as trustworthy, optionally wherein the predetermined discrepancy criterion comprises that the number of detected discrepancies exceeds a predefined threshold. [14] A computer-implemented method according to any one of the preceding claims 1 to 13, further comprising sending (800) one or more validity values. [15] Computer-implemented method according to one of the preceding claims 1 to 13, wherein the computer-based machine learning system is designed for image processing and wherein the data set of real elements contains image data. [16] The computer-implemented method of claim 15, wherein the computer-based machine learning module is an image classifier, optionally an image classifier that semantically segments image data pixel-wise or region-wise. [17] A computer program adapted to carry out the method according to any one of the preceding claims 1 to 16. [18] A monitoring module (20) adapted to execute the computer-implemented method according to any one of the preceding claims 1 to 16 and / or to execute the computer program according to claim 16. [19] A device comprising: a monitoring module (20) according to claim 18; and a module (30) comprising a computer-based machine learning system (70), the module is designed to: receive a data set of real elements received from the monitoring module (20) according to claim 18; calculating a first result by the computer-based machine learning system using the data set of real elements; and sending the first result to the monitoring module (20). [20] A distributed system comprising: a device according to claim 19; an additional module (40) comprising the computer-based machine learning system (70), the additional module being designed to: receive a data set of synthetic elements sent by the monitoring module (20) according to claim 18; generate a second result by the computer-based machine learning system using the data set of synthetic elements; to send the second result to the monitoring module (20).

Images