Safe control / monitoring of a computer-controlled system

Abstract

The present invention relates to a computer-implemented method (700) for classifying sensor data for use in controlling and / or monitoring a computer-controlled system. The classification model includes an inference model that determines, based on the sensor data, respective density parameters of a Dirichlet distribution of class probabilities for each of a plurality of classes. The classification model further includes a generative model that determines, based on the class probabilities, parameters of the probability distribution of the sensor data according to a training dataset for the classification model. The density parameters according to the inference model are used for anomaly detection by determining the probability that the sensor data is generated according to the generative model based on the density parameters. The same density parameters are also used for determining the class probabilities.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for classifying sensor data used when controlling and / or monitoring a computer-controlled system, and a corresponding system. The present invention further relates to a computer-implemented method for training a classification model used when controlling and / or monitoring a computer-controlled system, and a corresponding system. The present invention further relates to a computer-readable medium containing instructions and / or model data.

Background Art

[0002] Background Art Automobiles and other vehicles are increasingly making autonomous decisions based on the classification of sensor data. For example, in vehicles commercially available today, lane-keeping assistance systems use sensor data, such as images of the vehicle environment or features extracted from such images, to recognize driver tasks, such as lane keeping or lane changing. In this case, the lane-keeping assistance system can assist the driver in an appropriate manner using the recognized tasks. In future vehicles, more and more tasks should be automatically executed based on classified sensor data. However, if these tasks are executed incorrectly, tasks posing an increased risk will also be executed. The same can be said for many other computer-controlled systems, including robotic systems, household appliances, manufacturing machines, personal assistants, access management systems, drones, nanorobots, and heating control systems.

[0003] Misclassification of sensor data can lead to incorrect automatic decisions and potentially dangerous consequences. Such misclassification can occur especially when the classification model is applied to sensor data inputs that do not adequately correspond to the data during its training, for example, when applied to sensor data representing rare traffic situations not encountered in the training data set or sensor data resulting from incorrect measurements. Therefore, it is important to detect out-of-distribution (OOD) samples, i.e., sensor data inputs that do not adequately correspond to the training data set when the classification model was trained. If it is detected that the sensor data is OOD, for example, a warning to a human, a switch to a fallback mechanism, and / or a lower importance assignment to the output by the classification model can be made.

[0004] In the paper “Evidential Deep Learning to Quantify Classification Uncertainty” by M. Sensoy et al. (available at https: / / arxiv.org / abs / 1806.01768, which is hereby incorporated by reference), an explicit modeling of the prediction uncertainty of a multi-class classification model is proposed. The predictor for a multi-class classification problem is a Dirichlet distribution with parameters set by the continuous output of a neural network. Based on these parameters, both the class label and the uncertainty can be determined.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Summary of the Invention Known multi-class classifiers can be used to determine whether sensor data is out-of-distribution (OOD), e.g., whether the sensor data represents sensor misrecognition or an unusual situation, but have the drawback of not providing calibrated in-domain uncertainty. In-domain uncertainty is important because sensor data that is not OOD can still be difficult to classify. For example, sensor data can represent a boundary condition regarding whether a driver is maintaining a lane or attempting to change lanes. Also, the type of uncertainty is important for determining how to use the classifier output for control or monitoring. This is because the type of uncertainty also determines how strongly a control device can rely on a given classification.

[0007] Such in-domain uncertainty is particularly desirable to be determinable in the form of the probability that the input data belongs to a given calibrated class, e.g., in the form of the probability that out of an input set for which the model provides a 70% probability, approximately 70% of the inputs of that set belong to the given class. For example, by having uncertainty values calibrated in this way, it is possible to compare uncertainty information regarding different sensor inputs and / or uncertainty information calculated according to different versions of a similar model.

[0008] In addition to determining whether sensor data is out-of-distribution, it is desirable to provide a technique for classifying sensor data used when controlling and / or monitoring a computer-controlled system that provides better calibrated in-domain uncertainty.

Means for Solving the Problems

[0009] According to a first aspect of the present invention, a computer-implemented method and a corresponding system for classifying sensor data used when controlling and / or monitoring a computer-controlled system are provided, as defined in claims 1 and 13 respectively. According to another aspect of the present invention, a computer-implemented method and a corresponding system for training a classification model used when controlling and / or monitoring a computer-controlled system are provided, as defined in claims 12 and 14 respectively. According to one aspect of the present invention, a computer-readable medium is described, as defined in claim 15.

[0010] Various aspects relate to a classification model that classifies sensor data into one class from a set of multiple classes. For example, the classification model may output one overall classification of the sensor data, or may include multiple respective classifications into the set of classes for each part of the sensor data. For example, in time-series sensor data, each class can be determined at each point in time. In image data, the classification model can determine each class for each image part (e.g., pixel), and for example, the classification model may be an image segmentation model.

[0011] The classification model described in this specification may include both a trained inference model and a trained generation model. When sensor data is provided, the inference model can determine the concentration parameters of the Dirichlet distribution of class probabilities for each class into which the sensor data can be classified. Interestingly, these concentration parameters can be used as inputs for both determining out-of-distribution values using the generation model (at least when the out-of-distribution values show sufficient correspondence to the sensor data for the training dataset) and determining class probability values for each class. The generation model can be trained to determine the parameters of the probability distribution of the sensor data according to the training dataset when class probabilities are provided. Thus, based on the determined concentration parameters, the probability that the sensor data is generated according to the generation model can be determined, and this probability can be utilized as an out-of-distribution value indicating the correspondence of the sensor data to the training dataset of the classification model. Further, the probability that the sensor data belongs to a specific class can also be determined from the concentration parameters.

[0012] Interestingly, the inventors have found that well-calibrated class probabilities can be obtained by using the concentration parameters both when determining out-of-distribution values and when determining class probabilities. That is, when training the classification model, it can be effectively forced for the classification model to determine the concentration parameters such that not only can the classification model predict the correct class accurately, but also the sensor data can be reproduced accurately enough by the generation model. The inventors have found that this functions as an effective mechanism for forcing the model to determine concentration parameters that well account for the uncertainty regarding the sensor data.

[0013] For example, in “Evidential Deep Learning to Quantify Classification Uncertainty”, there is no such mechanism that provides calibrated class probabilities. Significantly, the model can be trained by a combination of model fitting and regularization. Through model fitting, the model is encouraged to output the current label of the training data with the greatest certainty. Regularization encourages the model to indicate that the model is uncertain about out-of-distribution values. However, neither such model fitting nor regularization provides a mechanism to encourage the model to provide calibrated probability values for in-distribution data. In contrast, as described herein, such a mechanism is provided by using the concentration parameter that is also used as an input to the generative model for OOD detection. In fact, the inventors have found that the class probability values derived according to the provided technique are more accurate with respect to calibration than the class probability values derived from the Dirichlet distribution of “Evidential Deep Learning to Quantify Classification Uncertainty”.

[0014] Interestingly, since out-of-distribution values are determined using a generative model trained to reproduce sensor data from concentration parameters, it may be possible to train a classification model without the need for OOD samples. This is advantageous because it is difficult to obtain a representative set of OOD samples by definition. Training a model on an explicit set of OOD samples results in a model that cannot detect OOD cases that do not resemble the OOD training set. Therefore, with respect to the accuracy of OOD detection, it is advantageous to learn the domain using only in-domain samples. For example, in an alternative approach, a classification model can be trained such that it outputs the correct class for in-domain samples of the training dataset and the maximum uncertainty for out-of-distribution samples of the training dataset. Such a training approach has the drawback of requiring OOD samples and does not provide a mechanism for providing calibrated in-domain probability values.

[0015] In contrast, by training a generative model on in-domain samples and using this model to determine out-of-distribution values, as proposed herein, training can be enabled only for in-domain samples and calibrated in-domain probability values can be determined. In particular, a classification model can be trained on a training dataset that includes sensor data inputs and corresponding target classes. The model can be trained by applying an inference model to the sensor data to obtain concentration parameters and deriving a training signal based on these concentration parameters. In this case, the training signal can be used to update the parameters of the inference model and / or the generative model.

[0016] Interestingly, the concentration parameter can be used to train both classification and OOD detection. For this purpose, the training signal can include the contribution to the training of the evidence classifier and the contribution to density estimation. Thus, the model can determine class probability values based on the concentration parameter that is further trained to cause the generative model to reproduce the sensor data, resulting in improved calibration.

[0017] In particular, in the training of the evidence classifier, the training signal can be based on the probability that the sensor data is classified into the target class based on the concentration parameter. The concentration parameter can define the probability distribution across the class probabilities for each class, and further, these class probabilities can define the probability that the sensor data is classified into the target class. The training signal may be configured to maximize the probability for the target class here. Thus, the model can be encouraged to provide a correct classification of the sensor data.

[0018] For density estimation, the training signal can further be based on the probability that sensor data is generated according to the generation model based on the concentration parameter. As described above, the concentration parameter can define a probability distribution over the class probabilities for each class. Based on the class probabilities, the generation model can determine the parameters of the probability distribution over the sensor data. The training signal can be configured to maximize the probability of the sensor data of the training data set generated according to the probability distribution. Therefore, the model can be trained such that the reproduction probability of the sensor data is higher for the sensor data according to the training data set than for the sensor data not according to the training data set by applying the inference model and subsequently the generation model to the sensor data according to the training data set. Thus, the probability can be used as an out-of-distribution value. This results in a more accurate out-of-distribution value and at the same time provides calibration of the class probability values because they are determined from the same concentration parameters that are also used in density estimation.

[0019] Optionally, applying a classification model can include outputting an out-of-distribution value. For example, the out-of-distribution value can be obtained by comparing the probability that sensor data is generated according to the generation model with a threshold. Class probability values for one or more classes, for example, class probability values for one or more most-likely classes, or class probability values for one or more requested classes, or class probability values for all classes, can also be output, for example, always or only if the out-of-distribution value shows sufficient correspondence.

[0020] These outputs, such as out-of-distribution values and / or class probabilities, can be used when controlling and / or monitoring a computer-controlled system. For example, the computer-controlled system can be a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access management system. These are examples of systems that are controlled based on sensor data measured from the system and / or its environment, and thus may have the problem of being applied in situations where the classification model has not been trained, or the problem that incorrect sensor measurements may be obtained. Therefore, the techniques provided herein can be advantageously applied. Exemplary systems that can be monitored include a surveillance system or a medical (imaging) system.

[0021] Basically, the techniques provided using out-of-distribution values and / or class probability values can also be used for purposes other than controlling and / or monitoring a computer-controlled system. Therefore, the provided method can be well regarded as a method for classifying sensor data, such as the image itself, and the provided system can be well regarded as a system for classifying sensor data itself, such as the image itself, where the classification result is not necessarily used for control and / or monitoring.

[0022] Overall, the techniques provided herein are applicable to many different types of sensor data, such as image data, audio data, and measurements of various other types of physical quantities, such as temperature, pressure, etc. Various examples are provided herein. In particular, each of the plurality of classification models provided herein can be applied to each of the plurality of types of sensor data, and thus classifications with comparable class probability values can be obtained. Application to sensor data is basically not required. For example, the classification model is operable for any model input for which OOD detection and / or determination of class probability is required.

[0023] As an optional means, the sensor data can include, for example, a time series of measurement values of one or more physical quantities at at least two time points, at least five time points, or at least ten time points. In fact, the classification of time series is often an important input when controlling and / or monitoring a computer-controlled system. The classification of time series can include the determination of one class for the entire time series and / or the determination of each class for each time point of the time series. Appropriate inference models known per se can be used. The generative model can include a recurrent model configured to determine the parameters of the probability distribution of the values of one or more physical quantities at a given time point of the time series based on the parameters of the probability distribution at the previous time points of the time series. The recurrent model can receive the overall classification of the time series as an overall input, or, for example, can receive the class probabilities at each time point as an input for determining the probability distribution parameters at that time point.

[0024] As an optional means, the classification model can be used in a lane keeping assistance system of a vehicle. The sensor data can include the position information of the vehicle and / or the position information of traffic participants in the vehicle environment, for example, the position information extracted from the image data of one or more cameras installed in the vehicle. Such positions can be used to classify the sensor data into various driver tasks represented by this sensor data. In particular, the driver tasks can include a class representing that the vehicle is maintaining a lane and one or more classes representing that the vehicle is changing lanes, for example, a change to the left or a change to the right.

[0025] As an optional means, a generative model including a recurrent model can be used to predict future values of one or more physical quantities at one or more future time points after a time series, for example, at one or more future time points to come, or at one or more time points where at least measured values are not yet available. In this case, these predicted future values can be used when controlling and / or monitoring a computer-controlled system. For example, a recurrent generative model can be used at a certain time point t, and by further developing this generative model over time, the positions of the vehicles closest at one or more future time points t+1, t+2 can be predicted. Future predictions can be used, for example, by a planning control module of an autonomous driving system to generate control amounts of speed and / or steering angle.

[0026] As an optional means, sensor data may represent captured images of a computer-controlled system and / or its environment. For example, the sensor data can be video data, radar data, LiDAR data, ultrasonic data, motion data, or thermal image data. As inference models and generative models, various machine-learnable models that operate on images, such as convolutional neural networks and in particular fully convolutional neural networks, can be used as is known in the art. Thus, the classification model can be an image classification model. For example, an image classification model can classify image data into a plurality of respective classes according to which of a plurality of respective objects are present in the image. The image classification model can also output whether one or more respective objects are present in the image. For example, the image classification model can be an object detection model. The image classification model can also classify each image part, for example individual pixels, into respective classes. For example, the image classification model can be a semantic segmentation model. For example, the plurality of classes of the image classification model can include traffic signs, road surfaces, pedestrians, vehicles, and / or the driver tasks of the detected vehicles.

[0027] As an optional means, when the classification model is a semantic segmentation model, the generation model can be configured to determine respective probability distribution parameters for respective image portions. Accordingly, using the generation model, respective out-of-distribution values for respective image portions can be determined. The out-of-distribution values can be used to highlight image regions where the labels are uncertain. Thereby, for example, these highlighted image regions can be excluded when controlling and / or monitoring a computer-controlled system, or other modalities can be prioritized for the regions.

[0028] As an optional means, determining the out-of-distribution values can include sampling class probabilities for a plurality of classes from a Dirichlet distribution, determining parameters of a probability distribution of the sensor data from the class probabilities, and determining a probability that the sensor data is sampled according to the parameters of the probability distribution. Thereby, it becomes possible to efficiently determine the out-of-distribution values as an approximation of the probability that the sensor data is generated according to the generation model. For example, the approximation here can also be used when the probability cannot be expressed in a closed-form expression.

[0029] As an optional means, the concentration parameter can be restricted to be 1 or more. For example, the inference model can be defined such that it can output only values of 1 or more. Basically, it is not necessary to enforce this by restricting the inference model, but when the concentration parameter is greater than 1, it is guaranteed that the parameter is located in a more regular part of the Dirichlet distribution, and the numerical stability during training is improved.

[0030] As an optional means, it is possible to determine whether to use a normal control module or a fallback control module using the determined out-of-distribution value and / or class probability value. If the out-of-distribution value shows sufficient correspondence and / or the class probability value for a certain class shows sufficient confidence in the classification to the class, control data can be determined using the normal control module. Otherwise, the control data can be determined using the fallback control module. Subsequently, the computer-controlled system can be controlled based on the control data. Since the provided technology provides more accurate OOD detection and more accurate class probabilities, safety is improved by switching to the fallback control module as necessary, and controllability is improved by using the normal control module when it is determined to be possible.

[0031] As an optional measure, when out-of-distribution values indicate non-compliance, sensor data can be stored for future use. Otherwise, the sensor data can be discarded. This has the advantage of being able to collect relevant training data more efficiently. For example, other machine learning models can be trained on the collected data. In autonomous vehicles and other types of computer-controlled systems, typically large amounts of data are collected, and often it is not possible to store all of the collected data and / or transmit all of the collected data for use by the training system. Therefore, it is important to select the training data that is expected to be relevant. For example, the sensor data may be vehicle sensor data for perception of autonomous driving. Such sensor data can be obtained by interacting with the real world during free-form driving. However, since data can constantly flow into the sensor, there may not be enough space to store all of it. By using the determined class probability values and / or the determined out-of-distribution values to perform information-theoretic calculations for the expected information gain, such as storage of sensor data, it is possible to determine whether to retain the sensor data. The sensor data can be stored in the vehicle itself, or it can be transmitted from the sensor to a central server and stored there. This enables smart data selection and can reduce data collection costs.

[0032] As an optional means, the generative model can be provided as an input of the value of the context variable. The context variable can determine a prior distribution for class probabilities, such as a concentration parameter. When applying the model, the value of the context variable can be determined from a set of context instances including respective sensor data and the corresponding target class. Thereby, a more flexible classification model that can be easily and dynamically adapted to various environments is obtained. For example, the context instance may be the latest measured value of a physical quantity of a computer-controlled system. In a lane keeping assist system, for example, the context instance can effectively represent the driver's intention over the period represented by this context instance. For example, longer-term measured values and / or measured values from other computer-controlled systems can also be used so that the classification model in an autonomous vehicle or a semi-autonomous vehicle can be adapted to various weather conditions, different types of roads, various countries and regions, etc.

[0033] Those skilled in the art will understand that two or more of the above-described embodiments, implementation forms, and / or aspects as optional means of the present invention can be combined in any manner considered useful.

[0034] Corresponding modifications and variations of any system and / or any computer-readable medium corresponding to the modifications and variations of the corresponding computer-implemented method to be described are executable by those skilled in the art based on the description herein.

[0035] These aspects and other aspects of the present invention will become clear and definite by referring to the embodiments described as examples in the following description and the accompanying drawings.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Figure 3

Figure 4a

Figure 4b

Figure 4c

Figure 5a

Figure 5b

Figure 6

Figure 7

Figure 8

[0037] Note that the drawings are purely schematic and not drawn to scale. In the drawings, elements corresponding to elements already described may be labeled with the same reference numerals.

Modes for Carrying Out the Invention

[0038] Detailed Description of Embodiments FIG. 1 shows a system 100 for training a classification model used when controlling and / or monitoring a computer-controlled system. The classification model can be configured to classify sensor data into one class from a set of multiple classes.

[0039] System 100 may include a data interface 120. The data interface 120 may be for accessing a training data set 030. The training data set 030 may include a plurality of training instances, for example, it may include at least 1,000, at least 10,000, or at least 100,000 training instances. A training instance may be a labeled training instance that includes sensor data and a corresponding target class from a set of multiple classes respectively. The training data set 030 typically does not include training instances designated as out-of-distribution. For example, the set 030 may be a set of in-distribution training data.

[0040] The data interface 120 may further be for accessing model data 040. The model data 040 can represent a classification model. The classification model can include a trainable inference model. The inference model can be configured to determine, based on sensor data, each concentration parameter of the Dirichlet distribution of class probabilities for each of a plurality of classes. The classification model may further include a trainable generative model. The generative model can be configured to determine parameters of the probability distribution of sensor data based on class probabilities. For example, the model data 040 may include trainable parameters of the inference model and / or the generative model. For example, the number of trainable parameters of the inference model may be at least 1,000, at least 10,000, or at least 1,000,000. For example, the number of trainable parameters of the generative model can be at least 1,000, at least 10,000, or at least 1,000,000. The model data 040 may be used, for example, by the system 200 of FIG. 2 or the system 300 of FIG. 3, when controlling and / or monitoring a computer-controlled system according to the methods described herein.

[0041] For example, as also shown in FIG. 1, the input interface may be configured by a data storage interface 120 capable of accessing data 030, 040 from the data storage 021. For example, the data storage interface 120 may be a memory interface or a persistent storage interface, such as a hard disk interface or an SSD interface, but may also be a personal area network interface, a local area network interface, or a wide area network interface, such as a Bluetooth, Zigbee, or Wi-Fi interface, or an Ethernet or optical fiber interface. The data storage 021 may be an internal data storage of the system 100, such as a hard drive or an SSD, or may also be an external data storage, such as a network-accessible data storage. In some embodiments, the data 030, 040 may be accessed from different data storages, respectively, for example, via different subsystems of the data storage interface 120. Each subsystem may be of the type described above with respect to the data storage interface 120.

[0042] The system 100 may further include a processor subsystem 140 that can be configured to train the classification model 040 during operation of the system 100.

[0043] To train the model, the processor subsystem 140 can select a training instance from the training data set 030. The training instance can include sensor data and a corresponding target class from a set of multiple classes. The processor subsystem 140 can further apply the inference model to the sensor data to obtain density parameters. Further, the processor subsystem 140 can derive a training signal for the training instance. The training signal can be based on the probability that the sensor data is classified into the target class based on the density parameters. The training signal can further be based on the probability that the sensor data is generated according to the generative model based on the density parameters. The processor subsystem 140 can update the parameters 040 of the inference model and / or the generative model based on the training signal.

[0044] System 100 may further include an output interface for outputting trained data 040 that represents a learned (or "trained") model. For example, as also shown in FIG. 1, the output interface may be constituted by a data interface 120, where in this embodiment, the interface is an input / output ("IO") interface, and through this input / output ("IO") interface, trained model data can be stored in the data storage 021. For example, the model data 040 that defines an "untrained" classification model can be replaced, at least partially, by the model data of a model that has been at least partially trained during or after training. In this case, the parameters of the model, such as the weights of the neural network and other types of parameters, can be adapted to reflect the training on the training data 030. This is shown in FIG. 1 by the fact that both the trained model data and the untrained model data on the data storage 021 are referred to by the same reference numeral 040. In other embodiments, the trained model data is stored separately from the model data that defines an "untrained" dynamics model. In some embodiments, the output interface may be separate from the data storage interface 120, but generally may be of the type described above with respect to the data storage interface 120.

[0045] FIG. 2 shows a system 200 for classifying sensor data 224 used when controlling and / or monitoring a computer-controlled system.

[0046] System 200 may include a data interface 220 for accessing model data 040 representing a classification model. The classification model can be configured to classify sensor data into one class from a set of multiple classes. The classification model may include a trained inference model. The inference model can be configured to determine, based on the sensor data, each concentration parameter of the Dirichlet distribution of class probabilities for each of the multiple classes. The classification model may further include a trained generation model. The generation model can be configured to determine, based on the class probabilities, the parameters of the probability distribution of the sensor data according to the training data set of the classification model. The model data may include the trained parameters of the generation model and / or the inference model. The model data 040 can be pre-trained, for example, by the system 100 of FIG. 1 as described herein, or as described elsewhere. System 200 can train the model in addition to applying it, for example, by combining system 200 with the system of FIG. 1.

[0047] For example, as also shown in FIG. 2, the data interface may be constituted by a data storage interface 220 that can access the data 040 from a data storage 022 storing the data 040. Generally, the data interface 220 and the data storage 022 may be of the same type as those described for the data interface 120 and the data storage 021 with reference to FIG. 1. The storage 022 may be part of the system 200, but may also be external. The data storage 022 may optionally also include sensor data.

[0048] System 200 can further include a processor subsystem 240, which can be configured to apply an inference model to sensor data to obtain concentration parameters during operation of the system 200. The processor subsystem 240 can further be configured to determine an out-of-distribution value indicating the degree of correspondence of the sensor data to the training data set. The out-of-distribution value can be determined by determining the probability that the sensor data is generated according to the generation model based on the concentration parameters. The processor subsystem 240 can further be configured to determine a class probability value from the concentration parameters when at least the out-of-distribution value exhibits sufficient correspondence (e.g., is less than a corresponding threshold or greater than a corresponding threshold). The class probability value can indicate the probability that the sensor data belongs to one class from a set of multiple classes. The processor subsystem 240 can further be configured to output the class probability value used when controlling and / or monitoring a computer-controlled system.

[0049] It will be understood that the considerations and implementation options similar to those of the processor subsystem 140 of FIG. 1 apply to the processor subsystem 240. Further, it will be understood that the considerations and implementation options similar to those of the system 100 of FIG. 1 generally apply to the system 200 unless otherwise stated.

[0050] System 200 may include a sensor interface 260 for obtaining sensor data 224 from, for example, sensor 072. The sensor can be placed within environment 082, but can also be placed at a location remote from environment 082, for example, in the case of a physical quantity that can be measured remotely. Note that sensor 072 may not be part of system 200. Sensor 072 may have any suitable form such as an image sensor, a LiDAR sensor, a radar sensor, a pressure sensor, a built-in temperature sensor, etc. In some embodiments, since sensor data 072 can be obtained from two or more different sensors that sense different physical quantities respectively, it can be sensor measurements of different physical quantities. The sensor data interface 260 may have any suitable form corresponding to the type of sensor, including but not limited to, for example, a low-level communication interface based on I2C data communication or SPI data communication, or a data storage interface of the type described above with respect to data interface 220.

[0051] In some embodiments, system 200 may include an actuator interface 280 for providing control data 226 to actuator 092 within environment 082. Such control data 226 can be generated by the processor subsystem 240 to control the actuator based on the determined out-of-distribution values and / or class probability values. The actuator may be part of system 200, for example, the system itself that system 200 is to control. For example, the actuator may be an electric actuator, a hydraulic actuator, a pneumatic actuator, a thermal actuator, a magnetic actuator, and / or a mechanical actuator. As non-limiting specific examples, an electric motor, an electroactive polymer, a hydraulic cylinder, a piezoelectric actuator, a pneumatic actuator, a servo mechanism, a solenoid, a stepping motor, etc. can be mentioned. This type of control will be described with reference to FIG. 3 for a (semi) autonomous vehicle.

[0052] In other embodiments (not shown in FIG. 2), system 200 may include output interfaces to rendering devices such as displays, light sources, speakers, vibration motors, etc., and using these output interfaces, sensor-sensitive output signals that can be formed based on out-of-distribution values and / or class probability values can be formed. The sensor-sensitive output signal may directly indicate the output of the sensor, but may also be, for example, a sensor-sensitive output signal derived for use in the guidance, navigation, or other types of control of a physical system.

[0053] Generally, each system described herein includes, but is not limited to, system 100 of FIG. 1 and system 200 of FIG. 2, and can be embodied as a single device or apparatus such as a workstation or server, or can be embodied within a single device or apparatus. The device may be an embedded device. The device or apparatus may include one or more microprocessors that execute appropriate software. For example, the processor subsystem of each system can be embodied by a single central processing unit (CPU), but may also be embodied by a combined device or system of such CPUs and / or other types of processing units. The software can be downloaded and / or stored in a corresponding memory, such as volatile memory like RAM or non-volatile memory like flash. Alternatively, the processor subsystem of each system can be implemented in the form of programmable logic within the device or apparatus, for example, as a field programmable gate array (FPGA). Generally, each functional unit of each system can be implemented in the form of a circuit. Each system can also be implemented in a distributed form including various devices or apparatuses, such as a distributed local server or a cloud-based server. In some embodiments, system 200 may be part of a vehicle, robot, or similar physical entity, and / or may be a control system configured to control a physical entity.

[0054] Figure 3 shows the above example, where a system for classifying sensor data, such as system 200 in Figure 2, is shown as part of a control system 300 of a (semi) autonomous vehicle 62 operating in environment 083. The vehicle 62 is autonomous in that it may include a driving assistance system, also referred to as an autonomous driving system or a semi-autonomous system.

[0055] The autonomous vehicle 62 may be one in which a classification system is incorporated as part of, for example, a lane keeping assistance system 300. The classification system can receive, as input, a time series of position information of the vehicle 62 and / or position information of traffic participants within the vehicle environment 083, for example representing a highway scene, and a time series of position information characterizing the behavior of the vehicle over time, for example the behavior of the vehicle over the most recent 5 seconds. For example, as shown, the position information may be extracted from an image of the environment of the vehicle 62 captured by a camera 22 attached to the vehicle.

[0056] The classification system can classify the time series sensor data into respective classes representing respective driver tasks of the vehicle driver, for example respective classes representing lane keeping or lane changing. To analyze such time series data, the inference model may include a recurrent model that outputs a classification based on the input data at each respective point in time, as is known per se. Also, the generative model used for OOD detection can be a recurrent model configured to determine the parameters of the probability distribution of one or more physical quantities at a given point in time in the time series based on the parameters of the probability distribution at a previous point in time in the time series. The use of existing types of recurrent models is possible.

[0057] The calibrated class probability values derived from the concentration parameters, and thus with higher reliability, can be used by the lane keeping assistance system 300 to perform an adequacy assessment for the prediction. Based on this adequacy assessment, the system can determine whether to use the normal control module of the lane keeping assistance system or to switch, for example, to a fallback control system that controls the vehicle wheels 42. For example, if the predictor reports an intermediate level of uncertainty, the comfort is reduced, but a safe fallback function such as a lane keeping assistance algorithm that allows for a high level of interpretation and is safer to use can be activated. For example, if the predictor reports a high level of uncertainty and / or the outlier values indicate that the sensor data is out-of-distribution, the lane keeping assistance system 300 can return control to the human driver.

[0058] Although not specifically shown in the figure, if the outlier values indicate non-compliance and / or the class probability values indicate insufficient reliability, the classification system can also be configured to store the sensor data for future use. For example, the system can store the sensor data itself or transmit the sensor data for external storage. The sensor data may be discarded after being used in other ways. In this way, the autonomous vehicle 62 can collect relevant training data for training the machine learning model used for the control of the (semi-)autonomous vehicle. By storing only the relevant sensor data, the limited storage capacity in the vehicle 62 and / or the limited transmission capacity for transmission to external storage can be used more efficiently.

[0059] As another example of switching to fallback control, the classification models described herein can also be used for anomaly detection in vehicle 62. In this example, the classification model can be a semantic segmentation model configured to classify each image portion of the images captured by camera 22 into respective classes. Based on the semantic segmentation, control system 300 can calculate the depth information of all pedestrians, calculate the trajectories around these pedestrians, and perform control so that vehicle 62 follows the trajectories sufficiently faithfully without colliding with the pedestrians. Also in this case, if the out-of-distribution values for each image portion show insufficient correspondence and / or the class probability values for each image portion show insufficient reliability, the semantic segmentation can be considered to have insufficient reliability, and a safe fallback control system can be utilized. Although (semi)-autonomous vehicle 62 is used as an example, it will be understood that similar techniques can be applied to any mobile robot to avoid humans in the environment.

[0060] FIG. 4a shows a non-limiting detailed example of a classification model that does not include a generative model. The figure shows a plate graph where the black circles represent observable random variables and the dashed circles represent latent random variables inferred from the observable random variables.

[0061] The figure shows an evidence classifier, also known as a prior network, designed according to, for example, "Evidential Deep Learning to Quantify Classification Uncertainty". The figure shows sensor data x, i.e., SD,410, and using this sensor data x, i.e., SD,410, class probability π, i.e., CP,420 is derived, and output variable y, i.e., CL is determined from this class probability π, i.e., CP,420. Such a model is π|x~Dir(π│g ψ (x)) y|π~Cat(y│π) It can be described as a hierarchical probability model, where Dir(·) is the Dirichlet distribution, and g ψ (x) is a function that maps a given input x, i.e., the SD, such as an observed image, to a Dirichlet concentration parameter (also known as the intensity parameter), and Cat(·) is a categorical distribution that assigns probability mass to the output variable y, i.e., the CL, given the previous sample π, i.e., the CP.

[0062] Such evidence models can be trained by minimizing the following loss for ψ, such as the weights when g ψ (·) is selected as a neural network, i.e.,

Number

Number

[0063] A similar classification model without the general model can be obtained by training on a set of in-domain samples and a set of OOD samples according to the following loss function, i.e.,

Number

[0064] These two loss functions L1, L2 follow the following more general pattern, i.e.,

Number

[0065] For example, loss functions following the above patterns without a generative model are generally observed to be insufficient to achieve both (i) a calibrated quantification of the predictive uncertainty for in-domain samples, and (ii) the detection of out-of-domain OOD samples that do not require the confirmation of exemplary OOD observations during training time. This is because the loss functions following the above structures do not have a mechanism to evaluate whether a given input sample x is within the target domain without explicit supervised learning from real OOD samples. Significantly, the above loss structure facilitates the model to provide model outputs for in-domain samples with very high certainty. The inventors have found that as a result, accurate class probability values cannot be derived from such models.

[0066] Figure 4b shows a non-limiting detailed example of a classification model including a generative model. Similar to Figure 4a, the model is shown as a plate graph.

[0067] The classification model in this figure is based on the input sensor data x, i.e., SD,410, and for each concentration parameter h of the Dirichlet distribution ψIt includes an inference model IM configured to determine (x). The Dirichlet distribution is a probability distribution of class probabilities π, i.e., CP,420, for each class into which the sensor data SD is to be classified. Based on the class probability CP,420, the classification CL,430 of the sensor data SD can be defined according to the categorical distribution CAT,460. The set of possible classes is finite. For example, the number of classes may be, for example, 2, at most or at least 5, or at most or at least 10.

[0068] The classification model further includes a generative model GM,450, which, based on the class probability CP, determines the parameters f of the probability distribution of the sensor data SD according to the training data set on which the classification model is trained θ (π). By determining the probability that the sensor data SD is generated according to the generative model GM based on the class probability CP or based on the concentration parameter that generates it, an out-of-distribution value indicating the correspondence of the sensor data SD to the training data set can be determined.

[0069] Therefore, in order to learn the evaluation of the domain relevance of a given sample SD without requiring explicit supervised learning, density estimation using the generative model GM can be performed simultaneously with the training of the evidence classifier using the inference model IM. Mathematically, the model can be expressed by the following generative design, i.e., π~Dir(π|1,…,1) x|π~p θ (x|f θ (π)) y|π~Cat(y|π) where p θ (x|f θ (π)) is a likelihood function in the input domain x, or in other words, a probability distribution of the sensor data. For example, the probability distribution may be, for example, a normal distribution, a categorical distribution, a Bernoulli distribution, etc., for example,

Number

[0070] As shown in the figure, the inference model IM is trainable such that a variational inference approximation of the class probabilities for the training data set is provided from the class probabilities CP sampled from the Dirichlet distribution according to the concentration parameters given by itself. In the figure, the dashed line of the inference model IM represents the variable dependence on q ψ (π n │x n ).

[0071] Specifically, when a training data set D = {(x1, y1), …, (x n or other types of sensor data and their corresponding labels y n} including N input-output pairs, for example, images x N , y N ) is given, the inference model is trainable to provide an accurate approximation to the intractable posterior distribution over the latent evidence variables p(π1, …, π Ν |D). The training loss for this objective can be derived by using variational inference to minimize, for example,

Number

Number

[0072] Interestingly, the model can be regarded as a hybrid form of evidence learning and variational autoencoder (VAE). Therefore, the model can be referred to as an evidence variational autoencoder. The model provides surprising advantages from both the perspective of evidence learning and the perspective of VAE. From the perspective of the evidence learning model, by introducing VAE characteristics, surprisingly, within-domain calibration becomes possible. From the perspective of using the model as a VAE, the use of the Dirichlet distribution in normal use as a VAE should not be expected to be beneficial. For example, a more typical choice in the settings here is the standard normal distribution. Also, a model used as a VAE typically has a relatively large latent space dimension, for example, at least 50 or at least 100. In the VAE setting, it is not expected that reducing such dimensions would be advantageous. Instead, in an embodiment, the latent space dimension may correspond to the number of classes of the classifier and thus, for example, be less than 50, for example, at most 5 or at most 10. Surprisingly, the inventors have found that such a design using a relatively small-dimensional Dirichlet distribution enables a combination with evidence learning and helps to significantly improve the within-domain calibration performance of the predictor.

[0073] In the above example, reimbursement is applied. For example, the approximate posterior distribution q ψ (π n │x n ) is defined to depend on the observed value D. Also, the model can be defined in a non-reimbursement manner, for example, using the posterior distribution q ψ (π n ).

[0074] FIG. 4c shows a non-limiting detailed example of a classification model that includes a generative model and uses context variables. This example is an extension of the example of FIG. 4b. In particular, similar to the example of FIG. 4b, this figure shows sensor data SD,410, inference model IM’,441, generative model GM,450, class probability CP,420, categorical distribution CAT,460, and classification CL,430.

[0075] In this example, for the inference model IM’, the value of the context variable z, i.e., CV,470, is additionally given as an input. The value of the context variable CV is determined from a set of context instances during the use of the classification model, where the context instances are context sensor data x j i.e., CSD,411, and, optionally, the corresponding context target class y j i.e., CCL,431.

[0076] In particular, as shown in the figure, the value of the context variable CV can be determined by applying a trained context model, e.g., h φ (x j ,y j ) to each context instance CSD,CCL to obtain the contribution of each context parameter, or by aggregating the contributions of each context parameter into a set of context parameters, e.g.,

Number

[0077] For example, the context variable may well be a real value or a discrete value, and can be selected from, for example, 2, up to or at least 5, or up to or at least 10 possible values. The context set (x1,y1),…,(x K ,y K ) As a context model that maps to the context variable CV, for example, the output y k can be ignored and the inference model IM’ can be used, or the input-output pair can be mapped and the context encoder network m(x k ,y k ) that is trained together with the inference model and the generation model can be used.

[0078] For example, using the following mathematical model, the context variable CV, the class probability CP, the classification CL, and the sensor data SD can be defined respectively (a normal distribution is shown for the sensor data SD, but it is also possible to use other probability distributions instead). That is,

Number

Number

[0079] Generally, a classification model may include one or more context variables CV. For example, the number of such context variables can be, at most or at least, 5, or at most or at least 10. By using the context variable CV, the classification model can effectively determine whether a newly presented sample is similar to previously observed ones. For example, for similar samples, it can provide a lower uncertainty than in the case of unfamiliar samples. Note that when context information CSD, CCL is not available, for example, a model using context variables can also be applied by using a prior distribution in the Dirichlet distribution instead.

[0080] FIG. 5a shows a non-limiting detailed example of how to classify sensor data using a classification model that includes a generative model such as the model described with reference to FIG. 4b or the model described with reference to FIG. 4c.

[0081] The figure shows, for example, a model input x that represents measurements of one or more physical quantities of a computer-controlled system and / or its environment to be controlled or monitored. * That is, SD, 510 is shown. For example, the sensor data SD can be an image, a time series of measurements of one or more physical quantities, etc.

[0082] When the sensor data SD is given, the inference model g ψ That is, IM, 541 is used to obtain the concentration parameters CPi, 542 of the Dirichlet distribution of the class probabilities for each of the plurality of classes into which the sensor data is to be classified. The inference model is a function g with free parameters ψ that maps from the input domain to K-dimensional positive real numbers α = g ψ (x). ψIt can be defined by (·), where K is the number of classes. This function can be implemented on a computer, for example, on a neural network with weights θ. Interestingly, the determined concentration parameter can be used both when determining out-of-distribution values and when determining class probability values.

[0083] A class probability value CPi,530 indicating the probability that sensor data SD belongs to one class from a set of multiple classes can be determined by defining the class probability value CPi using a categorical distribution Prob,560 from class probabilities generated according to a Dirichlet distribution defined by this concentration parameter CPi from the concentration parameter.

[0084] In particular, for a given test input x * with respect to the label y * the predictive distribution for [Number] can be calculated in closed form as follows, where Γ(·) is the gamma function, [Number] is the j-th output channel of the predictor g ψ (·), K is the number of classes in the classification problem, and k is the class queried for y * to which it is addressed.

[0085] When a context variable z determined from a set of context instances X C ,Y C (where the target class Y C is optional) is used, the class probability value can be calculated in the same way as [Number] as follows.

[0086] The input can be classified into a specific class by selecting the class with the maximum probability mass, i.e.,

Number

Number

Number

[0087] Interestingly, the concentration parameter CPi can also be used for OOD detection, in other words, outlier detection. The out-of-distribution value OODV,551 can be determined as an indication of the correspondence of the sensor data SD to the training dataset used for training the classification model. The out-of-distribution value can be determined as the probability that the sensor data SD is generated according to the generative model f θ by calculating the likelihood of the reconstruction of the input with respect to the sensor data SD, based on the concentration parameter CPi, in other words. Mathematically, this can be expressed as

Number

[0088] Also, the mathematical expression can be adapted as needed to take into account context variables.

[0089] Specifically, as shown in the figure, the out-of-distribution value OODV is the class probability π for a plurality of classes from the Dirichlet distribution, i.e., CP,520, for example q ψ (π|x * ) is sampled, and then the reconstruction probability [Number] Regarding this, it can be determined in a Monte Carlo format by evaluating this with SamG,550.

[0090] This may also include applying the generative model [Number] to determine the parameters of the probability distribution p θ of the sensor data. The generative model can be defined by a function f θ (π) having free parameters θ that perform a mapping from the domain of a K-dimensional simplex to the input domain, for example x = f θ (·). The function can be implemented on a computer, for example, on a neural network having weights θ, or by other known techniques.

[0091] Based on the determined parameters, the probability that the sensor data SD is generated according to the probability of the parameters can be determined. The calculation here can be, for example, based on sampling from p θ (x│λ) using an unbiased sampler. The probability distribution can be, for example, a normal distribution in which each mean value and, optionally, each standard deviation are output by the generative model.

[0092] The out-of-distribution value OODV can also be the determined probability, or can be obtained from the probability by threshold processing. For example, the sample x * is for a threshold ε > 0 selected based on the safety requirements applied at that time.

Number

Number

Number

[0093] Depending on the application, out-of-distribution values OODV and / or class probabilities CPi for one or more classes can be output. For example, class probabilities can be determined only if the sensor data SD is not determined to be out-of-distribution, but class probabilities can also be calculated regardless of whether the sensor data is OOD. For example, possible prediction outputs can be the predicted class

Number

[0094] In various embodiments, the sensor data SD may include a time series of measurements of one or more physical quantities. In this case, the inference model IM and / or the generation model SamG may include a recurrent model, such as a recurrent neural network. For example, the recurrent neural network may be a neural network based on a gated recurrent unit (GRU) or a neural network based on long short-term memory (LTSM). Based on the configuration of the Dirichlet concentration parameter CPi, various dynamics modalities are possible. For example, the concentration parameter CPi determined by the inference model IM can generate discrete class labels for each time point for a plurality of time points, whereby the classification model predicts a time series including continuous variables and / or discrete variables, for example, to perform sequence-to-sequence classification. The Dirichlet variable may be subject to a constraint to a single value in a single prediction, in which case the classification model can perform classification of the entire time series.

[0095] Also, the generation model SamG can also use a recurrent model to determine the parameters of the probability distribution for generating sensor data at each time point. For example, the model can determine the parameters at a given time point based on the parameters at a previous time point and the concentration parameters at the given time point. Interestingly, this can determine not only the out-of-distribution value OODV for the entire time series but also each out-of-distribution value at each time point, for example, which can be considered when using the classification at each time point.

[0096] In various embodiments, the sensor data SD may include image data. Also in this case, the inference model IM can perform classification of the entire image, but can also perform classification of each image region. For example, the classification model may be a semantic segmentation model. In the latter case, the generation model may be an image-to-image conversion model that determines the parameters of each probability distribution for each image part. Similar to the case of time series, this can determine not only the overall out-of-distribution values, but also the out-of-distribution values for each image part. For example, the generation model can determine each average value, and optionally can also determine the standard deviation for each image part. Suitable image-to-image conversion models are known per se in the art and are readily applicable.

[0097] FIG. 5b shows a non-limiting detailed example of how to train a classification model that includes a generation model such as the model described with reference to FIG. 4b or the model described with reference to FIG. 4c. Various options regarding such models described with reference to FIG. 5a are also equally applicable here.

[0098] Training can be performed based on a training data set that includes a plurality of training instances. The training instances are labeled. For illustrative purposes, a training instance is shown that includes sensor data x n i.e., SD,511 and the corresponding target class y n i.e., TC,531. The training data set can be described as D={(x1,y1),…,(x N ,y N )}, where x n is from the input domain and y n ∈{1,…,K} relates to the class count K.

[0099] As part of the training, the inference model g ψThat is, the inference model IM, 541 can be applied to the sensor data SD to obtain the concentration parameters CPi, 542 of the Dirichlet distribution Dir, 543 of the class probabilities π, i.e., CP, 520 for each of the plurality of classes. Based on the concentration parameter CPi, both the loss of the inference by the inference model IM and the loss when reconstructing the sensor data using the concentration parameter can be derived. Interestingly, the inventors have found that since the concentration parameter CPi is used not only for inference but also for reconstruction, the calibration of the class probabilities of the inference is improved.

[0100] Specifically, as shown in the figure, the training signal may include a loss term LOSS2, 582 determined at Prob, 560 based on the probability that the sensor data SD is classified into the target class TC based on the concentration parameter. The loss term can be used to learn an accurate label predictor. For example, the loss term can be

Number

[0101] Another loss term LOSS1, 581 is shown as being determined at SamG, 550 based on the probability that the sensor data SD is generated according to the generation model f θ based on the concentration parameter CPi. The term can be implemented by the expression

Number

[0102] As shown in the figure, the training signal may optionally include a third loss term LOSS3,583 determined at Div,580, which provides regularization of the determined class probability CP by penalizing, for example, the divergence from the class probability representing the maximum uncertainty. This term can be implemented, for example,

Number

[0103] For example, the above three losses can be combined into an evidence lower bound (ELBO) loss function to maximize with respect to the trainable parameters {θ,ψ} for each of the generative model and the inference model. That is,

Number

[0104] The above loss function can, for example, as described with reference to FIGS. 4c and 5a, use the context variable

Number

[0105] Based on training signals, such as the combination of LOSS1, LOSS2 and, optionally, LOSS3, the parameters PARψ,501 of the inference model and / or the parameters PARθ,502 of the generation model can be updated in the training operation Train,590. For example, the training can be executed in an iterative form where one or both of the sets of parameters are updated with each iteration. The trainable parameters used to determine the context variables can be updated as well when used.

[0106] The training Train can be executed using an unconstrained optimizer such as Adam, Stochastic Gradient Descent or RMSProp, as disclosed in Kingma and Ba, “Adam: A Method for Stochastic Optimization” (available at https: / / arxiv.org / abs / 1412.6980 and incorporated herein by reference). As is known, such optimization methods are heuristic and / or can reach local optima. The training can be executed on an instance-by-instance basis or in batches of up to or at least 64 or up to or at least 256 instances. The training signal can be implemented using an automatic differentiation library such as PyTorch or TensorFlow. By using the training Train, the learned values θ for the free parameters of the generation model f ψ (·) and the inference model g [Number] can be determined.

[0107] In many cases, the loss terms LOSS1 and LOSS3 can be implemented by calculating their respective analytical solutions. In some cases, the analytical solution of the loss term LOSS2 can also be used similarly. As an alternative, the use of the differential Monte Carlo sampling method is also possible, as shown by the sampling operation SamD,543 for sampling class probabilities and the sampling operation SamG for sampling the parameters of the probability distribution of sensor data. For example, the "r-sample" method (where "r" means reparameterizable) can be used in the same way as in known deep learning libraries.

[0108] FIG. 6 shows a block diagram of a computer-implemented method 600 for training a classification model. The model can be used when controlling and / or monitoring a computer-controlled system. The classification model can be configured to classify sensor data into one class from a set of multiple classes. The method 600 may correspond to the operation of the system 100 in FIG. 1. However, this is not limiting, and the method 600 can also be executed using other systems, other devices, or other apparatuses.

[0109] The method 600 can include obtaining 610 a training data set including a plurality of training instances as an operation referred to as "obtaining training data".

[0110] The method 600 can include accessing 620 model data representing a classification model as an operation referred to as "accessing model data". The classification model can include a trainable inference model. The inference model can be configured to determine, based on sensor data, the respective concentration parameters of the Dirichlet distribution of class probabilities for each of the multiple classes. The classification model can further include a trainable generation model. The generation model can be configured to determine the parameters of the probability distribution of sensor data based on class probabilities.

[0111] Method 600 can include, as an operation referred to as "training a model", training a classification model 630. Training the classification can include, as an operation referred to as "selecting an instance", selecting a training instance 640 from a training dataset. The training instance can include sensor data and a corresponding target class from a set of multiple classes. Training the classification can include, as an operation referred to as "applying an inference model", applying an inference model to the sensor data to obtain a density parameter 650. Training the classification can include, as an operation referred to as "deriving a training signal", deriving a training signal 660 for the training instance. The training signal can be based on the density parameter and based on the probability that the sensor data is classified into the target class. The training signal can further be based on the density parameter and based on the probability that the sensor data is generated according to a generative model. Method 600 can include, as an operation referred to as "updating a model", updating the parameters of the inference model and / or the generative model 670 based on the training signal.

[0112] FIG. 7 shows a block diagram of a computer-implemented method 700 for classifying sensor data used when controlling and / or monitoring a computer-controlled system. Method 700 can correspond to the operation of system 200 of FIG. 2 or system 300 of FIG. 3. However, this is not limiting, and method 700 can also be executed using other systems, other devices, or other apparatuses.

[0113] Method 700 can include, as an operation referred to as "obtain model data", obtaining 710 model data representing a classification model. For example, the classification model may have been pre-trained according to the methods described herein. The classification model can be configured to classify sensor data into one class from a set of multiple classes. The classification model may include a trained inference model. The inference model can be configured to determine, based on the sensor data, respective concentration parameters of a Dirichlet distribution of class probabilities for each of the multiple classes. The classification model may further include a trained generative model. The generative model can be configured to determine parameters of a probability distribution of the sensor data based on the class probabilities.

[0114] Method 700 can include, as an operation referred to as "obtain sensor data", obtaining 720 sensor data. The sensor data can represent measurements of one or more physical quantities of a computer-controlled system and / or its environment.

[0115] Method 700 can include, as an operation referred to as "apply inference model", applying the inference model to the sensor data to obtain 730 concentration parameters.

[0116] Method 700 can include, as an operation referred to as "detect OOD", determining 740 an out-of-distribution value indicating the correspondence of the sensor data to the training data set. Determining 740 can be performed by determining, based on the concentration parameters, the probability that the sensor data is generated according to the generative model.

[0117] Method 700 can include determining 760 a class probability value from a density parameter as an operation referred to as "determining a class probability". The class probability value can indicate the probability that sensor data belongs to one class from a set of multiple classes. Method 700 can further include outputting 770 the class probability value to be used during control and / or monitoring as an operation referred to as "outputting a class probability". Determining 760 and outputting 770 can be conditionally executed, for example, in determination 750 referred to as at least "having sufficient correspondence", if the out-of-distribution value shows sufficient correspondence.

[0118] Overall, each operation of method 600 in FIG. 6 and method 700 in FIG. 7 can be applied in any suitable order, subject to, for example, a specific order required by the relationship between input and output if applicable, for example, continuously or simultaneously or in combinations thereof. It will also be understood that some or all of the methods can be combined, for example, method 700 of applying a trained model can be applied following the trained model trained according to method 600.

[0119] The method can be implemented on a computer as a computer-implemented method, dedicated hardware, or a combination of both. As also shown in FIG. 8, instructions for a computer, such as executable code, may be stored on a computer-readable medium 800 in the form of, for example, a series of machine-readable physical marks 810 and / or as elements having a series of various characteristics or values, such as electrical or magnetic or optical characteristics or values. The executable code may be stored temporarily or non-temporarily. Examples of computer-readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. An optical disk 800 is shown in FIG. 8. Alternatively, the computer-readable medium 800 may include, for example, transient data or non-transient data 810 representing model data representing a classification model trained according to the methods described herein and / or used according to the methods described herein. The classification model may include a trainable inference model configured to determine, based on sensor data, respective concentration parameters of a Dirichlet distribution of class probabilities for each of a plurality of classes. The classification model may further include a trainable generative model configured to determine, based on the class probabilities, parameters of a probability distribution of sensor data according to a training data set of the classification model.

[0120] Features as examples, embodiments or optional means should not be understood as limiting the claimed invention, whether or not expressly stated as non-limiting.

[0121] Each of the above-described embodiments is for illustrative purposes and not for limiting the present invention. It should be noted that those skilled in the art can design many alternative embodiments without departing from the scope of the appended claims. The reference signs enclosed in parentheses in the claims should not be construed as limiting the scope of the claims. The use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those recited in the claims. The article "a" preceding an element does not exclude the presence of a plurality of such elements. Expressions such as "at least one of" placed with respect to an enumeration or listing of elements represent that all or any subset of elements is selected from the enumeration or listing. For example, the expression "at least one of A, B, and C" should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention can be implemented using hardware including several distinct elements and also using a computer appropriately programmed. In the claims of an apparatus invention listing several means, some of these means can be embodied by the same hardware item. The fact that specific measures are recited in mutually different dependent claims is merely a fact and does not indicate that a combination of these measures cannot be used to obtain an advantage.

Claims

1. A computer-implemented method (700) for classifying sensor data used when controlling and / or monitoring a computer-controlled system, comprising: - obtaining model data representing a classification model configured to classify sensor data into one class from a set of multiple classes (710), wherein the classification model comprises: - a trained inference model configured to determine, based on the sensor data, respective concentration parameters of a Dirichlet distribution of class probabilities for each of the multiple classes; and - a trained generative model configured to determine, based on the class probabilities, parameters of a probability distribution of the sensor data according to a training data set of the classification model; (710); - obtaining sensor data representing measurements of one or more physical quantities of the computer-controlled system and / or its environment (720); - applying the inference model to the sensor data to obtain concentration parameters (730); - determining an out-of-distribution value indicating the degree of correspondence of the sensor data to the training data set by determining the probability that the sensor data is generated according to the generative model based on the concentration parameters (740); - determining a class probability value indicating the probability that the sensor data belongs to one class from a set of multiple classes from the concentration parameters when at least the out-of-distribution value indicates sufficient correspondence (750); - outputting the class probability value for use in the control and / or monitoring (770). The method (700) comprising.

2. The sensor data includes a time series of measurements of the one or more physical quantities, The generative model includes a recurrent model configured to determine parameters of a probability distribution of values of the one or more physical quantities at a given point in time based on parameters of a probability distribution at a previous point in time. The method (700) according to claim 1.

3. The method (700) further comprises controlling a lane keeping assistance system of a vehicle based on the determined class probability value, The sensor data includes position information of the vehicle and / or position information of traffic participants in the vehicle environment. Each class represents a respective driver task of the driver of the vehicle. The method (700) according to claim 2.

4. The method (700) further comprises using the generation model to predict future values of the one or more physical quantities at one or more future time points after the time series, and outputting the predicted future values for use in the control and / or monitoring. including The method (700) according to claim 3.

5. The sensor data represents captured images of the computer-controlled system and / or its environment. The method (700) according to claim 1.

6. The classification model is a semantic segmentation model configured to classify each image part of the image into respective classes. The method (700) according to claim 5.

7. The generation model is configured to determine parameters of respective probability distributions for the respective image parts, and The method (700) includes determining respective out-of-distribution values for the respective image parts. The method (700) according to claim 6.

8. The concentration parameter is restricted to be 1 or more. The method (700) according to claim 1.

9. The method (700) includes determining control data for controlling the computer-controlled system using a normal control module when the out-of-distribution value shows sufficient correspondence and / or the class probability value shows sufficient confidence, and determining control data using a fallback control module otherwise, and controlling the computer-controlled system based on the control data. The method (700) according to claim 1.

10. The method (700) further comprises storing the sensor data for future use when the out-of-distribution value shows non-correspondence and / or the class probability value shows insufficient confidence, and discarding the sensor data otherwise. The method (700) according to claim 1.

11. The inference model is additionally given a value of a context variable as input. The method (700) further comprises determining the value of the context variable from a set of context instances, where one context instance includes sensor data and, optionally, a corresponding target class. The method (700) according to claim 1.

12. A computer-implemented method (600) for training a classification model used in controlling and / or monitoring a computer-controlled system, wherein the classification model is configured to classify sensor data into one class from a set of multiple classes. In the method (600), - obtaining a training data set including a plurality of training instances (610); - accessing model data representing the classification model (620), wherein the classification model · is a trainable inference model configured to determine, based on sensor data, each concentration parameter of a Dirichlet distribution of class probabilities for each of the multiple classes; · is a trainable generative model configured to determine, based on the class probabilities, parameters of a probability distribution of sensor data; and includes this (620); - the classification model is · selecting, from the training data set, a training instance including sensor data and a corresponding target class from a set of multiple classes (640); · applying the inference model to the sensor data to obtain concentration parameters (650); · deriving a training signal for the training instance (660), wherein the training signal is based on the probability that the sensor data is classified into the target class based on the concentration parameters, and the training signal is further based on the probability that the sensor data is generated according to the generative model based on the concentration parameters (660); · updating the parameters of the inference model and / or the generative model based on the training signal (670); and training by this (630); and includes the method (600).

13. A system (200, 300) for classifying sensor data used in controlling and / or monitoring a computer-controlled system, wherein the system A data interface (220) for accessing model data (040) representing a classification model configured to classify sensor data into one class from a set of multiple classes, wherein the classification model - A trained inference model configured to determine, based on the sensor data, respective concentration parameters of respective Dirichlet distributions of class probabilities for each of the multiple classes, - A trained generative model configured to determine, based on the class probabilities, parameters of a probability distribution of sensor data according to the training data set of the classification model, including the data interface (220), - A sensor interface (260) for acquiring sensor data (224) representing measurements of one or more physical quantities of the computer-controlled system (200, 300) and / or its environment (082, 083), - A processor subsystem (240), comprising, wherein the processor subsystem (240) - Applies the inference model to the sensor data to obtain concentration parameters, - Based on the concentration parameters, determines an out-of-distribution value indicating the degree of correspondence of the sensor data to the training data set by determining the probability that the sensor data is generated according to the generative model, - When at least the out-of-distribution value indicates a sufficient degree of correspondence, determines a class probability value indicating the probability that the sensor data belongs to one class from a set of multiple classes from the concentration parameters, and outputs the class probability value for use in the control and / or monitoring, a system (200, 300) configured as such.

14. A system (100) for training a classification model used for controlling and / or monitoring a computer-controlled system, wherein the classification model is configured to classify sensor data into one class from a set of multiple classes, in the system (100), the system (100) - A data interface (120) for accessing a training data set (030) including a plurality of training instances and model data (040) representing the classification model, wherein the classification model - A trainable inference model configured to determine, based on the sensor data, respective concentration parameters of a Dirichlet distribution of class probabilities for each of a plurality of classes; - A trainable generative model configured to determine, based on the class probabilities, parameters of a probability distribution of the sensor data; A data interface (120) including; - A processor subsystem (140) and The processor subsystem (140) trains the classification model by: - Selecting, from the training data set, a training instance including sensor data and a corresponding target class from a set of a plurality of classes; - Applying the inference model to the sensor data to obtain concentration parameters; - Deriving a training signal for the training instance, the training signal being based on a probability that the sensor data is classified into the target class based on the concentration parameters, and the training signal further being based on a probability that the sensor data is generated according to the generative model based on the concentration parameters; - Updating parameters of the inference model and / or the generative model based on the training signal. A system (100) configured to be trained by the above.

15. A computer-readable medium (800) storing non-transitory data (810), the data (810) being: - Instructions for causing the processor system to perform the computer-implemented method according to Claim 1 when executed by the processor system. The computer-readable medium (800).

Images