Apparatus and method for determining uncertainty of sensor signals synthesized by generative machine learning system

Abstract

An approach for determining the uncertainty of sensor signals synthesized by a generative machine learning system (61) u A computer-implemented method (900) for uncertainty relating to the probability of observing sensor signals in physical reality, the method comprising the steps of: obtaining (901) a noise sample (z); performing (902) Bayesian inference on the generative machine learning system (61) using the noise sample (z) as input to determine a posterior prediction distribution of a sensor signal that would otherwise be synthesized by the generative machine learning system (61) using the noise sample (z) as input to the generative machine learning system (61); and providing (903) a measure of the variability of the posterior prediction distribution as uncertainty about the sensor signal synthesized from the noise sample (z), wherein the method is characterized in that the posterior prediction distribution characterizes the distribution of potential features of the sensor signal that would otherwise be generated using the noise sample (z) as input.

Description

Existing technology

[0001] The BayesDiff was published by Kuo et al. in “BayesDiff: Estimating Pixel-wise Uncertainty in Diffusionvia Bayesian Inference”, 2024, https: / / arxiv.org / abs / 2310.11142.

[0002] Berry and Meger’s “Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators”, 2024, https: / / arxiv.org / abs / 2308.13498 discloses pairwise distance estimators for probability distributions.

[0003] Technical background and advantages of the present invention Modern technological systems that interact with physical reality typically require some form of internal model to model the current state of the corresponding environment in order to determine which action to take. Such a model can be implemented in the form of a machine learning model that can process sensor signals from the technological system's environment to extract meaningful information, such as the presence of other objects, people, or animals.

[0004] To determine the state of physical reality based on sensor signals, such machine learning systems typically require large amounts of data for accurate predictions. Obtaining such data is a tedious and costly task. Therefore, generative methods can be employed to determine the sensor signals that would normally occur in physical reality. Conditional generative models also allow for the input of additional information into the generation process to define the desired properties that should be present in the generated sensor signals.

[0005] While generative models such as StableDiffusion can generate very realistic sensor signals, generative models are machine learning models and therefore may suffer from inaccurate modeling of the probability distribution of sensor signals obtained from physical reality, and thus may provide outputs that do not well represent sensor signals from physical reality (e.g., it may contain artifacts or values ​​that are unlikely or improbable when recording sensor signals from physical reality).

[0006] Therefore, it is desirable to identify sensor signals generated (also known as “synthesized”) from generative machine learning systems that are unlikely to occur in physical reality. Known methods such as BayesDiff use Bayesian inference to determine an uncertainty score for the generated sensor signal in order to judge the likelihood that the generated sensor signal is an accurate representation of possible situations in physical reality. High uncertainty can be interpreted as the synthesized sensor signal having low quality (e.g., containing artifacts), while low uncertainty can be interpreted as the synthesized sensor signal likely having high quality (e.g., clear and realistic).

[0007] However, the inventors found that common methods assess the uncertainty of generative processes based on the generated sensor signal values. For example, when generating images, uncertainty is assessed based on pixel-level differences. Empirically, the inventors found that this assessment of uncertainty based on the "raw" values ​​of the synthesized sensor signals leads to insufficiently informative estimates of generative uncertainty. For instance, assessing pixel-level differences in images generated from machine learning systems may result in high uncertainty scores if only a small number of pixels show significant differences numerically.

[0008] Advantageously, the method having the features of claim 1 avoids this problem. Specifically, the method assesses the uncertainty of the synthesized sensor signal by determining the uncertainty of the latent factors (also called latent features) of the generated sensor signal. By relying on latent factors rather than the values ​​of the sensor signal, the assessment of uncertainty is performed at the “semantic” level of the generated sensor signal; in other words, the uncertainty of the content of the sensor signal is assessed rather than its original value. Empirically, the inventors have found that such a method leads to more accurate predictions of the validity of the synthesized sensor signal relative to physical reality, which in turn leads to more accurate machine learning systems trained using such generated data, or more reliable estimates of the performance of machine learning systems (if the machine learning system is tested with samples of low uncertainty). Summary of the Invention

[0009] In a first aspect, the present invention relates to a computer-implemented method for determining the uncertainty of a sensor signal synthesized by a generative machine learning system, the uncertainty relating to the probability of observing the sensor signal in physical reality, the method comprising the steps of: Obtain noise samples; Using the noise samples as input to the generative machine learning system, Bayesian inference is performed on the generative machine learning system to determine the posterior prediction distribution of the sensor signal that would otherwise be synthesized by the generative machine learning system using the noise samples as input to the generative machine learning system. The variability of the posterior prediction distribution is provided as a measure of the uncertainty of the sensor signal synthesized from the noise samples. The method is characterized by: The posterior prediction distribution characterizes the distribution of the potential features of the sensor signal that would otherwise be generated using the noise sample as input.

[0010] This method can be understood as generating synthetic sensor signals while simultaneously assessing the uncertainty surrounding those signals—that is, the likelihood that the synthetic sensor signals actually exist in physical reality. Physical reality can be understood as being modeled through a generative machine learning system: to generate synthetic samples of sensor signals, the generative machine learning system can be understood as needing to be trained based on a sample set of sensor signals, such as those recorded from physical reality. The generative machine learning system is then able to "model" physical reality using the empirical distribution of the sensor signals used to train it.

[0011] A generative machine learning system can be understood as being configured to accept samples from a noise distribution as input and provide an output representing a synthesized sensor signal. The output representing the synthesized sensor signal can be further understood as including the synthesized sensor signal or an output composed of synthesized sensor signals. To configure the generative machine learning system to achieve this, it can be specifically trained to make samples from the noise distribution become samples from the sensor signal distribution. For training, the distribution of sensor signals can, in particular, be represented by empirical samples of sensor signals recorded from physical reality.

[0012] Typically, generative machine learning systems can provide their output unconditionally (e.g., using only samples of the noise distribution as input) or conditionally (e.g., using additional information related to what should be represented in the synthesized sensor signal as input). The method itself is independent of the actual input to the generative machine learning model. For example, the generative machine learning model may include or consist of: normalized flow, continuous normalized flow, conditional normalized flow, neural ODE obtained through, for example, flow matching, conditional flow matching, variational flow matching, conditional variational flow matching, diffusion model, or conditional diffusion model.

[0013] This method utilizes Bayesian inference to determine the uncertainty about its output. Specifically, Bayesian inference can model the parameters of a generative machine learning system based on its parameter distribution. Alternatively, and when using a neural network as the generative machine learning system, or when using a neural network within a generative machine learning system, the original "point estimate" of the generative machine learning system's parameters can be transformed into a distribution of the generative machine learning system's parameters using a final Laplace approximation layer. Advantageously, this method can therefore be used "post-hoc" with any generative machine learning system; that is, the method does not require the generative machine learning system to use the distribution of its parameters.

[0014] The posterior prediction of the distribution of latent features characterizing a sensor signal can be understood as the distribution of latent features being the posterior prediction distribution. Alternatively, characterization may involve processing methods of the latent feature distribution, such as scaling the latent distribution or processing (e.g., normalizing or scaling) the latent features before determining their distribution.

[0015] Obtaining noise samples can be understood as extracting noise samples from a noise distribution. Alternatively, noise samples can also be provided as input to the method, for example, as input to the sensor signal to be synthesized later, but it should first be evaluated whether the corresponding noise samples will produce a real sensor signal.

[0016] Typically, common generative machine learning systems sample their output (e.g., sensor signals) using noisy samples as input. Therefore, methods for determining uncertainty can be viewed as checks to determine whether a sensor signal sampled using specific noisy samples as input to a generative machine learning system will be uncertain; in other words, whether the sensor signal is likely to be true given the training data of the generative machine learning system.

[0017] In a preferred embodiment, determining the predicted posterior distribution may further include the following steps: Monte Carlo sampling was used to extract multiple synthetic sensor signals from the generative machine learning system. Determine the corresponding potential features of the synthesized sensor signal; The distribution of the latent features is determined based on the identified latent features; The distribution of the latent features is provided as the predicted posterior distribution.

[0018] In other words, sensor signals from a machine learning model can be sampled to approximate a posterior predictive distribution. To do this, the sampled sensor signals are then processed to extract latent features characterizing the respective sensor signals. The distribution of these latent features can then be determined based on these samples.

[0019] Advantageously, these preferred embodiments do not introduce a propagation probability distribution through generative processes such as BayesDiff. Samples of the parameters (also known as weights) of the generative machine learning model can be randomly sampled and then used as fixed parameters in the model to determine the synthetic sensor signal. The determined sensor signal can then be directly used to extract latent features. In this way, determining the posterior prediction distribution is substantially more computationally efficient than known methods.

[0020] In a preferred embodiment, the latent features are determined by a feature extractor, wherein the feature extractor is configured to accept sensor signals as input and provide latent features of the sensor signals as output.

[0021] Feature extractors for extracting features from provided sensor signals are known in the art, especially feature extractors capable of learning feature extraction in an unsupervised manner. Common methods include (variational) autoencoders, (normalized) stream or diffusion models. However, feature extractors can also be trained in a supervised manner, for example, through contrastive learning. For example, when using images as sensor signals to be output by a generative machine learning system, the inventors found that the visual model of the CLIP model performed well as a feature extractor. However, CLIP-like methods can also be used for other modalities of sensor signals. For example, any type of sensor signal can be used in CLIP-like methods that use a general model as an equivalent of the "visual model," and can then be trained using standard CLIP methods (e.g., using pairs of sensor signals and corresponding textual descriptions of the sensor signals as input).

[0022] Training the feature extractor can be an optional further step in this method.

[0023] In particular, the distribution of latent features can be characterized by the following formula: , in M This refers to the number of samples used during the Monte Carlo sampling period. This is the first sample taken during the Monte Carlo sampling period. m Potential features identified from a sample z is a noise sample. D It is the training dataset for generative machine learning systems, and It is an optional offset of the covariance matrix (either as a scalar or as a matrix that includes offsets for each element of the covariance matrix).

[0024] In another aspect, the present invention relates to a computer-implemented method for creating or expanding a dataset for training and / or testing a machine learning system, wherein the method comprises the following steps: Extract noise samples from the noise distribution; The noise sample is provided to the method described above for determining uncertainty; If the uncertainty is equal to or lower than a predefined threshold, then: Sensor signals are synthesized from a generative machine learning system used in methods for determining uncertainty, wherein the noise samples are used as input to synthesize the sensor signals; Synthesized sensor signals are added to the dataset to create or expand the training dataset.

[0025] Advantageously, this method allows the inclusion of only sensor signals that are actually “real” relative to sensor signals sampled from physical reality into the dataset for training a generative machine learning system. Therefore, this results in training and / or testing the machine learning model using only high-quality sensor signals; in other words, sensor signals that would cause artifacts that make the sensor signals unrealistic relative to the real world are excluded from the dataset. For the training dataset, this results in training the machine learning system using only high-quality synthetic sensor signals, which in turn improves the performance of the machine learning system. For testing, the test results are more reliable because they only involve sensor signals that are likely to actually occur in the real world and do not constitute artifacts.

[0026] If the variability measure does not satisfy the if statement, new noise samples can be extracted, and the steps of the method can be repeated.

[0027] "Synthesizing sensor signals from a generative machine learning system used in the method for determining uncertainty" can be understood as using the same generative machine learning system used to assess uncertainty. This can also be understood as the steps used to assess uncertainty being part of the method used to create and / or expand the dataset.

[0028] If the uncertainty is equal to or below the predefined threshold, a sensor signal is synthesized from the generative machine learning system using noise samples as input. For Bayesian models, the mean and mode of the parameters (e.g., the expected value of each parameter) can be selected as point estimates so that these "frozen" parameters can then be used to determine the synthesized sensor signal. If the generative machine learning system is provided with point estimates of the parameters, and, for example, the last layer of Laplace approximation is used for Bayesian inference, then the point estimates of the parameters (i.e., their original parameters) provided in the generative machine learning system can be used to determine the synthesized sensor signal using noise samples as input.

[0029] In another aspect, the present invention relates to a computer-implemented method for training and / or testing a machine learning system, comprising the following steps: The training and / or testing datasets are created according to the methods used to create and / or augment the datasets; The machine learning system is trained and / or tested using the dataset.

[0030] As part of this method, the sensor signals used in the dataset can also be annotated by human annotators or by automated annotation methods. The machine learning system can then be trained in a supervised manner.

[0031] In different embodiments of all the methods described above, the sensor signal can be a digital image or an audio signal. The machine learning system is trained on this data.

[0032] In different embodiments of all the methods described above, the variability measure of the posterior prediction distribution may be the entropy of the posterior prediction distribution, or the variability may be determined based on the posterior prediction distribution using a pairwise distance estimator. Attached Figure Description

[0033] Embodiments of the invention will be discussed in more detail with reference to the following figures. The figures show: Figure 1 A flowchart of a method for determining the uncertainty of sensor signals synthesized from a generative machine learning system; Figure 2 A schematic plan of methods for creating datasets; Figure 3 A training system for training a machine learning system using the dataset; Figure 4 The control system, including the machine learning system that controls actuators in its environment; Figure 5 The control system controls at least partially autonomous robots; Figure 6 The control system controls the manufacturing machines. Detailed Implementation

[0034] Figure 1 A flowchart is shown of a computer-implemented method (900) for determining the uncertainty of a sensor signal synthesized by a generative machine learning system, the uncertainty relating to the probability of observing the sensor signal in physical reality.

[0035] In this method, a generative machine learning system is obtained as input. The generative machine learning system is configured to accept noisy samples as input and provide synthesized sensor signals as output. Optionally, the generative machine learning system may also be trained (not shown) during a step of the method. The generative machine learning system may, in particular, be a diffusion model, especially a stable diffusion model or a diffusion model adapted by an adapter such as ControlNet or LoRA. Specifically, the generative machine learning model may consist of or may include a neural network for generating the sensor signals.

[0036] exist Figure 1 In the specific embodiments depicted, the generated sensor signal is a digital image, but other modes of the sensor signal are also possible in this method.

[0037] In the first step (901) of the method, noise samples are obtained. This can be achieved by accepting noise samples as input or by sampling noise samples from a predefined probability distribution (particularly a normal distribution, and even more particularly a multivariate normal distribution). The noise samples can be given in the form of real-valued vectors, i.e., they can be sampled from real-valued multivariate probability or density distributions.

[0038] In the second step (902) of the method, the noise samples are used as input to a generative machine learning model to perform Bayesian inference on the generative machine learning system. As a result of the Bayesian inference, a posterior prediction distribution of the sensor signal that would otherwise be synthesized by the generative machine learning system using the noise samples as input is determined. The posterior prediction distribution is configured to characterize the distribution of latent features of the sensor signal that would otherwise be generated using the noise samples as input. This can be achieved, in particular, by performing Monte Carlo sampling on the generative machine learning system to extract multiple samples of the synthesized sensor signal, extracting corresponding latent features from said multiple samples, thereby determining samples of latent features. Based on these samples, the probability distribution of the latent features can be determined, for example, by using a predefined family of probability distributions for maximum likelihood estimation.

[0039] In the third step, a measure of the variability of the posterior prediction distribution is provided as an uncertainty regarding the sensor signal synthesized based on the noise samples.

[0040] Figure 2 This schematically illustrates how the method of claim 1 can be used in method (1000) to create or expand a dataset (T). Noise samples (z) are derived from a probability distribution ( dThe probability distribution is extracted from the noise sample (z), which can particularly match the probability distribution used to train the generative machine learning system (61). Noise samples (z) are provided as input to the generative machine learning system (61). In this embodiment, the generative machine learning system (61) is a stable diffusion model configured to generate images based on the noise sample (z), but other generative machine learning systems and corresponding modalities of sensor signals are also possible. To perform Bayesian inference, a last-layer Laplace approximation is used in this embodiment. Multiple sensor signals are sampled from the generative machine learning system (61) by Monte Carlo sampling of the weights and using these weights for inference. (That is, images). Images The image is provided as input to the feature extractor (62) to extract the corresponding image. Potential characteristics In this embodiment, the feature extractor is provided by the visual model of the CLIP model. Other feature extractor models are also possible.

[0041] Then the extracted multiple latent features It can be used in the uncertainty module (63) to determine the information about the noise sample ( z The uncertainty of the sensor signal (i.e., the image) generated. The uncertainty module (63) can estimate the latent features, in particular, for example, using maximum likelihood estimation. The distribution. In particular, latent features. It can be used to estimate the mean and covariance matrix of a multivariate normal distribution. Specifically, this estimation can be achieved using the following formula: , in M This refers to the number of samples used during the Monte Carlo sampling period. These are the latent features identified for the m-th sample drawn during the Monte Carlo sampling period. Let z be a noisy sample, and D be the training dataset of the generative machine learning system. It is an optional offset of the covariance matrix.

[0042] The uncertainty module (63) can then determine the measure of the variability of this distribution (also known as the posterior prediction distribution) as uncertainty. uPreferably, the entropy of the posterior prediction distribution can be provided as uncertainty, but other methods such as pairwise distance estimators can also be used to measure the variability of the posterior prediction distribution. Up to this step, method (1000) can be considered as an embodiment of method (900) for determining uncertainty. Then, method (1000) can further continue to determine whether the uncertainty is equal to or below a predefined threshold (th). If the uncertainty ( u If the value is equal to or lower than the predefined threshold, the image determined for the noise sample (z) (i.e., the image generated using the noise sample (z) as input and the expected value of the point estimate of the parameters of the generative machine learning system (61) or the posterior prediction distribution of the generative machine learning system (61) itself) is input into the dataset (T), thereby creating and / or augmenting the dataset (T).

[0043] Figure 3 An embodiment of a training system (140) for training a machine learning system (60) using a dataset (T) as a training dataset (T) is shown. The training dataset (T) includes multiple input signals used to train the machine learning system (60). (i.e., image), where for each input signal The training dataset (T) also includes the desired output signal. It corresponds to the input signal And characterizes the input signal The associated classification or regression results (i.e., continuous values).

[0044] For training, the training data unit (150) accesses a computer-implemented database (St2), which provides a training dataset (T). The training data unit (150) preferably randomly determines at least one input signal from the training dataset (T). and corresponding to the input signal The expected output signal and input signal The signal is transmitted to the machine learning system (60). The machine learning system (60) is based on the input signal. Determine the output signal .

[0045] Desired output signal and the determined output signal It is transferred to the modification unit (180).

[0046] Then, based on the desired output signal and the determined output signal The modification unit (180) determines new parameters for the machine learning system (60). For this purpose, the modified unit (180) uses a loss function to compare the desired output signal. and the determined output signal The loss function determines the first loss value, which characterizes the determined output signal. Deviation from desired output signal How far. In the given embodiment, the negative log-likelihood function is used as the loss function. Other loss functions may also be conceived in alternative embodiments.

[0047] Furthermore, it can be conceivable that the determined output signal and desired output signal Each includes, for example, multiple sub-signals in tensor form, where the desired output signal is... The sub-signal corresponds to the determined output signal. The first sub-signal. For example, one might imagine a machine learning system (60) configured for object detection, where the first sub-signal characterizes the object relative to the input signal. The probability of one of the occurrences is given, and the second sub-signal characterizes the exact location of the object. If the output signal is determined... and desired output signal If there are multiple corresponding sub-signals, then preferably a second loss value is determined for each corresponding sub-signal using a suitable loss function, and the determined second loss values ​​are appropriately combined, for example, by weighted summation, to form a first loss value.

[0048] The modification unit (180) determines the new parameters based on the first loss value. In a given embodiment, this is accomplished using a gradient descent method (preferably stochastic gradient descent, Adam, or AdamW). In a further embodiment, training may also be based on an evolutionary algorithm or a second-order method used to train the neural network.

[0049] In other preferred embodiments, the described training is iteratively repeated a predefined number of iterations, or iteratively repeated until a first loss value falls below a predefined threshold. Alternatively or additionally, it may be conceivable to terminate training when the average first loss value on the test or validation dataset falls below a predefined threshold. New parameters determined in previous iterations are used in at least one iteration. Parameters used as machine learning system (60) .

[0050] Furthermore, the training system (140) may include at least one processor (145) and at least one machine-readable storage medium (146) containing instructions that, when executed by the processor (145), cause the training system (140) to perform a training method according to one aspect of the present invention.

[0051] Figure 4 An embodiment of a control system (40) for using a machine learning system (60) to control an actuator (10) or a display (10a) is shown. The actuator (10) and its environment (20) will be collectively referred to as the actuator system. At preferably uniformly spaced time points, a sensor (30) senses the condition of the actuator system. The sensor (30) may include several sensors. Preferably, the sensor (30) is an optical sensor that captures an image of the environment (20). The output signal (S) of the sensor (30) (or, in the case where the sensor (30) includes multiple sensors, the output signal (S) of each sensor) is transmitted to the control system (40), which encodes the sensed condition.

[0052] Therefore, the control system (40) receives the sensor signal (S) stream. It then calculates a series of control signals (A) based on the sensor signal (S) stream and transmits the control signals (A) to the actuator (10).

[0053] The control system (40) receives the sensor signal (S) stream from the sensor (30) in an optional receiving unit (50). The receiving unit (50) converts the sensor signal (S) into an input signal ( x Alternatively, in the absence of a receiving unit (50), each sensor signal (S) can be directly used as an input signal. x Input signal () x The signal (S) can be given, for example, as a segment (excerpt) from the sensor signal (S). Alternatively, the sensor signal (S) can be processed to generate the input signal (S). x In other words, the input signal is provided based on the sensor signal (S). x ).

[0054] Input signal ( x Then it is passed to the machine learning system (60).

[0055] The machine learning system (60) consists of parameters Parameterization, parameters Stored in parameter storage device And by parameter storage device supply.

[0056] Machine learning systems (60) learn from input signals ( x Determine the output signal ( y Output signal () y This includes assigning one or more tags to the input signal. x Information about the output signal. yThe signal is transmitted to an optional conversion unit (80), which then outputs the signal. y The signal is converted into a control signal (A). The control signal (A) is then transmitted to the actuator (10) to control the actuator (10) accordingly. Alternatively, an output signal (A) is generated. y It can be directly used as a control signal (A).

[0057] The actuator (10) receives a control signal (A), is controlled accordingly, and performs an action corresponding to the control signal (A). The actuator (10) may include control logic that transforms the control signal (A) into another control signal, which is then used to control the actuator (10).

[0058] In another embodiment, the control system (40) may include a sensor (30). In a further embodiment, the control system (40) may alternatively or additionally include an actuator (10).

[0059] In yet another embodiment, it is conceivable that the control system (40) controls the display (10a) in addition to the actuator (10) or the actuator (10).

[0060] Furthermore, the control system (40) may include at least one processor (45) and at least one machine-readable storage medium (46) thereon storing instructions which, if executed, cause the control system (40) to perform a method according to one aspect of the invention.

[0061] Figure 5 An embodiment is shown in which a control system (40) is used to control a robot that is at least partially autonomous, such as a vehicle (100) that is at least partially autonomous.

[0062] The sensor (30) may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors. Some or all of these sensors are preferably, but not necessarily, integrated into the vehicle (100).

[0063] The machine learning system (60) can be configured to be based on the input image ( x This is used to detect objects near a robot that is at least partially autonomous. The output signal ( y This can include information characterizing the location of the object near the at least partially autonomous robot. Control signals (A) can then be determined based on this information, for example, to avoid collisions with the detected object.

[0064] The actuator (10), preferably integrated in the vehicle (100), can be provided by the vehicle (100)'s brakes, propulsion system, engine, transmission, or steering mechanism. The control signal (A) can be determined to cause the actuator (10) to be controlled so that the vehicle (100) avoids collisions with detected objects. The detected objects can also be classified according to what the machine learning system (60) deems most likely to be (e.g., pedestrians or trees), and the control signal (A) can be determined based on this classification.

[0065] Alternatively or additionally, the control signal (A) may also be used to control the display (10a), for example, to display objects detected by the machine learning system (60). It is also conceivable that the control signal (A) could control the display (10a) such that it generates a warning signal if the vehicle (100) is about to collide with at least one detected object. The warning signal could be an audible warning and / or tactile signal, such as vibration of the vehicle's steering wheel.

[0066] In another embodiment, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving, or walking. The mobile robot may, in particular, be a lawnmower or a cleaning robot that is at least partially autonomous. In all the above embodiments, the control signal (A) may be determined to control the propulsion unit and / or steering mechanism and / or brakes of the mobile robot, enabling the mobile robot to avoid collisions with the identified object.

[0067] In another embodiment, at least partially autonomous robotic operation may be provided by a gardening robot (not shown) that uses sensors (30) (preferably optical sensors) to determine the state of the plants in the environment (20). Actuators (10) may control nozzles and / or cutting devices, such as blades, for spraying liquid. Based on the identified plant species and / or the identified plant state, control signals (A) may be determined to cause the actuators (10) to spray and / or cut the plants with an appropriate amount of suitable liquid.

[0068] In a further embodiment, at least partially autonomous robots may be provided by household appliances (not shown), such as washing machines, stoves, ovens, microwave ovens, or dishwashers. Sensors (30) (e.g., optical sensors) can detect the state of an object, which is then processed by the household appliance. For example, in the case of a washing machine, sensor (30) can detect the state of the clothes inside the washing machine. A control signal (A) can then be determined based on the detected clothing material.

[0069] Figure 6One embodiment is shown in which a control system (40) controls a manufacturing machine (11) of a manufacturing system (200), such as a punch cutter, cutter, gun drill, or clamp, for example as part of a production line. The manufacturing machine may include transport equipment, such as a conveyor belt or assembly line, which moves manufactured products (12). The control system (40) controls actuators (10), which in turn control the manufacturing machine (11).

[0070] The sensor (30) can be provided by an optical sensor that captures, for example, the properties of the manufactured product (12). Therefore, the machine learning system (60) can be understood as an image classifier.

[0071] The machine learning system (60) can determine the position of the manufactured product (12) relative to the transport equipment. The actuator (10) can then be controlled based on the determined position of the manufactured product (12) for subsequent manufacturing steps of the manufactured product (12). For example, the actuator (10) can be controlled to cut the manufactured product at a specific location on the manufactured product itself. Alternatively, it is conceivable that the machine learning system (60) classifies whether the manufactured product is damaged and / or exhibits defects. The actuator (10) can then be controlled to remove the manufactured product from the transport equipment.

[0072] The term "computer" can be understood to encompass any device used to process predefined computational rules. These computational rules can be in the form of software, hardware, or a combination of both.

[0073] Typically, "multiple" can be understood as being indexed, meaning that each element in the multiple elements is assigned a unique index, preferably by assigning consecutive integers to the elements contained in the multiple elements. Preferably, if the multiple comprises N elements, where N is the number of elements in the multiple, then the elements are assigned integers from 1 to N. It can also be understood that the elements in the multiple elements can be accessed through their indices.

Claims

1. A method for determining sensor signals synthesized by a generative machine learning system (61) Uncertainty u The computer-implemented method (900) of the uncertainty ( u This involves observing the sensor signals in physical reality. The method includes the following steps: (901) noise samples (z) were obtained; Using the noise sample (z) as input to the generative machine learning system (61), perform (902) Bayesian inference on the generative machine learning system (61) to determine the posterior prediction distribution of the sensor signal that would otherwise be synthesized by the generative machine learning system (61) using the noise sample (z) as input to the generative machine learning system (61). Provides a measure of the variability of the posterior prediction distribution described in (903) as a measure of the variability of the sensor signal synthesized from the noisy sample (z). Uncertainty The method is characterized by: The posterior predictive distribution characterizes the sensor signal that would otherwise be generated using the noise sample (z) as input. Potential characteristics The distribution of .

2. The method (900) according to claim 1, wherein determining the predicted posterior distribution comprises the following steps: Monte Carlo sampling was used to extract multiple synthetic sensor signals from the generative machine learning system (61). ; Determine the synthesized sensor signal The corresponding potential features ; Based on the identified latent features Determine the latent features Distribution; Provide the distribution of the latent features As the predicted posterior distribution.

3. The method (900) according to claim 2, wherein the latent feature Determined by a feature extractor (62), wherein the feature extractor (62) is configured to accept sensor signals as input. And provide the sensor signal Potential characteristics As output.

4. The method (900) according to any one of the preceding claims, wherein the latent feature The distribution of is characterized by the following formula: ， in M This refers to the number of samples used during the Monte Carlo sampling period. This is the first sample drawn during the Monte Carlo sampling period. m Potential features identified in a sample z is a noise sample. D It is the training dataset of the generative machine learning system, and It is an optional offset of the covariance matrix.

5. A computer-implemented method (1000) for creating or expanding a dataset (T) for training and / or testing a machine learning system (60), wherein the method comprises the following steps: From noise distribution ( d Extract noise samples ( z ); The noise sample ( z The method (900) according to any one of claims 1 to 4 is provided to determine the uncertainty; If the uncertainty is equal to or lower than a predefined threshold (th), then: Sensor signals are synthesized from the generative machine learning system (61) used in the method according to any one of claims 1 to 4, wherein the noise samples are used as input to the generative machine learning system (61) to synthesize the sensor signals; The synthesized sensor signals are added to the dataset (T) to create or expand the dataset (T).

6. A computer-implemented method for training and / or testing a machine learning system (60), comprising the following steps: Create training and / or test datasets (T) according to claim 5; The machine learning system (60) is trained and / or tested using the dataset (T).

7. The method according to any one of the preceding claims, wherein the sensor signal It is a digital image or audio signal.

8. The method according to any one of the preceding claims, wherein the variability of the measure of the variability of the posterior prediction distribution is the entropy of the posterior prediction distribution, or wherein the variability is determined based on the posterior prediction distribution using a pairwise distance estimator.

9. A training system (140) configured to perform the training method according to any one of claims 6.

10. A control system (40) configured to determine a control signal (A) based on a classification of a machine learning system according to claim 6, wherein the control signal (A) is configured to control an actuator (10) and / or a display (10a).

11. A computer program configured to cause a computer to perform the method and all steps thereof according to any one of claims 1 to 8 if the computer program is executed by a processor (45, 145).

12. A machine-readable storage medium (46, 146) having a computer program as claimed in claim 11 stored thereon.

Images