Device and method for improving robustness against "adversarial examples"
By introducing a smooth classifier g into the classifier, using Gaussian perturbation and Monte Carlo estimation to evaluate robustness, the classifier's easy fooling problem under small perturbations is solved, ensuring the correct classification in a noisy environment, and improving the safety and reliability of the system.
Patent Information
- Application Number
- CN202080012339.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-02-04
- Filing Date
- 2020-01-10
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2040-01-10
AI Technical Summary
Existing classifiers are easily fooled when faced with small disturbances, lack effective robustness assessment methods, especially in safety-critical applications such as highly automated driving, which may lead to system errors.
Using smooth classifier g, by adding random Gaussian perturbation to the input signal, using the prediction results of the basic classifier f, calculate the probability distribution of the input signal under noise, determine the robustness value, and combine Monte Carlo estimation and Gaussian cumulative distribution function to evaluate the robustness of the classifier.
Proof-proven robustness boundaries for smooth classifiers are provided, ensuring the correctness of the classifier in a noisy environment, and improving the safety and reliability of the system.
Smart Images
Figure CN113348475B_ABST
Abstract
Description
[0001] The present invention relates to a method for evaluating the robustness of a smoothed classifier, a method for evaluating the overall robustness of a smoothed classifier, a method for training a smoothed classifier, a method for operating a smoothed classifier, a method for providing an actuator control signal, a computer program and a machine-readable storage medium. Existing technology
[0002] In safety-critical applications like eg highly automated driving, it is important to select which further actions of the system depend on whether the classification and / or semantic segmentation of the input signal is correct.
[0003] However, it is well known that classifiers such as neural network classification systems can be easily fooled without appropriate countermeasures. It is also known that classifiers based on deep learning can be sensitive to small perturbations. In order to deploy such systems in the physical world, it is important to provide proof of the system's robustness.
[0004] "Certified Robustness to Adversarial Examples with DifferentialPrivacy" (arXiv preprint arXiv:1802.03471v3, 2018, Mathias Lecuyer, VaggelisAtlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana) and "Second-Order AdversarialAttack and Certifiable Robustness" (arXiv preprint arXiv:1809.03113v1, 2018, Bai Li, Changyou Chen, Wenlin Wang, Lawrence Carin) disclose a randomization technique for creating a classifier based on a given f Create a provably robust classifier that maps input signals to classes Classifier f Also referred to in this context as Base classifier They also present provably robust bounds for such smooth classifiers.
[0005] Advantages of the present invention
[0006] Smoothing Classifier g It can be defined as follows: When presented with input x When the smoothing classifier g In the presence of noise, especially in the case of input signals xUnder random Gaussian perturbations, return the base classifier f Most likely prediction:
[0007]
[0008] where represents a predefined variance .
[0009] To ensure that the classification of the smoothed classifier g is correct, it is important to ensure robustness. The method with the features of independent claim 1 generates a robustness value (also known as Robustness bound ) which ensures that the classification generated by the smoothed classifier g is the same for all input vectors within a sphere of radius x around the input signal . SUMMARY OF THE INVENTION
[0010] Accordingly, in a first aspect, the present invention relates to a computer-implemented method for evaluating the robustness of a smoothed classifier ( g ) for classifying sensor signals received from a sensor (30), the method comprising the steps of: g - determining an input signal depending on the sensor signal
[0011] - determining a first value (
[0012] - by the smoothed classifier ( g ), the first value ( p c , p , B ,
[0013] , B , c , A , A , x , g , x , p , B , , B , c , A , A , p , p , c ,
[0012] ), the first value ( p A ) characterizing the probability that the input signal ( x ) will be classified as belonging to a first class ( c A ) of a predefined plurality of classes when subject to noise, wherein the first class ( c A ) is the most likely class,
[0013] - determining a second value ( p B ) by the smoothed classifier (g), the second value ( p B ) characterizing the probability that the input signal ( x ) will be classified as belonging to a second class ( c B ) of the predefined plurality of classes when subject to the noise, wherein the second class ( c B) is the second most likely class,
[0014] - According to the first value ( p A ) at the first inverse value of the standard Gaussian cumulative distribution function ( ) and / or depending on the second value ( p B ) at the second inverse value of the standard Gaussian cumulative distribution function ( ), determine the robustness value .
[0015] Then it can be determined that if and only if the robustness value When it is greater than a predefined threshold, the smoothing classifier ( g ) is robust.
[0016] For example, the predefined threshold may be given by an amount characterizing noise in the sensor signal.
[0017] In a preferred embodiment, the robustness value and Determined in proportion, preferably .
[0018] If it is chosen equal to
[0019]
[0020] Then the bound is tight. This bound is substantially larger and therefore more useful than previously known bounds. In mathematical terms, p A It is aimed at The lower bound, p B It is aimed at To calculate the upper bound of the input signal x The surrounding robustness value , it is necessary to calculate p A and p B However, if the input signal is high-dimensional, it is impossible to calculate exactly The distribution of possible classes This is because in order to calculate exactly , it is necessary to have in all possible states Calculated above , and along the collection Integral for Gauss.
[0021] Good approximations can be obtained by Monte Carlo estimation to construct bounds p Aand p B , which, with arbitrarily high probability over the Monte Carlo samples satisfies the following equation
[0022] .
[0023] When estimating p A and p B while identifying the top class c A is potentially inefficient. A simple implementation would compute c i for all classes and then would c A identify as the class with the largest p i . This is guaranteed to be exact, but computationally expensive.
[0024] One way is a two-step process. First, use the from n 0 samples to obtain a guess c A of the identity of the top class . Since a meaningful large radius can be proven only if puts almost all of its mass on the top class, it is possible that n 0 is made very small, e.g., n 0 = 3. Second, use the from n samples to obtain for a predefined value that satisfies (2) with probability p A and p B . It can be observed that it is much more typical for the mass of c A outside the top class to be completely assigned to one remaining class than to be evenly spread over all remaining classes. Thus, it is possible that p B is determined as , which is very efficient while being a very accurate upper bound.
[0025] In another aspect of the present invention, it is possible to apply the above method to a plurality of test input signals ( xi ), as the test input signal as described above ( x i ), it is possible to determine a plurality of test robustness values , each test robustness value corresponding to a test input signal ( x i ). Then, it is possible to determine an overall robustness value characterizing the set of the determined test robustness values , for example, its minimum value.
[0026] Then it can be determined that the smoothing classifier ( ) is robust if and only if the overall robustness value g is greater than a predefined threshold.
[0027] Alternatively, then, it is possible to determine whether the smoothing classifier ( ) is robust based on the relative magnitudes of the robustness values ( ) and the overall robustness value. The relative magnitude is an indicator of whether the input signal (x) is meaningfully represented by the test set, or in other words, whether the input signal (x) represents the test set. For example, it can be determined that the smoothing classifier ( g ) is robust if and only if the relative magnitude (e.g., the quotient ) is greater than a second predefined threshold. g ).
[0028] In a further aspect of the present invention, it is contemplated to use this determination of whether the smoothing classifier ( g ) is robust as a stopping criterion for training the smoothing classifier ( g ), where if the smoothing classifier ( g ) is determined to be not robust, training continues.
[0029] In yet a further aspect of the present invention, it is possible to use the relative magnitude for training during the operation of the smoothing classifier. It is contemplated to operate the smoothing classifier ( g ) such that it is determined whether the smoothing classifier ( g ) is robust using the above method, and if the smoothing classifier ( g ) is determined to be not robust based on the relative magnitude, the input signal (x) is sent to a remote server, the target classification of the input signal (x) is received from the remote server, and the smoothing classifier ( g ) is further trained using the input signal (x) and the target classification. For example, the target classification may have been provided by a human expert.Remote This can mean that the remote server and the computer executing the smoothing classifier are not integrated in a single unit.
[0030] In yet another aspect of the present invention, it is possible to use the robustness value of the smoothing classifier ( g ), determined by using the above method when provided with the input signal (x), and adjust the operating parameters of the sensor (30) depending on the robustness value, to use the robustness value as a trigger for closer inspection. In particular, the sensor (30) can be controlled to amplify areas classified as non-robust.
[0031] In yet another aspect of the present invention, it is possible to use the robustness value for the safe operation of an actuator: evaluate whether the smoothing classifier ( g ) is robust using the above method, and determine an actuator control signal (A) for controlling the actuator based on the result of the evaluation, in particular if the smoothing classifier ( g ) is considered non-robust due to the result of the evaluation, then determine the actuator control signal (A) such that the actuator operates in a safe mode.
[0032] Naturally, the above method is not limited in its application to automated driving. In other applications, it is conceivable to have a robot (100) and / or a manufacturing machine (200) and / or an access control system (300) in which the actuator control is at least partially autonomous.
[0033] Embodiments of the present invention will be discussed in more detail with reference to the following figures. The figures show:
[0034] Figure 1 A control system with a classifier that controls an actuator in its environment;
[0035] Figure 2 A control system that controls a robot (100) that is at least partially autonomous;
[0036] Figure 3 A control system that controls a manufacturing machine;
[0037] Figure 4 A control system that controls an automated personal assistant;
[0038] Figure 5 A control system that controls an access control system;
[0039] Figure 6 A control system that controls a supervision system;
[0040] Figure 7 A control system for controlling an imaging system;
[0041] Figure 8 An embodiment of the structure of the classifier;
[0042] Figure 9 An algorithm for determining a robustness value Flowchart of the algorithm;
[0043] Figure 10 An algorithm for determining a robustness value Flowchart of a part of an alternative algorithm;
[0044] Figure 11 Flowchart of a method for determining whether a classifier is robust;
[0045] Figure 12 Flowchart illustrating a method for determining whether classifier 60 is overall robust;
[0046] Figure 13 Flowchart illustrating a method for determining whether an input signal x is well represented in a test set;
[0047] Figure 14 Flowchart illustrating another method for operating classifier 60;
[0048] Figure 15 Flowchart illustrating another method for operating classifier 60;
[0049] Figure 16 Flowchart illustrating another method for operating classifier 60;
[0050] Figure 17 Flowchart illustrating a method for providing an actuator control signal A.
[0051] Description of the embodiment
[0052] Figure 1 Shown in the [description] is an embodiment of actuator 10 in its environment 20. Actuator 10 interacts with control system 40. Actuator 10 and its environment 20 will be collectively referred to as the actuator system. Preferably, at uniformly spaced distances, sensors 30 sense the condition of the actuator system. Sensors 30 may include several sensors. Preferably, sensors 30 are optical sensors that capture images of environment 20. The output signal S of sensors 30 (or, in the case where sensors 30 include multiple sensors, the output signal S of each sensor) is transmitted to control system 40, and this output signal S encodes the sensed condition.
[0053] Thus, the control system 40 receives a stream of sensor signals S. It then calculates a series of actuator control commands A depending on the stream of sensor signals S, which are then transmitted to the actuators 10.
[0054] The control system 40 receives the stream of sensor signals S of the sensors 30 in an optional receiving unit 50. The receiving unit 50 transforms the sensor signal S into an input signal x. Alternatively, in the absence of the receiving unit 50, each sensor signal S can be taken directly as the input signal x. The input signal x can be given, for example, as an excerpt from the sensor signal S. Alternatively, the sensor signal S can be processed to generate the input signal x. The input signal x includes image data corresponding to the images recorded by the sensors 30. In other words, the input signal x is provided according to the sensor signal S.
[0055] The input signal x is then passed to a classifier 60, which can be given, for example, by an artificial neural network.
[0056] The classifier 60 is parameterized by parameters, which are stored in a parameter storage device and provided by it.
[0057] The classifier 60 determines an output signal y according to the input signal x. The output signal y includes a top class c A and a robustness value . Naturally, the top class c A and the robustness value can correspond to the classification of the entire input signal x. Alternatively, it can be given as semantic segmentation, for example, by assigning classes to bounding boxes. The output signal y is transmitted to an optional conversion unit 80, which is configured to determine the actuator control commands A. The actuator control commands A are then transmitted to the actuators 10 for correspondingly controlling the actuators 10. Alternatively, the output signal y can be taken directly as the control command A.
[0058] The actuators 10 receive the actuator control commands A, are correspondingly controlled, and perform actions corresponding to the actuator control commands A. The actuators 10 can include control logic that transforms the actuator control commands A into additional control commands, which are then used to control the actuators 10.
[0059] In a further embodiment, the control system 40 can include the sensors 30. In yet a further embodiment, the control system 40 can alternatively or additionally include the actuators 10.
[0060] In still further embodiments, it is contemplated that the control system 40 controls the display 10a instead of the actuator 10.
[0061] In addition, the control system 40 may include a processor 45 (or processors) and at least one machine-readable storage medium 46 storing instructions thereon, which instructions, if executed, cause the control system 40 to perform a method according to one aspect of the present invention.
[0062] Figure 2 An embodiment is shown in which the control system 40 is used to control an at least partially autonomous robot, such as an at least partially autonomous vehicle 100.
[0063] The sensor 30 may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more position sensors (such as GPS). Some or all of these sensors are preferably but not necessarily integrated in the vehicle 100. Alternatively or additionally, the sensor 30 may include an information system for determining the state of the actuator system. An example of such an information system is a weather information system that determines the current or future state of the weather in the environment 20.
[0064] For example, using the input signal x, the classifier 60 may detect an object near the at least partially autonomous robot. The output signal y may include information characterizing where the object is located near the at least partially autonomous robot. Then a control command A may be determined based on this information, for example to avoid colliding with the detected object.
[0065] The actuator 10, which is preferably integrated in the vehicle 100, may be given by the brakes, propulsion system, engine, powertrain, or steering of the vehicle 100. An actuator control command A may be determined such that the actuator (or actuators) 10 is controlled so that the vehicle 100 avoids colliding with the detected object. The detected objects may also be classified according to what the classifier 60 thinks they are most likely to be - for example a pedestrian or a tree - and the actuator control command A may be determined depending on the classification.
[0066] In further embodiments, the at least partially autonomous robot may be given by another mobile robot (not shown), which may move, for example, by flying, swimming, diving, or walking. The mobile robot may in particular be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In all of the above embodiments, an actuator command control A may be determined such that the propulsion unit and / or steering device and / or brakes of the mobile robot are controlled so that the mobile robot can avoid colliding with the identified object.
[0067] In a further embodiment, the at least partially autonomous robot can be provided by a gardening robot (not shown) that uses sensors 30, preferably optical sensors, to determine the state of plants in the environment 20. The actuator 10 can be a nozzle for spraying chemicals. Depending on the type and / or state of the identification of the plant, an actuator control command A can be determined such that the actuator 10 sprays a suitable amount of a suitable chemical onto the plant.
[0068] In an even further embodiment, the at least partially autonomous robot can be provided by a household appliance (not shown), such as a washing machine, stove, oven, microwave oven, or dishwasher. The sensor 30, such as an optical sensor, can detect the state of an object that will undergo processing by the household appliance. For example, in the case where the household appliance is a washing machine, the sensor 30 can detect the state of the laundry inside the washing machine. Then, the actuator control signal A can be determined depending on the detected laundry material.
[0069] Figure 3 Shown is an embodiment in which a control system 40 is used to control a manufacturing machine 11 (such as a stamping tool, a cutting tool, or a gun drill) of a manufacturing system 200, for example as part of a production line. The control system 40 controls the actuator 10, which in turn controls the manufacturing machine 11.
[0070] The sensor 30 can be provided by an optical sensor that captures, for example, the characteristics of the manufactured product 12. The classifier 60 can determine the state of the manufactured product 12 based on these captured characteristics. Then, the actuator 10 that controls the manufacturing machine 11 can be controlled depending on the determined state of the manufactured product 12 for subsequent manufacturing steps of the manufactured product 12. Alternatively, it is conceivable that during the manufacture of the subsequent manufactured product 12, the actuator 10 is controlled depending on the determined state of the manufactured product 12.
[0071] Figure 4 Shown is an embodiment in which the control system 40 is used to control an automated personal assistant 250. The sensor 30 can be an optical sensor, for example for receiving video images of the gestures of the user 249. Alternatively, the sensor 30 can also be an audio sensor, for example for receiving voice commands of the user 249.
[0072] The control system 40 then determines an actuator control command A for controlling the automated personal assistant 250. The actuator control command A is determined based on the sensor signal S from the sensor 30. The sensor signal S is transmitted to the control system 40. For example, the classifier 60 can be configured to, for example, implement a gesture recognition algorithm to identify a gesture made by the user 249. The control system 40 can then determine the actuator control command A for transmission to the automated personal assistant 250. Then, it transmits the actuator control command A to the automated personal assistant 250.
[0073] For example, the actuator control command A can be determined based on the identified user gesture recognized by the classifier 60. Then, it can include information that causes the automated personal assistant 250 to retrieve information from a database and output the retrieved information in a form suitable for the user 249 to receive.
[0074] In another embodiment, it is contemplated that instead of the automated personal assistant 250, the control system 40 controls a household appliance (not shown) that is controlled based on the identified user gesture. The household appliance can be a washing machine, a stove, an oven, a microwave oven, or a dishwasher.
[0075] Figure 5 Shown is an embodiment in which the control system controls an access control system 300. The access control system can be designed to physically control access. For example, it can include a door 401. The sensor 30 is configured to detect a scenario related to determining whether access is to be granted. For example, it can be an optical sensor for providing image or video data and for detecting a person's face. The classifier 60 can be configured to, for example, interpret the image or video data by matching the identity with known persons stored in a database to determine the identity of the person. Then, depending on the interpretation of the classifier 60, for example, based on the determined identity, an actuator control signal A can be determined. The actuator 10 can be a lock that grants or denies access depending on the actuator control signal A. Non - physical, logical access control is also possible.
[0076] Figure 6 Shown is an embodiment in which the control system 40 controls a supervision system 400. This embodiment is largely equivalent to the embodiment shown in Figure 5 Therefore, only the different aspects will be described in detail. The sensor 30 is configured to detect the supervised scenario. The control system does not necessarily control the actuator 10, but rather controls a display 10a. For example, a machine learning system 60 can determine a classification of the scenario, such as whether the scenario detected by the optical sensor 30 is suspicious. The actuator control signal A transmitted to the display 10a can then be configured, for example, to cause the display 10a to adjust the displayed content depending on the determined classification, such as highlighting an object that the machine learning system 60 deems suspicious.
[0077] Figure 7 An embodiment of a control system 40 for controlling an imaging system 500 such as, for example, an MRI device, an x-ray imaging device, or an ultrasonic imaging device is shown. The sensor 30 can be, for example, an imaging sensor. The machine learning system 60 can then determine the classification of all or part of the sensed image. An actuator control signal A can then be selected based on the classification to control the display 10a. For example, the machine learning system 60 can interpret a region of the sensed image as a potential anomaly. In such a case, the actuator control signal A can be determined such that the display 10a displays the imaging and highlights the potential anomaly region.
[0078] Figure 8 The structure of one embodiment of the classifier 60 is schematically illustrated. The classifier 60 includes a smoothing classifier g and a base classifier f , such as, for example, a deep neural network. An input signal x is input into the smoothing classifier g , and the smoothing classifier g is configured to sample a plurality of random numbers from a standard Gaussian distribution having a predefined standard deviation and add them to the input signal x. For each random number , the base classifier f is configured to return a corresponding classification . Using these multiple classifications , the smoothing classifier g is then configured to determine the top class Figure 9 and the robustness value Figure 10 illustrated in, for example, by using the algorithms illustrated in c A and outputs the top class A and the robustness value c together as an output signal y. Due to the structure of the classifier 60, the smoothing classifier g and the classifier 60 can be used interchangeably.
[0079] Figure 9 A flowchart of an embodiment of a method for determining the robustness value is shown. First (901), a predefined number of samples n 0 are provided, and is sampled from a standard Gaussian distribution having a predefined standard deviation n 0 random numbers , which are added to the input signal x. For each random number , basic classifier f Return the corresponding category Then, the estimated top class Identified as belonging to these corresponding categories The most frequently occurring class.
[0080] Then (902), a predefined second number of samples is provided n , and from a Sampling from a standard Gaussian distribution n Random numbers , add it to the input signal x. For each random number , basic classifier f Return the corresponding category . By the top class of estimates Count the number of occurrences to calculate the estimated top class The relative frequency Here, the estimated top class The absolute frequency of occurrence is indicated as k .
[0081] Next (903), because of n The statistical sample of data points is subject to fluctuations, so it satisfies The lower bound p A is calculated to have at least The confidence lower bound of the probability of . This expression can be conveniently expressed by the Beta distribution with parameters k and n-k+1 The quantiles are given.
[0082] For example, the following formula can be used, where Is a parameter a and b The F distribution of 1 - c Quantile. Then, the lower bound p A It can be given by the following equation
[0083] .
[0084] Upper bound p B is calculated as .
[0085] Then (905) checks whether , in this case, the robustness value is calculated using equation (2) .if , the algorithm abstains and issues a default error message, for example by setting This ends the method.
[0086] Figure 10 Illustrated is an alternative embodiment of a method for determining top classes c A such that if the two top classes have similar quality, the method is less likely to abstain. First (911), sample a number of random numbers from a standard Gaussian distribution with a predefined standard deviation n and add them to the input signal x. For each random number , the base classifier returns the corresponding classification f . .
[0087] Then (912) the two top classes c A , c B are selected as the two most frequently occurring classes among these classifications . n A and n B indicate their absolute frequencies. Then, check whether a two-sided binomial hypothesis test extracted from the binomial returns a n A value less than . If this is the case, return the top class p , otherwise the algorithm abstains. c A .
[0088] Figure 11 Shown in is a flowchart of an embodiment of a method for determining whether classifier 60 is robust. First (1100), provide an input signal x derived from the sensor signal S. The input signal x may be noisy because the sensor signal S may be noisy. Then (1200), provide a predefined threshold . For example, it may be possible to provide the environmental conditions to which sensor 30 is subject, such as the temperature or humidity value of the air surrounding sensor 30. Then, the threshold can be determined depending on the said environmental conditions, for example by means of a look-up table. If sensor 30 is an image or video sensor, the threshold can be selected to characterize the noise generated by sensor 30, such as the standard deviation or statistical variance of the pixel values. Then, provide a robustness value for the input signal x. Then (1400), determine whether the robustness value If this is the case (1500), then it is determined that the classifier 60 is robust. If not (1600), then it is determined that the classifier 60 is not robust.
[0089] Figure 12 Shown is a flowchart of an embodiment of a method for determining whether the classifier 60 is generally robust. To this end, a test data set (2100) is provided that includes test input signals x i . Preferably, the test data set represents real-world input data. Then (2200), for each test input signal x i , for example, by using the method illustrated in Figure 9 and providing the test input signal x i as the input signal x, the corresponding robustness value is determined. Next (2300), by obtaining the value that characterizes the set of robustness values (e.g., the minimum value, i.e., ), the overall robustness value is determined from the set. Alternatively, the overall robustness value can be selected differently, e.g., as the median or average of all . Then (2400), it is determined whether the overall robustness value is greater than a second threshold , and this second threshold can be provided as the threshold in step (1200). If this is the case, the classifier 60 is considered to be generally robust (2500), and if not (2600), it is considered not to be generally robust.
[0090] Figure 13 Shown is a flowchart of an embodiment of a method for determining whether the input signal x is well represented in the test set and thus whether the classifier 60 trained using the test set is robust around the input signal x.
[0091] Steps (2100) and (2200) are equivalent to the steps in the method illustrated in Figure 12 , and steps (1100), (1200), and (1300) are equivalent to the steps in the method illustrated in Figure 11 . After determining the robustness value and the overall robustness value , it is checked (2600) whether they satisfy the inequality
[0092] .
[0093] If this is the case (2700), the input signal x is considered not to be well represented in the test set, and thus the classifier 60 trained using the test set is considered not to be robust. If this is not the case (2800), the input signal x is considered to be well represented in the test set, and thus the classifier 60 is considered to be robust.
[0094] Figure 14 Shown is a flowchart of an embodiment of a method for training a smoothed classifier g The method is conveniently implemented by training a base classifier g associated with the smoothed classifier and also simply referred to as "training classifier 60". First, the classifier 60 is trained (3100). Then, the overall robustness f is determined based on the test set used in the training, and it is determined whether the classifier 60 is overall robust. If this is the case, the method ends (3300). However, if this is not the case, the method branches back to (3100) and the training continues. Then, the actuator control signal (A) can be determined (916) according to the parameter vu, and the actuator (10) can be controlled according to the actuator control signal (A). For example, if the parameter vu indicates non-vulnerability, the actuator control signal (A) can be determined to correspond to the normal operation mode, while if the parameter vu indicates vulnerability, the actuator control signal (A) can be determined to correspond to the fail-safe operation mode, which, for example, by reducing the motion dynamics of the actuator (10).
[0095] Shown is a flowchart of an embodiment of a method for operating the classifier 60. First (4100), the classifier 60 is operated. When provided with a new input signal x, the method according to claim 9 is used to determine (4200) whether the classifier 60 is robust. If this is not the case, the method ends (4500). However, if this is the case, the new input signal x is sent (4300) to a remote server, where it can be presented to a human expert, for example, and the remote server receives a manual classification from the human expert as the target classification. Then, the target classification is received (4400) from the remote server, and the pair of the new input signal (x) and the received target classification is added to the training set, which can be used to train the classifier 60 before the operation is restored or at a later time point. The method then branches back to step (4100).
[0096] Figure 15
[0097] Figure 16 Shown is a flowchart of an embodiment of a method for operating classifier 60. First (5100), classifier 60 is operated. When provided with a new input signal x, it is determined (5200) whether classifier 60 is robust using the method according to claim 9. If this is the case, the method ends (5400). However, if this is not the case, the operating parameters of sensor 30 are adjusted (5300), in particular, the scaling of sensor 30 can magnify the region of the new input signal x for which classifier 60 is considered not to be robust.
[0098] Figure 17 Shown is a flowchart of an embodiment of a method for providing an actuator control signal A for controlling actuator 10 depending on the output signal y of classifier 60. First (6100), classifier 60 is operated. When provided with a new input signal x, it is determined (6200) whether classifier 60 is robust, for example by using Figure 9 the algorithm illustrated herein. The actuator control signal is now determined based on the result of the evaluation). If the operator 60 is considered to be robust (6300), the actuator control signal A is determined such that actuator 10 operates in the normal mode. However, if this is not the case, the actuator control signal (A) is determined such that the actuator (10) operates in the safe mode.
[0099] The term "computer" encompasses any device for processing predefined computing instructions. These computing instructions can be in the form of software, or can be in the form of hardware, or can also be in a hybrid form of software and hardware.
[0100] It is further understood that the processes can not only be implemented entirely in software as described. They can also be implemented in hardware, or in a hybrid form of software and hardware.
Claims
1. A computer-implemented method for evaluating the robustness of a smoothing classifier for classifying sensor signals received from a sensor for fail-safe operation of an actuator, the sensor sensing the condition of the actuator, the method comprising the steps of: - Providing an input signal depending on the sensor signal, - Determine a first value p by a smooth classifier A , the first value p A characterizes the probability that the input signal will be classified as belonging to a first class c in a predefined plurality of classes when subject to noise, A where the first class c A is the most likely class. -Determine a second value p by a smooth classifier B , where the second value p B represents the probability that the input signal will be classified as belonging to a second class c of the predefined plurality of classes when subject to the noise, B where the second class c B is the second most likely class. - based on the first value p A the first inverse value Φ of the standard Gaussian cumulative distribution function Φ at -1 (p A ) and / or depending on the second value p B the second inverse value Φ of the standard Gaussian cumulative distribution function Φ at -1 (p B ), determine the robustness value - depending on the robustness value Evaluating whether the smoothing classifier is robust and determining an actuator control signal for controlling the actuator based on the result of the evaluation, wherein if the smoothing classifier is considered not robust as a result of the evaluation, the actuator operates in a fail-safe mode.
2. The method according to claim 1, wherein the robustness value depends on the standard deviation σ and the first inverse value Φ -1 (p A ) minus the second inverse value Φ -1 (p B ) and is determined by the product of the difference.
3. The method according to any one of claims 1 and 2, wherein before determining the first value p A the method includes a step of estimating which class among the predefined plurality of classes is the first class c A .
4. The method according to claim 1 or 2, wherein the first value p A is a lower bound of the probability that the input signal will be classified as belonging to the first class c A when subject to the noise.
5. The method according to claim 1 or 2, wherein the second value p B is an upper bound of the probability that the input signal will be classified as belonging to the second class c B when subject to the noise.
6. The method according to claim 5, wherein the second value p B is determined as p B = 1 - p A .
7. A computer-implemented method for evaluating the overall robustness of a smooth classifier for classifying sensor signals received from sensors, the method comprising the steps of: - Providing a training set including test input signals - Determining a test robustness value for each of the test input signals by using the method according to any one of claims 1 to 6 and using the test input signal as the input signal instead of receiving it from the sensor, -The overall robustness is determined as a value characterizing the determined set of test robustness values .
8. The method according to claim 7, wherein based on the robustness value and the overall robustness value to determine whether the smoothing classifier is robust according to the relative magnitudes thereof.
9. A computer-implemented method for operating a system including a smoothing classifier and a sensor providing an input signal to the smoothing classifier, comprising the steps - When provided with the input signal, determine the robustness value of the smoothing classifier using the method according to any one of claims 1 to 6 -depending on the robustness value Adjust the operating parameters of the sensor.
10. A computer-implemented method for providing an actuator control signal for controlling an actuator depending on an output signal of a smoothing classifier, comprising the steps of: - Evaluating whether the smoothing classifier is robust using the method according to any one of claims 1 to 6, and determining the actuator control signal based on the result of the evaluation, and if the smoothing classifier is considered not robust due to the result of the evaluation, determining the actuator control signal such that the actuator operates in a safe mode.
11. The method according to claim 10, wherein the actuator controls at least a partially autonomous robot and / or a manufacturing machine and / or an access control system.
12. A computer program product configured to cause a computer to perform the method according to any one of claims 1 to 11 and all of its steps if the computer program is executed by a processor.
13. A machine-readable storage medium having stored thereon the computer program product according to claim 12.