Automatically quantifying robustness of object detection model applied to control task and / or monitoring task

Abstract

A computer-implemented method for automatically quantifying the robustness of an object detection model (f) applied to a control task and / or a monitoring task, comprising:-receiving (S1) an object detection model (f) trained so as to detect (S2) the object detection model (f) when image data (x) is input into the object detection model (f); the method comprises the steps of:-outputting a predicted object as a function of a position in the image data (x) and of an object category of a set of object categories,-applying (S2) a set of robustness requirements to an object detection model (f),-deriving (S3) from each robustness requirement a cross Lipschitz-nick function (g) which quantifies the robustness requirements conditioned by the object detection model (f) for the input image data (x), and-outputting (S3) the predicted object as a function of the object detection model (f) for the input image data (x). -determining (S4) a robustness value (RV) of the object detection model (f) for disturbed image data (xp) deviating from the undisturbed image data (x0) of the image data (x) by calculating a cross Lipschitz-Ness function, the cross Lipschitz-Ness function being integrated into a network robustness cross Lipschitz extremum CLEVER score of the object detection model (f), the cross Lipschitz extremum CLEVER score of the object detection model (f), the cross Lipschitz extremum CLEVER score of the object detection model (f) being a cross Lipschitz extremum CLEVER score of the cross Lipschitz extremum CLEVER score of the object detection model (f); -comparing (S5) the determined robustness value (RV) with a predefined robustness threshold (RT), and-if the robustness value (RV) is below the predefined robustness threshold (RT), outputting (S6) a positive proof (C) for applying the object detection model (f) in the control task and / or the monitoring task.

Description

Technical Field

[0001] The present disclosure relates to an aid and a computer-implemented method for automatically quantifying the robustness of an object detection model, preferably for an object detection model (f) applied to control and / or monitor an industrial manufacturing process or an autonomously driven vehicle. Background Art

[0002] There's a trend toward digitalization in industry. For example, the manufacturing process of a product can be digitally controlled. Consider complex factories, which often consist of different components, parts, modules, or units with multiple independent functions. These units and their functions must be interactively controlled and regulated. The increasing degree of digitalization allows, for example, the automated manufacturing or industrial installation of products in factory production lines by robotic or other autonomous units.

[0003] The manufacturing process itself must also be monitored and controlled. The quality of the resulting manufactured product must be monitored to identify early degradation of the product and to derive corrective control measures for the manufactured components, for example to adapt the settings of the components.

[0004] Artificial intelligence ("AI") models for object detection based on image data are deployed and operated in industrial environments as well as public transportation environments, for example, for control tasks, quality tasks, or surveillance tasks. These object detection models must be "industrial-grade."

[0005] Another application area for object detection methods is identifying and localizing objects on streets or in the surrounding environment based on sensor data and / or image data in autonomous vehicles or traffic control systems to identify obstacles or traffic volume that could affect traffic flow. This requires AI models to be reliable and robust, even when surrounding conditions such as lighting conditions or positioning may change. AI-based object detection models must be proven and published for use in public or private transportation, such as trains or cars.

[0006] A validation component for validating artificial intelligence is known from EP 4105846 A1, and a method for determining the influence of attributes in a recurrent neural network trained on treatment prediction is known from EP 3564862 A1.

[0007] Typically, the robustness of object detection models is quantified using heuristic search methods. However, object detection methods are usually based on very deep neural networks, and an exhaustive search with tens of thousands of forward passes requires a lot of computational resources. Furthermore, it is unclear how local robustness should be defined, i.e., local robustness with respect to a certain test sample. What can be done is to natively perturb a batch of images to different degrees. Then, based on common metrics such as mean average precision (mAP), one can decide at which perturbation level the mAP value is no longer acceptable, and report this threshold as proof of robustness.

[0008] This approach has several drawbacks: First, it only computes global robustness. It cannot be measured quantitatively on a sample-by-sample basis. Second, its computational complexity depends on the number of samples in the batch and the granularity of the heuristic search. Third, the threshold for acceptable mAP is arbitrary, and for this purely empirical approach, there are no theoretical guarantees that ensure meaningful proof. Importantly, it circumvents specifying a true definition of robustness by using only mAP as a proxy.

[0009] Although several methods have been applied to prove the robustness of classification models, especially for image input data, it is not yet known how to prove the robustness of object detection models. The main difference is that object detection models do not produce a single class prediction, but rather a set of bounding box coordinates with a corresponding probability distribution over the possible classes. Summary of the Invention

[0010] Therefore, the object of the present application is to provide a method and a test device that automatically provide robustness quantification with reduced computational complexity, which robustness quantification should be used for realistic and reliable certification standards for object detection models based on industrial-grade or security-grade deep neural networks.

[0011] This object is solved by the features of the independent claim. The dependent claims contain further developments of the invention.

[0012] A first aspect relates to a computer-implemented method for automatically quantifying the robustness of an object detection model applied to a control task and / or a surveillance task, comprising the following steps:

[0013] - receiving an object detection model, the object detection model being trained to, when image data is input into the object detection model, output a predicted object based on a location in the image data and an object class from a set of object classes,

[0014] -apply a set of robustness requirements to object detection models,

[0015] - derive a cross-Lipschitz-ness function from each robustness requirement that quantifies the robustness requirement conditioned on the object detection model used for the input image data,

[0016] - determining a robustness value of the object detection model by computing a cross-Lipschitz-ness function for perturbed image data that deviates from the undisturbed image data, the cross-Lipschitz-ness function being incorporated into a network robustness cross-Lipschitz extreme value CLEVER score of the object detection model,

[0017] - If the robustness value is below a predefined robustness threshold, a positive proof for applying the object detection model in a control task and / or a surveillance task is output.

[0018] This approach provides a realistic and detailed definition of robust object detection. A cross-Lipschitz-ness function is derived from each robustness requirement and integrated into the CLEVER framework. By using the cross-Lipschitz-ness function for this purpose, a reliable measure of the robustness of object detection models is provided, which requires a much smaller number of backward passes and therefore less processing power.

[0019] Each robustness requirement is defined by stating the scenarios for which the object detection model (f) is not considered robust.

[0020] This allows for a clear definition of the robustness requirements and an explicit formulation of the Lipschitz-ness function. Thus, clear limits can be formulated and provided.

[0021] According to a further embodiment, the object detection model outputs a class probability for each bounding box in a set of bounding boxes depending on the image data, wherein each bounding box specifies a location region in the image data, and wherein each bounding box whose class probability is higher than a predefined probability value is a predicted bounding box.

[0022] This object detection model provides features that are well-suited for specifying robustness analysis. This type of object detection model is widely used and provides reliable results. This type of well-known and applied object detection model is a model similar to or based on the YOLO model.

[0023] In a further embodiment according to the first robustness requirement, the object detection model is not robust if at least one predicted bounding box of the object detection model processed with the perturbed image data is misclassified compared to the prediction of the object detection model processed with the unperturbed image data.

[0024] This robustness requirement covers situations where perturbations in the image data cause object detection models to often provide incorrect classifications.

[0025] In a further embodiment according to the second requirement, if at least one bounding box output by the object detection model (f) for the unperturbed image data (x0) is ignored and is no longer output by the object detection model (f) as a predicted bounding box for the perturbed image data (xp), then the object detection model is not robust.

[0026] This provides a key criterion of robustness with respect to false negative predictions.

[0027] In a further embodiment according to the third robustness requirement, the object detection model is not robust if at least one bounding box is output by the object detection model as a predicted bounding box for the perturbed image data, but the same bounding box is not output by the object detection model as a predicted bounding box for the unperturbed image data.

[0028] This provides a key criterion for robustness with respect to false positive predictions.

[0029] In a further embodiment according to the fourth robustness requirement, the object detection model is not robust if at least one predicted bounding box output by the object detection model for the perturbed image data differs in size and position in the image data from the same predicted bounding box output by the object detection model for the unperturbed image data.

[0030] This requirement specifies that an object detection model is not robust if the location or size of the “same” bounding box is different when its output for perturbed image data is compared to unperturbed image data. This provides a key robustness requirement for robustness in predicting object locations.

[0031] According to a further embodiment, the consistency of at least one predicted bounding box output for the perturbed image data (xp) and at least one predicted bounding box output for the unperturbed image data (x0) is determined by an intersection over union functional unit.

[0032] The IoU functional unit is a commercially available unit that provides fast processing and requires only minor adaptation efforts. Thus, applying the IoU functional unit is cost-effective and processing-power-efficient.

[0033] According to a further embodiment, the perturbed image data are sampled from a hypersphere centered on the undisturbed image data.

[0034] Evaluating object detection models with this perturbed image data also considers adversarial attacks, which are based on perturbations in the image data that are mostly invisible to the human eye but cause the object detection model to output incorrect predictions. Here, an incorrect prediction means that the prediction has changed compared to the prediction for the unperturbed image data.

[0035] According to a further embodiment, all data elements of the image data (x) are undisturbed image data (x0), or

[0036] Only data elements of the image data (x) that are within at least one bounding box of the image data (x) are undisturbed image data (x0), and these data elements are used to determine the robustness value.

[0037] The data elements of image data are synonymously named pixels.

[0038] Considering only pixels of the image data that lie within the bounding box reduces the number of pixels to be processed and also reduces processing power.

[0039] According to a further embodiment, the object detection model is applied to control and / or monitor an industrial manufacturing process or an autonomously driven vehicle.

[0040] Therefore, object detection models have been shown to be useful in application environments where particularly reliable and robust models are required.

[0041] According to a further embodiment, if the robustness value is above a predefined robustness threshold, the received object detection model is retrained using a set of training image data, the set of training image data being selected to optimize the robustness value.

[0042] This allows optimizing object detection models, especially with respect to defects identified during robustness quantization.

[0043] According to a further embodiment, the method further comprises the following steps:

[0044] - receiving at least one further different object detection model, the model being trained to output at least one predicted object class,

[0045] - determine the robustness value of each received object detection model,

[0046] - outputs a list of robustness values ​​for all object detection models that have a positive proof,

[0047] - select one of the listed object detection models, and

[0048] - Using the selected object detection model for control tasks and / or surveillance tasks.

[0049] This allows selecting the most appropriate object detection model with respect to the robustness requirements of the considered control or surveillance task.

[0050] A second aspect relates to an attestation apparatus comprising at least one processor configured to:

[0051] - receiving an object detection model, the object detection model being trained to, when image data is input into the object detection model, output a predicted object based on a location in the image data and an object class from a set of object classes,

[0052] - Receive a set of robustness requirements for your object detection model,

[0053] - derive a cross-Lipschitz-ness function from each robustness requirement that quantifies the robustness requirement conditioned on the object detection model used for the input image data,

[0054] - determining a robustness value of the object detection model by computing a cross-Lipschitz-ness function for perturbed image data that deviates from the undisturbed image data, the cross-Lipschitz-ness function being incorporated into a network robustness cross-Lipschitz extreme value CLEVER score of the object detection model,

[0055] - If the robustness value is below a predefined robustness threshold, output a positive justification for applying the object detection model in a control task and / or a monitoring task, where each robustness requirement is defined by stating scenarios in which the object detection model (f) is not considered robust.

[0056] The attestation device provides a certification of the received object detection model under consideration in a process-optimized manner. The attestation device may be part of a certification system that evaluates the received object detection model according to further criteria related to certification of "industrial-grade" or "transportation-grade" object detection models. The attestation device may include an output interface directly coupled to a controller that applies the object detection model to control and / or monitoring tasks.

[0057] A third aspect relates to a computer program product directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps as described above when said product is run on said digital computer.

[0058] The computer program product may be stored on a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the steps described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention will be explained in more detail with reference to the accompanying drawings. Similar objects will be marked with the same reference numerals.

[0060] Figure 1 An embodiment of the method according to the invention is shown by a flow chart.

[0061] Figure 2 The output generated by the object detection model received in an embodiment of the method of the present invention is shown in schematic form.

[0062] Figure 3 An embodiment of a scenario corresponding to the first robustness requirement of the method of the present invention is shown in schematic form.

[0063] Figure 4 An embodiment of a scenario corresponding to the second or third robustness requirement of the method of the present invention is shown in schematic form.

[0064] Figure 5 An embodiment of a scenario corresponding to a fourth robustness requirement of the method of the present invention is shown in schematic form.

[0065] Figure 6 An embodiment of the process steps of the method of the invention is schematically shown, and

[0066] Figure 7 An embodiment of the inventive certification device in an industrial or transport environment is schematically shown. DETAILED DESCRIPTION

[0067] Note that in the detailed description of the following embodiments, the drawings are schematic only, and the elements shown are not necessarily shown to scale. On the contrary, the drawings are intended to illustrate the collaboration of functions and functions or components. Here, it should be understood that any connection or coupling of functional blocks, devices, components or other physical or functional elements can also be achieved by indirect connection or coupling, for example, via one or more intermediate elements. The connection or coupling of elements or components or nodes can be achieved, for example, by wired, wireless connection and / or a combination of wired and wireless connections. Functional units can be implemented by dedicated hardware (for example, processors, firmware) or software, and / or by a combination of dedicated hardware and firmware and software. It should also be noted that each functional unit described for the device can perform the functional steps of the related method, and vice versa.

[0068] Object detection models are or should be used in industrial environments to perform quality control, such as detecting misalignment on printed circuit boards, and to control the autonomous motion of robots on factory floors. In autonomous vehicles such as cars and trains, object detection models are applied to detect obstacles, which trigger control commands to adapt the vehicle's motion to the identified environmental conditions. For the safety of industrial or traffic environments, it is extremely important that the applied object detection model is not only reliable in terms of the correctness of its output predictions, but also robust in providing correct predictions even when the input image data is corrupted due to natural disturbances and malicious attacks.

[0069] In machine learning, robustness quantification refers to the task of identifying the maximum perturbation of data that does not change the model's predictions. For example, a binary image classification model may classify a given image as class A with 90% confidence and class B with 10% confidence. As the brightness of the image changes towards 0, meaning complete darkness, it can be observed that the confidence for class A decreases while the confidence for class B increases. At some point, the confidence for class B exceeds that for class A, and the model changes its predictions.

[0070] But there are also artificial (e.g., hostile) perturbations. Object detection models are typically constructed as artificial deep neural networks. By identifying the patterns these neural networks use to operate, attackers can modify input data so that the deep neural network finds matches that are unrecognizable to human observers. For example, an attacker can make subtle changes to an image so that the deep neural network finds a match for an object category, even if the image bears no resemblance to the matched object to a human observer. This type of manipulation is called an "adversarial attack."

[0071] Finding the exact maximum value of the degree of perturbation for which the model retains its original predictions provides a quantitative measure of the model's robustness.

[0072] There are two main categories of methods for quantifying robustness, depending on the type of perturbation:

[0073] i) For natural perturbations such as brightness changes, mesh deformations, rotations, etc., heuristic methods can be used to locate the maximum value of the perturbation. These include binary search, recursive grid search, or simply increasing the perturbation from 0 by a small value until the model changes its predictions. These methods typically require significant computational resources to repeatedly evaluate the model, but can be parallelized.

[0074] ii). For adversarial perturbations, also known as gradient-based adversarial attacks, not only heuristic searches can be performed, but also more efficient methods such as mixed integer linear programming, random smoothing, or calculating cross-Lipschitz-ness via the gradient norm. Calculating cross-Lipschitz-ness is described by the following articles: "Weng, Tsui-Wei, et al. in "Evaluating the robustness of neural networks: An extreme value theory approach." ICLR (2018)", and "Hein, Matthias and Maksym Andriushchenko "Formal guarantees on the robustness of a classifier against adversarial manipulation.", in Advances in neural information processing systems 30 (2017)". The gradient norm mentioned is disclosed in the following articles: Remigijus, and Julius "Analysis of different norms and corresponding Lipschitz constants for global optimization."Technological and Economic Development ofEconomy 12.4(2006):301-306".

[0075] These methods have been applied to prove the robustness of classification models, especially for image input data. However, it is not known how to prove the robustness of object detection models, which do not produce a single category prediction, but rather a set of bounding box coordinates with a corresponding probability distribution over the possible categories. Such object detection models (e.g., YOLO described in the paper "Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016") and its variants do not satisfy the strict piecewise linear constraints required by mixed integer linear programming. Random smoothing is only suitable for simple classification tasks.

[0076] Quantifying the robustness of object detection models using heuristic search methods requires enormous processing power. Object detection methods are typically based on very deep neural networks, and an exhaustive search with tens of thousands of forward passes requires a lot of computing resources. Furthermore, it is unclear how local robustness should be defined, i.e. with respect to a certain test sample. For adversarial perturbations, also known as gradient-based adversarial attacks, not only heuristic search can be performed, but also more efficient methods such as mixed integer linear programming, random smoothing or computational cross-Lipschitz-ness, as described in the article by Weng et al. In this article, a new robustness metric called CLEVER, which is an abbreviation for Cross-Lipschitz Extremum of Network Robustness, is described.

[0077] The method described below consists of two main features. First, a set of effective robustness requirements is implemented for object detection models, which output a set of bounding box coordinates with a corresponding probability distribution over possible object categories. Specifically, each of these robustness requirements is defined by stating scenarios in which the model is not considered robust. Second, a cross-Lipschitz-ness function is derived from each robustness requirement, which is applied in conjunction with the CLEVER score as described in the aforementioned papers by Weng et al. and Hein et al. to demonstrate the robustness of the model.

[0078] A computer-implemented method for automatically quantifying the robustness of object detection models is proposed. Figure 1 shown and described in detail below.

[0079] Object detection models are applied to control and / or monitoring tasks in, for example, industrial manufacturing workshops or autonomous driving environments.

[0080] In a first step S1, an object detection model f is received. Object detection model f is trained to output a set of predicted objects and is intended to be applied to control or surveillance tasks. For image data x input to object detection model f, object detection model f outputs predicted objects. Each of these object predictions consists of i) the location of a bounding box that may contain the potential object, ii) a feasibility score that quantifies the likelihood of the object being present in the bounding box, and iii) a distribution over all known classes, indicating which class the object is likely to belong to.

[0081] In a second step S2, a set of robustness requirements Ri is applied to the object detection model. From each robustness requirement Ri, a cross-Lipschitz-ness function g is derived that quantifies the robustness requirement Ri, conditioned on the object detection model f for the input image data x (see step S3). The robustness value RV of the object detection model f is determined by computing the cross-Lipschitz-ness function for perturbed image data xp that deviates from the unperturbed image data x0. This cross-Lipschitz-ness function is incorporated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model (see step S4). The unperturbed image data x0 is identical to the image data x.

[0082] In step S5, the determined robustness value RV is compared with a predefined robustness threshold RT. Finally, if the robustness value RV is lower than the predefined robustness threshold RT, a positive proof C is output for application of the object detection model in the control and / or monitoring task (see step S6). Robustness thresholds must be defined in advance and require domain knowledge. Proof C indicates that the received object detection model is sufficiently robust to meet the predefined robustness threshold RV for a set of robustness requirements defined for the application scenario.

[0083] If the determined robustness value RV is not lower than the robustness threshold, a message indicating that the certification requirement is not met is output, see step S7. In one embodiment, additional information about at least one robustness requirement that caused the robustness threshold to be not met is output. In a further embodiment, the received object detection model f is retrained with a set of training image data. The set of training image data is selected based on the additional information to optimize the robustness value. The received object detection model f is retrained with the set of training image data to generate an optimized object detection model f', which is input to the method of step S1 for quantifying the robustness of the optimized object detection model f', see Figure 1 The dashed arrow in .

[0084] In a first embodiment, all data elements of the image data are used to determine the robustness value. In a second embodiment, only data elements of the image data that are within at least one bounding box output by the object detection model f for the image data (x) are used to determine the robustness value. For the second embodiment, additional method steps are performed, see Figure 1In method step S11, all data elements of the unperturbed image data x are processed by the received object detection model f. As a result, the model outputs bounding boxes based on their locations within the image data. In steps S2-S7, only those data elements xbb that lie within these bounding boxes are processed to determine a robustness value and output a proof. This significantly reduces the processing power required to perform the method. In one embodiment, data elements of a subset of the output bounding boxes are used to perform steps S2-S7.

[0085] In a further embodiment, method steps S1 to S5 are performed for at least one further different object detection model f1, which is trained to output at least one predicted object class. A robustness value is determined for each received further object detection model f1, and a list of robustness values ​​for all further object detection models f1 with a positive certification is output. A further object detection model f1 is selected from the list, for example, the further object detection model with the lowest robustness value, and is used for the control and / or monitoring task.

[0086] The structure of the received object detection model f is interpreted with respect to the input data x, and the output data f(x) is given in Figure 2 and explained below.

[0087] The object detection model f receives as input image data

[0088] x∈[0,255] H×W×3

[0089] Where H and W represent the height and width of the image. The value xi of each single data element is a scalar value, for example, indicating a color value in the range of 0 to 255. The output f(x) of the model produces a set of bounding boxes BB, which are annotated class probabilities:

[0090] f(x)=(BB(x) [1] ,BB(x) [2] ,...,BB(x )[C] )

[0091] The number C of bounding boxes BB in the set of bounding boxes BB is predefined, and each bounding box includes the following information:

[0092] BB(x) [c] =(o(x) [c] , l0(x) [c] ,u0(x) [c] , l1(x) [c] ,u1(x) [c] , p(x) [c] )

[0093] in

[0094] o(x) [c] represents a logistic regression to predict the “objectness score” for each bounding box,

[0095] are the x and y coordinates of the center of the bounding box BB, and

[0096] h [c] ,w [c] are the height and width of the bounding box BB,

[0097] l0,u0 respectively indicate the leftmost and rightmost coordinates on the x-axis.

[0098] l1,u1 indicate the leftmost and rightmost coordinates on the y-axis.

[0099] p [c] ∈[0,1] K is the probability distribution over K possible categories.

[0100] Note that each of these terms depends on the input x. The perturbation function π(x|ε) is parameterized by the degree ε. The cross-Lipschitz-ness function is denoted by g(), following the notation from Redmon's paper, see above.

[0101] Object detection models, such as YOLO version 4 and later, are constructed as convolutional neural networks that use features from the entire image to predict each bounding box BB. It also simultaneously predicts all bounding boxes BB for all classes across the image.

[0102] The robustness requirements that the object detection model f must satisfy are stated. Specifically, each of these robustness requirements Ri is defined by stating the scenarios in which the object detection model f is not considered robust.

[0103] A first robustness requirement is applied, which indicates that the object detection model is not robust if the predicted bounding boxes are misclassified compared to the unperturbed predictions. That is, if at least one predicted bounding box of the object detection model processed with the perturbed image data xp is misclassified compared to the prediction of the object detection model processed with the unperturbed image data x0, then the first robustness requirement indicates that the object detection model is not robust. This scenario is like Figure 3 As shown in the figure. While for the unperturbed image data x0, the object detection model f outputs the bounding box BB1 with the highest probability of the object category oc1, when the perturbed image data xp is input, the object detection model f outputs the same bounding box BB1 with the highest probability of the object category oc2. In scenario 1, for at least one bounding box, the distribution of object categories has changed under the perturbation.

[0104] This is formally expressed as:

[0105]

[0106] For all bounding boxes c=1…C, the corresponding cross-Lipschitz-ness function is defined as

[0107]

[0108] in is sampled from a hypersphere Bp centered at the original data point x. Corresponds to the perturbed image data. x corresponds to the unperturbed image data. π is the perturbation added to the unperturbed image data x. Note that this is equivalent to Algorithm 2 in the article by Weng et al.

[0109] According to the second robustness requirement, if at least one bounding box output by the object detection model f for the unperturbed image data x0 as a predicted bounding box is ignored and is no longer output by the object detection model f for the perturbed image data xp as a predicted bounding box, then the object detection model is not robust. In other words, if the originally predicted bounding box is ignored under perturbation, that is, it is a false negative prediction, then the object detection model f is not robust. Specifically, for the YOLO model after version 3, it applies logistic regression to predict the "target score" of each bounding box, which is formally expressed as:

[0110]

[0111] Where o() indicates that the input is the logistic regression of image data, and is the hypersphere B centered at the original data point x. p Sampling.

[0112] According to the third robustness requirement, if at least one bounding box is output as a predicted bounding box by the object detection model for the perturbed image data xp, but the same bounding box is not output as a predicted bounding box by the object detection model f for the unperturbed image data x0, then the object detection model f is not robust. In other words, if the bounding box is predicted under perturbation, but it is originally ignored, then the object detection model f is not robust. This means that the object detection model outputs a false positive prediction, which is the opposite of the scenario of the second robustness requirement.

[0113] Formally, the third robustness requirement is expressed as follows:

[0114]

[0115] Both cases, i.e., the second and third robustness requirements share the same formula for the cross-Lipschitz-ness function, namely:

[0116]

[0117] in is sampled from a hypersphere Bp centered at the original data point x. As mentioned before, is the disturbed image data xp, and x is the undisturbed image data x0.

[0118] Figure 4 The following scenarios are related to the second and third requirements. Under perturbation, and for at least one bounding box, the objectness score o(x) has changed from < 0.5 to > 0.5 and vice versa. Figure 4 , the objectness score o(x) of the predicted bounding box BB1 determined for the unperturbed image data x0 changes from a value > 0.5 to a value < 0.5 output by the object detection model f for the perturbed image data xp, see bounding box BB1'. For bounding box BBn, the objectness score o(x) that had a value < 0.5 when determined for the unperturbed input image data x0 has changed to a value > 0.5 for the perturbed image data xp, see BBn'.

[0119] According to the fourth requirement, if at least one bounding box output by the object detection model f for the perturbed image data xp differs in size and position in the image data from the same bounding box output by the object detection model f for the unperturbed image data x0, then the object detection model is not robust. In other words, if the bounding box predicted under perturbation is inconsistent with the bounding box predicted without perturbation, then the object detection model f is not robust.

[0120] This agreement of at least one bounding box output for the perturbed image data xp and at least one predicted bounding box output for the unperturbed image data x0 is determined by an IoU function, wherein a threshold for the maximum value of the determined IoU is set as an IoU threshold θ. The value of the IoU threshold θ typically varies between 0.5 and 0.95, depending on the specific use case.

[0121] This robustness requirement is formally provided by:

[0122]

[0123] Specifically, the IoU between two bounding boxes predicted based on input x and z is calculated as:

[0124]

[0125] in

[0126] Int(BB(x) [c] ,BB(z) [c] )=min(u0(x) [c] ,u0(z) [c] )-max(l0(*) [c] ,l0(z) [c] )×min(u1(x) [c] ,u1(z) [c] )-max(l1(x) [c] ,l1(z) [c] )

[0127] and

[0128] Uni(BB(x), [c] BB(z) [c] )=A(BB(x) [c] )+A(BB(z) [c] )-Int(BB(x) [c] ,BB(z) [c] )

[0129] A(BB(x) [c] )=(u0(x) [c] -l0(x) [c] )×(u1(x) [c] -l1(x) [c] )

[0130] A(BB(z) [c] )=(u0(z) [c] -l0(z) [c] )×(u1(z) [c] -l1(z) [c] )

[0131] The corresponding cross-Lipschitz-ness function is proposed by the following formula:

[0132]

[0133] That is, the Lipschitz-ness function is simply the Intersection over Union (IoU) between the bounding boxes output by the object detection model for the perturbed image data and the original (i.e., unperturbed) image data. This is exactly what the robustness requirement itself states.

[0134] This function is in is differentiable and is therefore compatible with the CLEVER framework described by Weng et al. However, if the perturbed bounding box has no intersection with the original bounding box (the numerator is 0), then The gradient in will become 0. To this end, we propose an alternative method to the actual IoU calculation, which should be more numerically stable:

[0135]

[0136] in is sampled from a hypersphere Bp centered at the original data point x. The data points of the hypersphere are located at a fixed distance from the data point x, which is given by the radius of the hypersphere.

[0137] This is the distance between the perturbed and original coordinates in Lp space. For p = 2, it corresponds to the common training loss for object detection models.

[0138] This scenario is like Figure 5 As shown in Figure 2, the position and size of the bounding box BB1 output by the object detection model f for the unperturbed image data x (also denoted as x0) are significantly different from the position and size of the bounding box BB1' output by the object detection model f for the perturbed image data (also denoted as xp). The area IU shared by the bounding boxes BB1 and BB1' is provided by the intersection-over-union functional unit.

[0139] therefore, Figure 5 The scenario shown in illustrates the fourth requirement, where at least one predicted bounding box under perturbation differs significantly from the predicted bounding box without perturbation in terms of intersection over union. The Ip norm can be used as a more stable calculation of the corresponding coordinate distance.

[0140] In the following two embodiments, an algorithm for quantizing the robustness value is provided according to pseudo code, comprising steps S1-S4. In the first embodiment, all data elements of the image data x are used for quantization. In the second embodiment, only a subset of the data elements of the image data x is used.

[0141] In a first embodiment, Algorithm 1 performs the integration of the cross-Lipschitz-ness function into the CLEVER framework as described by Weng et al. The known CLEVER function is enhanced by the robustness requirement. The resulting robustness value μ [c] provides the minimum perturbation that causes a change in the output of the object detection model f. In other words, the robustness value μ [c] It provides a robustness guarantee that for each bounding box c, the object detection model f is robust to any perturbation Δ of the image data (where ||Δ|| p <μ) outputs the same result. [c] is a scalar value representing the radius of a hypersphere centered at the given data element. The guarantee is that as long as the perturbation does not leave the sphere, the model remains robust, i.e., follows its original predictions.

[0142] Algorithm 1

[0143]

[0144]

[0145] The core of Algorithm 1 is to calculate the gradient Steps such as Figure 6 As shown. For simplicity, the superscript [c] is omitted. The output of these steps is the gradient. Their norm will be used to perform the maximum likelihood estimation (MLE) under the Weibull assumption, and thus the minimum value and the sampling radius will produce the final result.

[0146] exist Figure 6 In Figure 2, the copy step is marked by a dash-dot arrow, the sampling step is marked by a dotted arrow, the backward pass is marked by a solid arrow, and the forward pass is marked by a dashed arrow. A total of four forward passes and three backward passes are performed to collect one observation of the gradient norm.

[0147] In the first embodiment, Algorithm 1, the entire image data including all pixels contributes to the cross-Lipschitz-ness function. In some cases, the perturbation within a certain bounding box is sufficient to affect the prediction about that bounding box. Therefore, the derived gradient norm will be much smaller, and the robust threshold will be larger. See Figure 1 , the following additional steps in S11 are proposed to take this into account:

[0148] i). Given an original / unadvertised sample image, collect all predicted bounding boxes.

[0149] ii). For each bounding box, define a binary masking matrix such that all pixels outside the bounding box have a value of 0 and all pixels inside the bounding box have a value of 1.

[0150] iii) Multiply the masking matrix with the image x and perform the algorithm described above while ignoring all other bounding boxes.

[0151] Algorithm 2 of the second embodiment is as follows:

[0152]

[0153]

[0154] That is, Algorithm 2 will first perform a forward pass to compute the bounding box predictions. This information is used to formulate the masking matrix Algorithm 1 is then executed with the masked input and a single bounding box index c.

[0155] Execution of Algorithm 2 requires far fewer processing steps than Algorithm 1. And Algorithm 1 requires far less processing power than commonly discussed and applied methods.

[0156] Figure 7 A proof device 30 is shown for providing positive proof of at least one object detection model f, which is a candidate for application in a monitoring or control task in a transportation environment 10 (for example, in an autonomous train) or in an industrial environment 20 (for example, in a manufacturing process or motion control of a robot in an industrial workshop).

[0157] The proving apparatus includes an input unit 31, a processing unit 32, and an output unit. The input unit 31 is configured to receive an object detection model f, which is trained to output a predicted object based on a location in the image data x and an object category from a set of object categories when image data x is input to the object detection model. The apparatus also includes a processing unit configured to: apply a set of robustness requirements to the object detection model; derive a cross-Lipschitz-ness function g from each robustness requirement, which quantifies the robustness requirement conditioned on the object detection model f for the input image data x. The processing unit 32 is configured to: determine a robustness value of the object detection model f by calculating a cross-Lipschitz-ness function for perturbed image data xp that deviates from unperturbed image data x0 of the image data x, wherein the cross-Lipschitz-ness function is incorporated into a network robustness cross-Lipschitz extreme value CLEVER score of the object detection model; and compare the determined robustness value RV with a predefined robustness threshold RT. The output unit is configured to output a positive proof for applying the object detection model f in a control task and / or a monitoring task if the robustness value is below a predefined robustness threshold.

[0158] The attestation device is configured to receive at least one additional object detection model f1 and output a list of those object detection models for which a positive attestation has been received. Optionally, the attestation device receives additional image data, for example, image data sampled from a transportation environment 10 or an industrial environment 20, to retrain the object detection model f under consideration, thereby overcoming deficiencies and generating an optimized object detection model f', which is again analyzed by the attestation device 30.

[0159] Heuristic search methods quantify the robustness of object detection models, for example, by perturbing multiple input data elements at different degrees and calculating batch mAP. This approach requires a large number of model predictions in the form of forward passes on the perturbed image data and does not directly calculate robustness, but only calculates mAP as a proxy.

[0160] The proposed method gives a practical and detailed definition of robust object detection. The corresponding cross-Lipschitz-ness function is derived from each robustness requirement and integrated into the CLEVER framework. This enables the use of extreme value theory by Weng et al. and et al. identified a connection between Lipschitz-ness and gradient norm, using far fewer back-propagations to estimate model robustness.

[0161] Compared to Weng et al., we largely generalize the definition of cross-Lipschitz-ness to more complex situations, such as object detection tasks. We propose to compute multiple cross-Lipschitz-ness functions and take the worst-case norm of the corresponding gradients.

[0162] It should be understood that the above description of the examples is intended to be illustrative and that the components shown are susceptible to various modifications. For example, the concepts shown can be applied to different technical systems, and in particular to different subtypes of the corresponding technical systems with only minor adaptations.

Claims

1. A computer-implemented method for automatically quantifying the robustness of an object detection model (f) applied to a control task and / or a surveillance task, comprising: - receiving (S1) the object detection model (f), the object detection model being trained to output a predicted object based on a location in the image data (x) and an object class from a set of object classes when image data (x) is input into the object detection model (f), - applying (S2) a set of robustness requirements to said object detection model (f), - deriving (S3) a cross-Lipschitz-ness function (g) from each robustness requirement, said cross-Lipschitz-ness function quantifying the robustness requirement conditioned on said object detection model (f) for the input image data (x), - determining (S4) a robustness value (RV) of the object detection model (f) by calculating the cross-Lipschitz-ness function for perturbed image data (xp) deviating from the undisturbed image data (x0) of the image data (x), the cross-Lipschitz-ness function being integrated into a network robustness cross-Lipschitz extreme value CLEVER score of the object detection model (f), - comparing (S5) the determined robustness value (RV) with a predefined robustness threshold (RT), and - if the robustness value (RV) is below the predefined robustness threshold (RT), outputting (S6) a positive proof (C) for applying the object detection model (f) in the control task and / or the monitoring task, wherein each robustness requirement is defined by stating scenarios in which the object detection model (f) is not considered robust.

2. The computer-implemented method of claim 1 , wherein the object detection model (f) outputs a class probability for each bounding box (BB) in a set of bounding boxes depending on the image data (x), wherein each bounding box (BB) specifies a location region in the image data (x), and wherein each bounding box (BB) whose class probability is higher than a predefined probability value is a predicted bounding box.

3. The computer-implemented method of claim 2 , wherein according to a first robustness requirement, the object detection model (f) is not robust if at least one predicted bounding box (BB′) of the object detection model (f) processed with the perturbed image data (xp) is misclassified compared to the prediction of the object detection model (f) processed with the unperturbed image data (x0).

4. A computer-implemented method according to any one of claims 2 and 3, wherein according to a second robustness requirement, the object detection model is not robust if at least one bounding box output by the object detection model (f) for the unperturbed image data (x0) as a predicted bounding box (BB) is ignored and is no longer output by the object detection model (f) for the perturbed image data (xp) as a predicted bounding box (BB').

5. The computer-implemented method according to any one of claims 2 to 4, wherein according to a third robustness requirement, the object detection model (f) is not robust if at least one bounding box (BB′) is output by the object detection model as a predicted bounding box for the perturbed image data (xp), but the same bounding box (BB) is not output by the object detection model (f) as a predicted bounding box for the unperturbed image data (x0).

6. A computer-implemented method according to any one of claims 2 to 5, wherein according to a fourth robustness requirement, the object detection model (f) is not robust if at least one predicted bounding box (BB') output by the object detection model (f) for the perturbed image data (xp) differs in size and position in the image data from the same predicted bounding box (BB) output by the object detection model (f) for the unperturbed image data (x0).

7. The computer-implemented method of claim 6 , wherein the consistency of at least one predicted bounding box (BB′) output for the perturbed image data (xp) and at least one predicted bounding box (BB) output for the unperturbed image data (x0) is determined by an intersection over union function.

8. The computer-implemented method according to any one of claims 3 to 6, wherein the perturbed image data (xp) is sampled from a hypersphere centered at the unperturbed image data (x0).

9. A computer-implemented method according to any one of the preceding claims, wherein all data elements of the image data (x) are used to determine the robustness value (RV), or Only data elements of the image data (x) that are within at least one bounding box (BB) output by the object detection model (f) for the image data (x) are used to determine the robustness value (RV).

10. The computer-implemented method according to any of the preceding claims, wherein the object detection model (f) is applied to the control and / or monitoring of an industrial manufacturing process or an autonomous driving vehicle.

11. A computer-implemented method according to any of the preceding claims, wherein if the robustness value (RV) is above the predefined robustness threshold (RT), the received object detection model is retrained using a set of training image data (xt), the set of training image data being selected to optimize the robustness value (RV).

12. The computer-implemented method of any preceding claim, further comprising: - receiving at least one further different object detection model (f1), said at least one further different object detection model being trained to output at least one predicted object class, - determine the robustness value (RV) of each received object detection model (f, f1), - output a list of robustness values ​​(RV) of all object detection models (f, f1) with positive proof, - select one of the listed object detection models (f, f1), and - using the selected object detection model (f) for the control task and / or the monitoring task.

13. A certification device comprising: The input unit (31) is configured as follows: - receiving an object detection model (f) trained to, when image data (x) is input into the object detection model (f), output a predicted object based on a location in the image data (x) and an object class from a set of object classes, The processing unit (32) is configured to: - applying a set of robustness requirements to the object detection model (f), - deriving a cross-Lipschitz-ness function (g) from each robustness requirement, said cross-Lipschitz-ness function quantifying the robustness requirement conditioned on said object detection model (f) for the input image data (x), determining a robustness value (RV) of the object detection model (f) by calculating the cross-Lipschitz-ness function for perturbed image data (xp) deviating from the undisturbed image data (x0) of the image data (x), the cross-Lipschitz-ness function being integrated into a network robustness cross-Lipschitz extreme value CLEVER score of the object detection model (f), - comparing the determined robustness value (RV) with a predefined robustness threshold (RT), and The output unit (33) is configured to: - outputting a positive proof for applying the object detection model (f) in the control task and / or the monitoring task if the robustness value (RV) is below the predefined robustness threshold (RT), Each robustness requirement is defined by stating the scenarios for which the object detection model (f) is not considered robust.

14. A computer program product directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of claims 1 to 12 when said product is run on said digital computer.

Images