Robustness of automatic quantization applied to object detection models for control and / or monitoring tasks

By using the cross-Lipschitz-ness function and the CLEVER score framework, the problem of robustness quantification of object detection models is solved, and the robustness value is clearly defined and calculated in industrial-grade and safety-grade deep neural networks, ensuring the reliability of the model when faced with disturbances.

CN120677475BActive Publication Date: 2026-11-17SIEMENS AG
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202480011277.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2023-02-07
Filing Date
2024-02-02
Publication Date
2026-11-17
Estimated Expiration
2044-02-02

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively quantify the robustness of object detection models, especially in industrial-grade and security-grade deep neural networks. They cannot be measured on a sample-by-sample basis, and their computational complexity is high, lacking theoretical guarantees.

Method used

By employing the cross-Lipschitz-ness function and the CLEVER score framework, the robustness of the object detection model is quantified by calculating the robustness requirements of the model. The cross-Lipschitz-ness function is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model, providing an explicit definition and calculation of the robustness value.

Benefits of technology

Robustness quantification of industrial-grade and security-grade object detection models has been achieved, reducing computational complexity and providing a true and detailed definition of robustness values ​​to ensure the reliability of the model in the face of natural disturbances and malicious attacks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120677475B_ABST
    Figure CN120677475B_ABST
Patent Text Reader

Abstract

A computer-implemented method for automatically quantifying the robustness of an object detection model (f) applied to control and / or surveillance tasks, comprising: - receiving (S1) an object detection model (f) trained to output predicted objects based on location in the image data (x) and object category from a set of object categories when image data (x) is input to the object detection model (f); - applying (S2) a set of robustness requirements to the object detection model (f); - deriving (S3) a cross-Lipschitz-ness function (g) from each robustness requirement, which quantifies the robustness requirements conditioned on the object detection model (f) for the input image data (x); - for deviations from the image data (x)... The perturbed image data (x0) and the perturbed image data (xp) are used to determine the robustness value (RV) of the object detection model (f) by calculating the cross-Lipschitz-ness function (S4). The cross-Lipschitz-ness function is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model (f). The determined robustness value (RV) is compared with a predefined robustness threshold (RT) (S5). If the robustness value (RV) is lower than the predefined robustness threshold (RT), an output (S6) is provided as a positive proof (C) for applying the object detection model (f) in control and / or monitoring tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to auxiliary devices and computer-implemented methods for automatically quantifying the robustness of object detection models, preferably for use in object detection models (f) for controlling and / or monitoring industrial manufacturing processes or autonomous vehicles. Background Technology

[0002] There is currently a trend towards digitalization in the industrial sector. Therefore, for example, product manufacturing processes can be digitally controlled. Considering the complexity of factories, which typically consist of various components, parts, modules, or units with multiple independent functions, these units and their functions must be controlled and regulated interactively. The increasing level of digitalization allows for the automated manufacturing or industrial assembly of products on factory production lines, for example, by robotic units or other autonomous units.

[0003] The manufacturing process itself must also be monitored and controlled. The quality of the resulting manufactured products must be monitored to identify early degradation and to derive corrective control measures for the manufactured components, such as adapting the component settings.

[0004] Artificial intelligence (“AI”) models for object detection based on image data are deployed and operated in industrial and public transportation environments, such as for control, quality, or surveillance tasks. These object detection models must be “industrial grade”.

[0005] Another application area for object detection methods is identifying and locating objects in streets or surrounding environments based on sensor data and / or image data from autonomous vehicles or traffic control systems, in order to identify obstacles or traffic volumes affecting traffic flow. This requires AI models to be reliable and robust, even if conditions around their application scenario may change, such as lighting conditions or location. AI-based object detection models must be proven and published for use in public or private transportation, such as trains or cars.

[0006] A verification component for verifying artificial intelligence is known from EP 4105846 A1, and a method for determining the influence of attributes in a recurrent neural network trained on treatment prediction is known from EP 3564862A1.

[0007] Typically, the robustness of object detection models is quantified using heuristic search methods. However, object detection methods are often based on very deep neural networks, and a thorough search with thousands of forward passes requires significant computational resources. Furthermore, it is unclear how to define local robustness—that is, robustness with respect to a given test sample. One approach is to natively perturb a batch of images to varying degrees. Then, based on common metrics such as mean average precision (mAP), it can be determined at what level of perturbation the mAP value is no longer acceptable, and this threshold is reported as proof of robustness.

[0008] This approach has several drawbacks: First, it only computes global robustness, making it impossible to quantify the measurement on a sample-by-sample basis. Second, its computational complexity depends on the number of samples in the batch and the granularity of the heuristic search. Third, the threshold for acceptable mAP is arbitrary, and for this purely empirical approach, there is no theoretical guarantee to ensure a meaningful proof. Importantly, it avoids specifying a true definition of robustness by simply using mAP as a substitute.

[0009] While several methods have been applied to demonstrate the robustness of classification models, particularly for image input data, it remains unclear how to demonstrate the robustness of object detection models. The key difference lies in the fact that object detection models do not produce single-class predictions, but rather sets of bounding box coordinates with corresponding probability distributions for the possible classes. Summary of the Invention

[0010] Therefore, the purpose of this application is to provide a method and a testing apparatus that automatically provides robustness quantization with reduced computational complexity, which is applied to realistic and reliable proof standards for object detection models based on industrial-grade or security-grade deep neural networks.

[0011] This objective is achieved through the features of the independent claims. The dependent claims contain further developments of the invention.

[0012] The first aspect relates to a computer-implemented method for automatically quantifying the robustness of an object detection model applied to control and / or surveillance tasks, comprising the following steps:

[0013] - Receive an object detection model, which is trained to output predicted objects based on their location in the image data and their category from a set of object categories when image data is input into the model.

[0014] - Apply a set of robustness requirements to the object detection model.

[0015] - Derive a cross-Lipschitz-ness function from each robustness requirement, which quantifies the robustness requirement conditioned on the object detection model used for the input image data.

[0016] - For both undisturbed and perturbed image data that deviate from the original image data, the robustness value of the object detection model is determined by calculating the cross-Lipschitz-ness function. This cross-Lipschitz-ness function is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model.

[0017] - If the robustness value is below a predefined robustness threshold, output a positive proof for applying the object detection model in control and / or monitoring tasks.

[0018] This method provides a realistic and detailed definition of robust object detection. Cross-Lipschitz-ness functions are derived from each robustness requirement and integrated into the CLEVER framework. By using cross-Lipschitz-ness functions for this purpose, a reliable measure of the robustness of the object detection model is provided, requiring far fewer backpropagations and therefore less processing power.

[0019] Each robustness requirement is defined by stating the scenarios in which the object detection model (f) is not considered robust.

[0020] This allows for a clear definition of robustness requirements and a clear formula for the Lipschitz-ness function. Therefore, clear constraints can be formulated and provided.

[0021] According to a further embodiment, the object detection model depends on the class probability of each bounding box in a set of bounding boxes output from the image data, where each bounding box specifies a location region in the image data, and where each bounding box with a class probability higher than a predefined probability value is a predicted bounding box.

[0022] This object detection model provides features well-suited for specifying robustness analysis. This type of object detection model is widely used and provides reliable results. Well-known and applied object detection models of this type are those similar to or based on the YOLO model.

[0023] In a further embodiment according to the first robustness requirement, the object detection model is not robust if at least one predicted bounding box of the object detection model processed with perturbed image data is misclassified compared with the prediction of the object detection model processed with undisturbed image data.

[0024] This robustness requirement covers situations where perturbations in the image data cause object detection models to typically provide incorrect classifications.

[0025] In a further embodiment according to the second claim, the object detection model is not robust if at least one bounding box output by the object detection model (f) as a predicted bounding box for undisturbed image data (x0) is ignored, and the object detection model (f) no longer outputs as a predicted bounding box for disturbed image data (xp).

[0026] This provides a key criterion for the robustness of false negative predictions.

[0027] In a further embodiment according to the third robustness requirement, if at least one bounding box is output by the object detection model as a predicted bounding box of the perturbed image data, but the same bounding box is not output by the object detection model as a predicted bounding box of the undisturbed image data, then the object detection model is not robust.

[0028] This provides a key criterion for the robustness of false positive predictions.

[0029] In a further embodiment according to the fourth robustness requirement, the object detection model is not robust if at least one predicted bounding box output by the object detection model for disturbed image data differs in size and position in the image data from the same predicted bounding box output by the object detection model for undisturbed image data.

[0030] This requirement designates the object detection model as not robust if the position or size of the "same" bounding boxes differs when the output of the object detection model is compared to that of undisturbed image data. This provides a key robustness requirement for predicting object location.

[0031] According to a further embodiment, the cross-union function unit determines the consistency between at least one predicted bounding box for the output of disturbed image data (xp) and at least one predicted bounding box for the output of undisturbed image data (x0).

[0032] Intersection over Union (IoU) functional units are commercially available units that offer fast processing and require only minimal adaptation work. Thus, using IoU functional units is cost-effective and efficient in terms of processing power.

[0033] According to a further embodiment, disturbed image data is sampled from a hypersphere centered on undisturbed image data.

[0034] Evaluating object detection models using this perturbed image data also considers adversarial attacks, which are based on perturbations in the image data that are virtually invisible to the human eye but can cause the object detection model to output incorrect predictions. Here, incorrect predictions mean that the predictions have changed compared to those made using undisturbed image data.

[0035] According to a further embodiment, all data elements of the image data (x) are undisturbed image data (x0), or

[0036] Only the data elements of the image data (x) located within at least one bounding box of the image data (x) are undisturbed image data (x0), and these data elements are used to determine the robustness value.

[0037] The data elements of image data are synonymously named pixels.

[0038] Considering only the pixels of image data within the bounding box reduces the number of pixels to be processed and also reduces processing power.

[0039] According to a further embodiment, the object detection model is applied to control and / or monitor industrial manufacturing processes or autonomous vehicles.

[0040] Therefore, the proven object detection model is applicable to application environments that require particularly reliable and robust models.

[0041] According to a further embodiment, if the robustness value is higher than a predefined robustness threshold, the received object detection model is retrained using a set of training image data, and this set of training image data is selected to optimize the robustness value.

[0042] This allows for the optimization of object detection models, particularly regarding defects identified during robustness quantization.

[0043] According to a further embodiment, the method further includes the following steps:

[0044] - Receive at least one other distinct object detection model, which is trained to output at least one predicted object category.

[0045] - Determine the robustness value of each received object detection model.

[0046] - Output a list of robustness values ​​for all object detection models with positive proof.

[0047] - Select one of the listed object detection models, and

[0048] - Use the selected object detection model for control and / or monitoring tasks.

[0049] This allows for the selection of the most suitable object detection model relative to the robustness requirements of the control or monitoring task under consideration.

[0050] The second aspect relates to a proof apparatus, comprising at least one processor configured to perform the following steps:

[0051] - Receive an object detection model, which is trained to output predicted objects based on their location in the image data and their category from a set of object categories when image data is input into the model.

[0052] - Receive a set of robustness requirements for the object detection model.

[0053] - Derive a cross-Lipschitz-ness function from each robustness requirement, which quantifies the robustness requirement conditioned on the object detection model used for the input image data.

[0054] - For both undisturbed and perturbed image data that deviate from the original image data, the robustness value of the object detection model is determined by calculating the cross-Lipschitz-ness function. This cross-Lipschitz-ness function is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model.

[0055] - If the robustness value is below a predefined robustness threshold, output a positive proof for applying the object detection model in control and / or monitoring tasks, where each robustness requirement is defined by stating a scenario in which the object detection model (f) is not considered robust.

[0056] The verification device provides verification of the received object detection model in a processing-optimized manner. The verification device may be part of a verification system that evaluates the received object detection model against further criteria related to verifying an "industrial-grade" or "transport-grade" object detection model. The verification device may include an output interface directly coupled to a controller that applies the object detection model to control and / or monitoring tasks.

[0057] The third aspect relates to a computer program product that can be directly loaded into the internal memory of a digital computer, including a software code portion for performing the steps as described above when the product is run on the digital computer.

[0058] The computer program product may be stored on a computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the steps as described above. Attached Figure Description

[0059] The invention will be explained in more detail with reference to the accompanying drawings. Similar objects will be labeled with the same reference numerals.

[0060] Figure 1 An embodiment of the method of the present invention, illustrated by a flowchart, is shown.

[0061] Figure 2 The output generated by the object detection model received in an embodiment of the method of the present invention is shown in schematic form.

[0062] Figure 3 An embodiment of a scenario corresponding to the first robustness requirement of the method of the present invention is illustrated in schematic form.

[0063] Figure 4 An embodiment corresponding to the second or third robustness requirement of the method of the present invention is illustrated in schematic form.

[0064] Figure 5 An embodiment of a scenario corresponding to a fourth robustness requirement of the method of the present invention is illustrated in schematic form.

[0065] Figure 6 An embodiment of the processing steps of the method of the present invention is illustrated schematically, and

[0066] Figure 7 An embodiment of the proof device of the present invention in an industrial or transportation environment is illustrated schematically. Detailed Implementation

[0067] Note that in the following detailed description of the embodiments, the drawings are merely schematic, and the elements shown are not necessarily shown to scale. Rather, the drawings are intended to illustrate functionality and the cooperation of functions or components. It should be understood here that any connection or coupling of functional blocks, devices, components, or other physical or functional elements may also be achieved through indirect connections or couplings, such as via one or more intermediate elements. Connections or couplings of elements or components or nodes may be achieved, for example, through wired, wireless, and / or combinations of wired and wireless connections. Functional units may be implemented by dedicated hardware (e.g., processor, firmware) or software, and / or by a combination of dedicated hardware, firmware, and software. It should also be noted that each functional unit described for the apparatus may perform functional steps of the associated method, and vice versa.

[0068] Object detection models are applied, or should be applied, in industrial environments to perform quality control, such as detecting misalignments on printed circuit boards and controlling the autonomous movement of robots in factory workshops. In autonomous vehicles such as cars and trains, object detection models are used to detect obstacles that trigger control commands to adapt the vehicle's movement to the identified environmental conditions. For safety in industrial or transportation environments, it is crucial that the applied object detection model is not only reliable in terms of the accuracy of the output predictions but also robust in providing accurate predictions, even when the input image data is disturbed due to natural perturbations and malicious attacks.

[0069] In machine learning, robustness quantization refers to the task of identifying the maximum extent to which data perturbations do not change the model's predictions. For example, a binary image classification model can classify a given image into class A with 90% confidence and class B with 10% confidence. When the image's brightness changes towards 0, meaning complete darkness, we can observe that the confidence of class A decreases while the confidence of class B increases. At some point, the confidence of class B will exceed that of class A, and therefore the model changes its prediction.

[0070] However, there are also artificial (e.g., hostile) perturbations. Typically, object detection models are constructed as artificial deep neural networks. By identifying the patterns these neural networks use to operate, attackers can modify the input data, causing the deep neural network to find matches that are unrecognizable to human observers. For example, an attacker can make subtle alterations to an image, causing the deep neural network to find a match regarding an object category, even if the image appears completely dissimilar to the matched object to a human. This manipulation is known as an "adversarial attack."

[0071] Finding the exact maximum value of the perturbation that preserves the model's original predictions provides a quantitative measure of the model's robustness.

[0072] Depending on the type of disturbance, there are two main types of methods for quantifying robustness:

[0073] i) For natural perturbations, such as brightness variations, mesh deformation, rotation, etc., heuristic methods can be used to locate the maximum value of the perturbation, including binary search, recursive mesh search, or simply increasing the perturbation level from 0 to a small value until the model changes its prediction. These methods typically require significant computational resources to repeatedly evaluate the model, but they can be parallelized.

[0074] ii) For adversarial perturbations, also known as gradient-based adversarial attacks, not only heuristic search can be performed, but also more efficient methods such as mixed-integer linear programming, stochastic smoothing, or cross-Lipschitzness computed via gradient norm. The computation of cross-Lipschitzness is described in the following articles: “Weng, Tsui-Wei, et al. in 'Evaluating the robustness of neural networks: An extreme value theory approach.' ICLR (2018)”, and “Hein, Matthias and Maksym Andriushchenko 'Formal guarantees on the robustness of a classifier against adversarial manipulation.' in Advances in neural information processing systems 30 (2017)”. The gradient norm mentioned in these articles is disclosed in the following articles: Remigijus, and Julius "Analysis of different norms and corresponding Lipschitz constants for global optimization."Technological and Economic Development ofEconomy 12.4(2006):301-306".

[0075] These methods have been applied to demonstrate the robustness of classification models, particularly for image input data. However, it remains unclear how to demonstrate the robustness of object detection models that produce not single-class predictions, but rather sets of bounding box coordinates with corresponding probability distributions across possible classes. Such object detection models (e.g., YOLO, described in “Redmon, Joseph, et al.. You only look once: Unified, real-time object detection.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016) and their variants do not satisfy the strict piecewise linear constraints required by mixed-integer linear programming. Stochastic smoothing is only suitable for simple classification tasks.

[0076] Quantifying the robustness of object detection models using heuristic search methods requires enormous processing power. Object detection methods are typically based on very deep neural networks, and thorough searches involving tens of thousands of forward passes require significant computational resources. Furthermore, it remains unclear how to define local robustness, i.e., robustness with respect to a given test sample. For adversarial perturbations, also known as gradient-based adversarial attacks, not only heuristic search can be performed, but also more efficient methods such as mixed-integer linear programming, stochastic smoothing, or computation of cross-Lipschitzness, as described in the paper by Weng et al. In that paper, a novel robustness metric called CLEVER is described, an abbreviation for the network robustness cross-Lipschitz extremum.

[0077] The method described below comprises two main features. First, a set of effective robustness requirements is deployed for the object detection model, whose outputs are sets of coordinates of bounding boxes with corresponding probability distributions over possible object categories. Specifically, each of these robustness requirements is defined by stating scenarios in which the model is not considered robust. Second, a cross-Lipschitz-ness function is derived from each robustness requirement and applied in conjunction with a CLEVER score as described in the aforementioned paper by Weng et al. and Hein et al., to demonstrate the robustness of the model.

[0078] A method for a robust computer implementation of an automatic quantization object detection model is proposed, such as... Figure 1 As shown, and described in detail below.

[0079] Object detection models are applied to control and / or monitoring tasks in environments such as industrial manufacturing workshops or autonomous driving environments.

[0080] In the first step S1, an object detection model f is received. The object detection model f is trained to output a set of predicted objects and is designed for application to control or surveillance tasks. For image data x input to the object detection model f, the object detection model f outputs predicted objects. Each of these object predictions includes i) the location of a bounding box that may contain a potential object, ii) a feasibility score quantifying the probability of an object being present within the bounding box, and iii) a distribution across all known categories, indicating which category the object might belong to.

[0081] In the second step S2, a set of robustness requirements Ri is applied to the object detection model. A cross-Lipschitz-ness function g is derived from each robustness requirement Ri, quantifying the robustness requirement Ri conditioned on the object detection model f used for the input image data x (see step S3). For the perturbed image data xp of the undisturbed image data x0 deviating from the image data x, the robustness value RV of the object detection model f is determined by calculating the cross-Lipschitz-ness function, which is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model (see step S4). The undisturbed image data x0 is the same as the image data x.

[0082] In step S5, the determined robustness value RV is compared with a predefined robustness threshold RT. Finally, if the robustness value RV is lower than the predefined robustness threshold RT, a positive proof C for applying the object detection model in control and / or monitoring tasks is output, see step S6. The robustness threshold must be predefined and requires domain knowledge. Proof C indicates that the received object detection model is robust enough to meet the preset robustness threshold RV, which represents a set of robustness requirements defined for the application scenario.

[0083] If the determined robustness value RV is not lower than the robustness threshold, a message indicating that the proof requirement is not met is output, see step S7. In one embodiment, additional information regarding at least one robustness requirement that leads to the failure to meet the robustness threshold is output. In a further embodiment, the received object detection model f is retrained using a set of training image data. The set of training image data is selected based on the additional information to optimize the robustness value. The received object detection model f is retrained using the set of training image data to generate an optimized object detection model f', which is input to the method of step S1 for quantifying the robustness of the optimized object detection model f', see step S7. Figure 1 The dashed arrow in the middle.

[0084] In the first embodiment, all data elements of the image data are used to determine the robustness value. In the second embodiment, only data elements of the image data located within at least one bounding box output by the object detection model f for the image data (x) are used to determine the robustness value. For the second embodiment, additional method steps are performed; see [link to documentation]. Figure 1In step S11 of the method, all data elements of the undisturbed image data x are processed by the received object detection model f, which outputs bounding boxes based on their positions within the image data. In steps S2-S7, only those data elements xbb located within these bounding boxes are processed to determine robustness values ​​and output proofs. This significantly reduces the processing power required to perform the method. In one embodiment, a subset of data elements of the output bounding boxes are applied to perform steps S2-S7.

[0085] In a further embodiment, method steps S1 to S5 are performed for at least one additional, different object detection model f1, which is trained to output at least one predicted object category. A robustness value is determined for each received additional object detection model f1, and a list of robustness values ​​for all additional object detection models f1 with positive proof is output. An additional object detection model f1 is selected from the list, for example, the one with the lowest robustness value, and is used for the control and / or monitoring tasks.

[0086] The structure of the received object detection model f is interpreted relative to the input data x, and the output data f(x) is... Figure 2 It is shown in the figure and explained below.

[0087] The object detection model f obtains the image data as input.

[0088] x∈[0,255] H×W×3

[0089] Where H and W represent the height and width of the image. The value xi of each individual data element is a scalar value, for example, indicating a color value in the range of 0 to 255. The model's output f(x) produces a set of bounding boxes BB, which are the labeled class probabilities:

[0090] f(x) = (BB(x)) [1] ,BB(x) [2] , ...,BB(x )[C] )

[0091] The number C of bounding boxes BB in this group of bounding boxes is predefined, and each bounding box includes the following information:

[0092] BB(x) [c] =(o(x) [c] , l0(x) [c] ,u0(x) [c] , l1(x) [c] , u1(x) [c] , p(x) [c] )

[0093] in

[0094] o(x) [c] This indicates that logistic regression is used to predict the "target score" for each bounding box.

[0095] These are the x and y coordinates of the center of the bounding box (BB), respectively.

[0096] h [c] ,w [c] These are the height and width of the bounding box (BB).

[0097] l0 and u0 represent the leftmost and rightmost coordinates on the x-axis, respectively.

[0098] l1 and u1 indicate the leftmost and rightmost coordinates on the y-axis.

[0099] p [c] ∈[0,1] K It is a probability distribution over K possible categories.

[0100] Note that each of these terms depends on the input x. The perturbation function π(x|ε) is parameterized by the degree ε. The cross-Lipschitz-ness function is denoted by g(), following the notation in the Redmon paper, see above.

[0101] Object detection models, such as YOLO version 4 and later, are constructed as convolutional neural networks that use features from the entire image to predict each bounding box (BB). It also predicts BBs for all categories across the image simultaneously.

[0102] The effective robustness requirements that the object detection model f must satisfy are stated. Specifically, each of these robustness requirements Ri is defined by stating the scenarios in which the object detection model f is not considered robust.

[0103] The first robustness requirement indicates that an object detection model is not robust if the predicted bounding boxes are misclassified compared to the unperturbed predictions. That is, if at least one predicted bounding box of an object detection model processed with perturbed image data xp is misclassified compared to the predictions of an object detection model processed with unperturbed image data x0, the first robustness requirement indicates that the object detection model is not robust. This scenario is as follows: Figure 3 As shown. For undisturbed image data x0, the object detection model f outputs the bounding box BB1 with the highest probability, representing object category oc1. When the input is perturbed image data xp, the object detection model f outputs the same bounding box BB1 with the highest probability, representing object category oc2. In scenario 1, for at least one bounding box, the distribution of object categories has changed under perturbation.

[0104] This is expressed formally as:

[0105]

[0106] For all bounding boxes c = 1…C, the corresponding cross-Lipschitz-ness function is defined as

[0107]

[0108] in It is sampled from the hypersphere Bp centered on the original data point x. The image data corresponds to the perturbed image data. x corresponds to the undisturbed image data. π is the perturbation added to the undisturbed image data x. Note that this is equivalent to Algorithm 2 in the paper by Weng et al.

[0109] According to the second robustness requirement, if at least one bounding box output by object detection model f as a predicted bounding box for undisturbed image data x0 is ignored and no longer output by object detection model f as a predicted bounding box for perturbed image data xp, then the object detection model is not robust. In other words, if the original predicted bounding box is ignored under perturbation, i.e., it is a false negative prediction, then object detection model f is not robust. Specifically, for YOLO models after version 3, logistic regression is applied to predict the "objectivity score" of each bounding box, which is formally expressed as:

[0110]

[0111] Where o() denotes logistic regression with image data as input, and From the hypersphere B centered at the original data point x p Sampling.

[0112] According to the third robustness requirement, if at least one bounding box is output as a predicted bounding box by the object detection model for perturbed image data xp, but the same bounding box is not output as a predicted bounding box by the object detection model f for undisturbed image data x0, then the object detection model f is not robust. In other words, if the bounding box is predicted under perturbation but was originally ignored, then the object detection model f is not robust. This means that the object detection model outputs false positive predictions, which is the opposite of the scenario in the second robustness requirement.

[0113] Formally, the third robustness requirement is expressed by the following formula:

[0114]

[0115] In both cases, namely the second and third robustness requirements, the same formula for the cross-Lipschitz-ness function is shared:

[0116]

[0117] in It is sampled from the hypersphere Bp centered at the original data point x. As mentioned earlier, x is the disturbed image data xp, and x is the undisturbed image data x0.

[0118] Figure 4 The scenarios related to the second and third requirements are illustrated. Under the perturbation, and for at least one bounding box, the objective score o(x) has changed from <0.5 to >0.5, and vice versa. Figure 4 In the original text, the target score o(x) of the predicted bounding box BB1 determined for undisturbed image data x0 changes from a value >0.5 to a value <0.5 output by the object detection model f for the undisturbed image data xp, see bounding box BB1'. For bounding box BBn, the target score o(x) with a value <0.5 when determined for undisturbed input image data x0 has been changed to a target score with a value >0.5 for the undisturbed image data xp, see BBn'.

[0119] According to the fourth requirement, if at least one bounding box output by object detection model f for perturbed image data xp differs in size and position in the image data from the same bounding box output by object detection model f for undisturbed image data x0, then the object detection model is not robust. In other words, if the bounding box predicted under perturbation is inconsistent with the bounding box predicted under the original, unperturbed condition, then the object detection model f is not robust.

[0120] This consistency between at least one bounding box output for the perturbed image data xp and at least one predicted bounding box output for the undisturbed image data x0 is determined by the Cross-Union-of-Union (CUI) functional unit, where a threshold for the maximum value of the determined CUI is set as the CUI threshold θ. The value of the CUI threshold θ typically varies between 0.5 and 0.95, depending on the specific use case.

[0121] This robustness requirement is formally provided by the following formula:

[0122]

[0123] Specifically, the IoU between the two bounding boxes predicted based on the inputs x and z is calculated as follows:

[0124]

[0125] in

[0126] Int(BB(x) [c] , BB(z) [c] )=min(u0(x) [c] ,u0(z) [c] )-max(l0(*) [c] ,l0(z) [c] )×min(u1(x) [c] ,u1(z) [c] )-max(l1(x) [c] ,l1(z) [c] )

[0127] and

[0128] Uni(BB(x), [c] BB(z) [c] )=A(BB(x) [c] )+A(BB(z) [c] )-Int(BB(x) [c] ,BB(z) [c] )

[0129] A(BB(x) [c] )=(u0(x) [c] -l0(x) [c] )×(u1(x) [c] -l1(x) [c] )

[0130] A(BB(z) [c] )=(u0(z) [c] -l0(z) [c] )×(u1(z) [c] -l1(z) [c] )

[0131] The corresponding cross-Lipschitz-ness function is proposed by the following equation:

[0132]

[0133] In other words, the Lipschitz-ness function is simply the intersection-over-union ratio (IoU) between the bounding boxes output by the object detection model for perturbed image data and the original (i.e., undisturbed) image data. This is precisely what the robustness requirement itself states.

[0134] This function is in The bounding box is differentiable and therefore compatible with the CLEVER framework described by Weng et al. However, if the perturbed bounding box has no intersection with the original bounding box (0 in the molecule), then... The gradient in the equation will become 0. Therefore, we propose an alternative to the actual IoU calculation, which should be numerically more stable:

[0135]

[0136] in It is sampled from a hypersphere Bp centered at the original data point x. The data points on the hypersphere are located at a fixed distance from the data point x, which is provided by the radius of the hypersphere.

[0137] This is the distance between the perturbed coordinates and the original coordinates in the Lp space. For p=2, it corresponds to the common training loss of the object detection model.

[0138] This scenario is like Figure 5 As shown, the position and size of the bounding box BB1 output by the object detection model f for undisturbed image data x (also denoted as x0) are significantly different from the position and size of the bounding box BB1' output by the object detection model f for undisturbed image data (also denoted as xp). The region IU shared by bounding boxes BB1 and BB1' is provided by the intersection-over-union (IoU) functional unit.

[0139] therefore, Figure 5 The scenario illustrated demonstrates the fourth requirement, where at least one predicted bounding box under perturbation differs significantly from the unperturbed predicted bounding box in terms of intersection-over-union ratio (IoU). The Ip norm can be used as a more stable calculation for the corresponding coordinate distance.

[0140] In the two embodiments below, algorithms for quantizing robustness values ​​are provided according to pseudocode, including steps S1-S4. In the first embodiment, all data elements of image data x are used for quantization. The second embodiment uses only a subset of the data elements of image data x.

[0141] In the first embodiment, Algorithm 1 integrates the cross-Lipschitz-ness function into the CLEVER framework as described by Weng et al. The known CLEVER function is enhanced through robustness requirements. The resulting robustness value μ [c] It provides the minimum perturbation that causes a change in the output of the object detection model f. In other words, the robustness value μ [c] Robustness is guaranteed, meaning that for each bounding box c, the object detection model f is robust against any perturbation Δ in the image data (where ||Δ||). p <μ) The output remains unchanged. [c] It is a scalar value representing the radius of the hypersphere centered on a given data element. The guarantee is that the model remains robust as long as the perturbation does not leave the sphere, i.e., it follows its original predictions.

[0142] Algorithm 1

[0143]

[0144]

[0145] The core of Algorithm 1 is calculating the gradient. The steps, such as Figure 6 As shown. For simplicity, the superscript [c] is omitted. The outputs of these steps are gradients. Their norms will be used to perform maximum likelihood estimation (MLE) under the Weibull assumption, and thus the minimum and sampling radius will produce the final result.

[0146] exist Figure 6 In the diagram, the replication step is marked by a dashed-dot arrow, the sampling step by a dotted arrow, the backpropagation by a solid arrow, and the forward propagation by a dashed arrow. A total of four forward propagations and three backward propagations are performed to collect an observation of the gradient norm.

[0147] In Algorithm 1 of the first embodiment, the entire image data, including all pixels, contributes to the cross-Lipschitz-ness function. In some cases, a perturbation within a bounding box is sufficient to affect the prediction about that bounding box. Therefore, the derived gradient norm is much smaller, and the robustness threshold is thus larger. See also Figure 1 To consider this, the following additional steps in S11 are proposed:

[0148] i) Given the original / undisturbed sample images, collect all predicted bounding boxes.

[0149] ii) For each bounding box, define a binary masking matrix such that all pixels outside the bounding box have a value of 0, and all pixels inside the bounding box have a value of 1.

[0150] iii) Multiply the masking matrix by the image x and perform the algorithm described above, while ignoring all other bounding boxes.

[0151] Algorithm 2 of the second embodiment is as follows:

[0152]

[0153]

[0154] In other words, Algorithm 2 will first perform a forward pass to compute the bounding box prediction. This information is used to formulate the masking matrix. Then, Algorithm 1 is executed using the masked input and a single bounding box index c.

[0155] Algorithm 2 requires far fewer processing steps than Algorithm 1. Furthermore, Algorithm 1 requires significantly less processing power than the methods typically discussed and applied.

[0156] Figure 7 A proof device 30 is shown for providing a positive proof of at least one object detection model f, which is a candidate for a monitoring or control task in a transportation environment 10 (e.g., in an autonomous train) or an industrial environment 20 (e.g., in the manufacturing process or motion control of a robot in an industrial shop).

[0157] The proof device includes an input unit 31, a processing unit 32, and an output unit. The input unit 31 is configured to receive an object detection model f, which is trained to output predicted objects based on the location in the image data x and an object category from a set of object categories when image data x is input into the object detection model. It also includes a processing unit configured to: apply a set of robustness requirements to the object detection model; derive a cross-Lipschitz-ness function g from each robustness requirement, which quantifies the robustness requirements conditioned on the object detection model f used for the input image data x. The processing unit 32 is configured to: determine the robustness value of the object detection model f by calculating the cross-Lipschitz-ness function for perturbed image data xp that deviates from the image data x0, the cross-Lipschitz-ness function being integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model; and compare the determined robustness value RV with a predefined robustness threshold RT. The output unit is configured to output positive proof for applying the object detection model f in control and / or monitoring tasks if the robustness value is below a predefined robustness threshold.

[0158] The proof device is configured to receive at least one additional object detection model f1 and output a list of those object detection models that have received positive proof. Optionally, the proof device receives additional image data, such as image data sampled from transportation environment 10 or industrial environment 20, to retrain the considered object detection model f, thereby overcoming defects and generating an optimized object detection model f', which is then analyzed again by the proof device 30.

[0159] Heuristic search methods quantify the robustness of object detection models, for example, by perturbing multiple input data elements to varying degrees and computing batch mAP. This approach requires a large number of model predictions in a forward-passive form on the perturbed image data and does not directly compute robustness, but only computes mAP as a proxy.

[0160] The proposed method provides a practical and detailed definition of robust object detection. Corresponding cross-Lipschitz-ness functions are derived from each robustness requirement and integrated into the CLEVER framework. This enables the utilization of the extremum theory of Weng et al., as well as... The relationship between Lipschitz-ness and gradient norm, as determined by [authors' name], allows for the estimation of model robustness using far fewer backpropagations.

[0161] Compared to Weng et al., we significantly extend the definition of cross-Lipschitz-ness to more complex cases, such as object detection tasks. We propose computing multiple cross-Lipschitz-ness functions and taking the worst-case norm of the corresponding gradients.

[0162] It should be understood that the above description of the examples is intended to be illustrative, and the components shown are easily modified in various ways. For example, the concepts shown can be applied to different technical systems, and especially to different subtypes of the corresponding technical systems with only minor adaptations.

Claims

1. A computer implementation method for automatically quantifying the robustness of an object detection model (f) applied to control and / or monitoring tasks, comprising: - Receive (S1) the object detection model (f), which is trained to output a predicted object based on the location in the image data (x) and the object category from a set of object categories when image data (x) is input into the object detection model (f). - Apply a set of robustness requirements (S2) to the object detection model (f). - Derive (S3) a cross-Lipschitz-ness function (g) from each robustness requirement, the cross-Lipschitz-ness function quantifying the robustness requirement conditioned on the object detection model (f) used for the input image data (x). - For the perturbed image data (xp) that deviates from the undisturbed image data (x0), the robustness value (RV) of the object detection model (f) is determined (S4) by calculating the cross-Lipschitz-ness function, which is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model (f). - Compare the determined robustness value (RV) with a predefined robustness threshold (RT) (S5), and - If the robustness value (RV) is lower than the predefined robustness threshold (RT), then output (S6) a positive proof (C) for applying the object detection model (f) in the control task and / or the monitoring task, wherein each robustness requirement is defined by stating a scenario in which the object detection model (f) is not considered robust.

2. The computer-implemented method of claim 1, wherein the object detection model (f) depends on the class probability of each bounding box (BB) in a set of bounding boxes output by the image data (x), wherein each bounding box (BB) specifies a location region in the image data (x), and wherein each bounding box (BB) with a class probability higher than a predefined probability value is a predicted bounding box.

3. The computer-implemented method of claim 2, wherein, according to a first robustness requirement, the object detection model (f) is not robust if at least one predicted bounding box (BB') of the object detection model (f) processed with perturbed image data (xp) is misclassified compared with the prediction of the object detection model (f) processed with undisturbed image data (x0).

4. The computer-implemented method of claim 3, wherein, according to the second robustness requirement, if at least one bounding box output by the object detection model (f) as a predicted bounding box (BB) for undisturbed image data (x0) is ignored, and the object detection model (f) no longer outputs a predicted bounding box (BB') for disturbed image data (xp), then the object detection model is not robust.

5. The computer-implemented method according to any one of claims 2-4, wherein, according to the third robustness requirement, if at least one bounding box (BB') is output by the object detection model as a predicted bounding box of the perturbed image data (xp), but the same bounding box (BB) is not output by the object detection model (f) as a predicted bounding box of the undisturbed image data (x0), then the object detection model (f) is not robust.

6. The computer-implemented method of claim 2, wherein, according to a fourth robustness requirement, the object detection model (f) is not robust if at least one predicted bounding box (BB') output by the object detection model (f) for disturbed image data (xp) differs in size and position in the image data from the same predicted bounding box (BB) output by the object detection model (f) for undisturbed image data (x0).

7. The computer-implemented method of claim 6, wherein the cross-intersection over union (CUI) functional unit determines the consistency between at least one predicted bounding box (BB') output for the disturbed image data (xp) and at least one predicted bounding box (BB) output for the undisturbed image data (x0).

8. The computer-implemented method according to any one of claims 3-4, wherein the disturbed image data (xp) is sampled from a hypersphere centered on the undisturbed image data (x0).

9. The computer-implemented method according to any one of claims 1-4, wherein all data elements of the image data (x) are used to determine the robustness value (RV), or Only data elements of the image data (x) located within at least one bounding box (BB) output by the object detection model (f) for the image data (x) are used to determine the robustness value (RV).

10. The computer-implemented method according to any one of claims 1-4, wherein the object detection model (f) is applied to the control and / or monitoring of an industrial manufacturing process or an autonomous vehicle.

11. The computer-implemented method according to any one of claims 1-4, wherein if the robustness value (RV) is higher than the predefined robustness threshold (RT), the received object detection model is retrained using a set of training image data (xt), which is selected to optimize the robustness value (RV).

12. The computer-implemented method according to any one of claims 1-4, further comprising: - Receive at least one additional different object detection model (f1), said at least one additional different object detection model being trained to output at least one predicted object category. - Determine the robustness value (RV) of each received object detection model (f, f1). - Output a list of robustness values ​​(RV) for all object detection models (f, f1) with positive proof. - Select one of the listed object detection models (f, f1), and - Use the selected object detection model (f) for the control task and / or the monitoring task.

13. A proof device, comprising: Input unit (31) is configured as follows: - Receive object detection model (f), which is trained to output predicted objects based on the location in the image data (x) and the object category from a set of object categories when image data (x) is input into the object detection model (f). The processing unit (32) is configured to: - Apply a set of robustness requirements to the object detection model (f). - Derive a cross-Lipschitz-ness function (g) from each robustness requirement, the cross-Lipschitz-ness function quantifying the robustness requirement conditioned on the object detection model (f) used for the input image data (x). - For the perturbed image data (xp) that deviates from the undisturbed image data (x0), the robustness value (RV) of the object detection model (f) is determined by calculating the cross-Lipschitz-ness function, which is integrated into the network robustness cross-Lipschitz extremum CLEVER score of the object detection model (f). - Compare the determined robustness value (RV) with a predefined robustness threshold (RT), and Output unit (33) is configured as follows: - If the robustness value (RV) is lower than the predefined robustness threshold (RT), then output a positive proof for applying the object detection model (f) in control and / or monitoring tasks. Each robustness requirement is defined by stating the scenarios in which the object detection model (f) is not considered robust.

14. A computer program product that can be directly loaded into the internal memory of a digital computer, comprising a software code portion for causing the digital computer to perform all steps of the method according to any one of claims 1-12 when the software code portion is run on the digital computer.

Citation Information

Patent Citations

  • Determining influence of attributes in recurrent neural networks trained on therapy prediction

    EP3564862A1

  • Verification component for verifying an artificial intelligence, ai, model

    EP4105846A1

  • Robustness detection method and device for image processing model, and program product

    CN112907552A

  • Robust neural network training method based on sample-driven target loss function optimization

    CN115438786A