Robustness measuring apparatus, robustness measuring method, and program

Abstract

To provide a robustness measuring apparatus, a robustness measuring method, and a program that measure robustness of a machine learning model in a simple and highly accurate manner, regardless of whether a supervisor exists or not.SOLUTION: In a robustness measuring apparatus, an acquisition part acquires a request of robustness measurement that is transmitted from a terminal device T, via a network NW. An inference result group generating part generates an inference result group obtained by grouping inference results associated with each other, among inference results of input images and change images. A robustness measuring part measures robustness of an inspection object model based on the inference result group. The inference result group is an object area candidate group of object detection results. The robustness measuring part determines, as statistical processing, whether an object area candidate group is a pseudo correct detection defined as correct detection, or a pseudo excessive detection defined as erroneous detection, and further, calculates a pseudo correct detection rate and a pseudo excessive detection rate on the basis of a determination result, in order to measure robustness of an inspection object model on the basis of the pseudo correct detection rate and the pseudo excessive detection rate.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a robustness measurement device, a robustness measurement method, and a program. [Background technology]

[0002] In the field of machine learning (Artificial Intelligence: AI) technology, methods have been proposed to evaluate the noise tolerance of a test model for a specific input image in class classification or regression problems. This method calculates the accuracy rate when arbitrary noise is added to the input image based on the correct data, and evaluates the noise tolerance of the test model. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Understanding Local Robustness of Deep Neural Networks under Natural Variations”, [online], [Retrieved November 10, 2022], Internet<URL:https: / / arxiv.org / abs / 2010.04821> Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional evaluation methods require the calculation of accuracy rates, so they assume the existence of correct answer data (correct labels) to serve as teacher data at the time of inference (evaluation). For this reason, if correct answer data to serve as teacher data does not exist at the time of inference, it is not possible to evaluate the noise resistance of a machine learning model. Furthermore, conventional evaluation methods are not compatible with machine learning models used for object detection.

[0005] The problem to be solved by the present invention is to provide a robustness measurement device, a robustness measurement method, and a program that can measure the robustness of a machine learning model simply and with high accuracy, regardless of whether it is supervised or unsupervised. [Means for solving the problem]

[0006] A robustness measurement device according to an embodiment includes an acquisition unit, a modified image generation unit, an inference result group generation unit, and a robustness measurement unit. The acquisition unit acquires an input image. The modified image generation unit generates at least one modified image by modifying the input image. The inference result group generation unit generates an inference result group by grouping inference results that are associated with each other, among inference results for at least one of the input image and the modified image. The robustness measurement unit measures the robustness of the model to be inspected based on the inference result group. The inference result is an object detection result for at least one of the input image and the modified image. The inference result group is a group of object region candidates for the object detection result. The robustness measurement unit measures the robustness of the model to be inspected by performing statistical processing on the object detection results included in the object region candidate group. In addition, as the statistical processing, the robustness measurement unit determines whether the group of object region candidates is a pseudo-positive detection, which is defined as a correct detection, or a pseudo-overdetection, which is defined as an erroneous detection, calculates a pseudo-positive detection rate and a pseudo-overdetection rate based on the determination result, and measures the robustness of the model to be inspected based on the pseudo-positive detection rate and the pseudo-overdetection rate. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a functional block diagram showing an example of the functional configuration of a robustness measurement device 1 according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing how a boundary box BB is set for an image according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing another example of how a boundary box BB is set for an image according to the first embodiment. [Figure 4]FIG. 10 is a diagram showing yet another example of how a boundary box BB is set for an image according to the first embodiment. [Figure 5A] FIG. 2 is a diagram showing an example of a robustness calculation method according to the first embodiment. [Figure 5B] FIG. 5B is a diagram showing an example of a method for calculating a false positive detection rate in the robustness calculation method of FIG. 5A. [Figure 5C] FIG. 5B is a diagram showing an example of a method for calculating a false overdetection rate in the robustness calculation method of FIG. 5A. [Figure 6] 10 is a flowchart showing an example of robustness measurement processing according to the first embodiment. [Figure 7A] 4 is a flowchart showing an example of a first method of robustness measurement according to the first embodiment. [Figure 7B] 10 is a flowchart showing an example of a second method of robustness measurement according to the first embodiment. [Figure 7C] 10 is a flowchart showing an example of a third method of robustness measurement according to the first embodiment. [Figure 7D] 10 is a flowchart showing an example of a fourth method of robustness measurement according to the first embodiment. [Figure 7E] 10 is a flowchart showing an example of a fifth method of robustness measurement according to the first embodiment. [Figure 8] FIG. 10 is a functional block diagram showing an example of the functional configuration of a robustness measurement device 1A according to a second embodiment. [Figure 9] 10A and 10B are diagrams illustrating a process of setting a weight for each noise condition according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a determination result (with a supervised method) of overdetection and non-detection according to the third embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a determination result (unsupervised) of correct detection and overdetection according to the third embodiment. [Figure 12] FIG. 10 is a diagram showing an example of an inference result group generation method according to a modified example. [Figure 13] FIG. 10 is a diagram showing an example of an inference result group generating method according to a comparative example of the modified example. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a robustness measurement device, a robustness measurement method, and a program according to an embodiment will be described with reference to the drawings.

[0009] First Embodiment The robustness measurement device of the first embodiment measures the robustness of a machine learning model against arbitrary noise. This robustness measurement device makes it possible to simply and accurately measure the robustness of a machine learning model, regardless of whether it is supervised or unsupervised (regardless of whether ground truth data is present or not). Robustness refers to the ability to accurately perform inference on various data. For example, robustness refers to the ability to prevent deviations in inference results when images with various noises are input to the model under test.

[0010] In the following description, the robustness measurement device will be described as an example in which it is implemented as a single server device connected to a user's terminal device and the inference device via a network. However, the robustness measurement device may be implemented as a plurality of devices, or may be implemented by being installed in a user's terminal device. Alternatively, the robustness measurement device may be implemented using cloud computing technology. The robustness measurement device and the inference device may be implemented as a single device.

[0011] [Device configuration] 1 is a functional block diagram showing an example of the functional configuration of a robustness measurement device 1 according to the first embodiment. The robustness measurement device 1 is communicably connected to a terminal device T and an inference device ID via a communication network NW. The communication network NW includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider device, a wireless base station, etc.

[0012] The terminal device T is operated by a user U who inspects, for example, a machine learning model to be inspected (hereinafter referred to as a "model to be inspected") or software that performs inference processing using the machine learning model. The terminal device T is, for example, a device such as a personal computer, a smartphone, or a tablet computer.

[0013] The inference device ID, for example, performs a predetermined inference process (e.g., object detection process) on an input image in response to a request from an external device (robustness measurement device 1), and provides the inference result to the requestor. The inference device ID may be realized, for example, by software that operates using a plug-in function. The inference device ID includes, for example, an inference unit 100 and a memory unit 110. The inference unit 100 inputs an input image input from outside into a machine learning model, and provides the inference result, which is the output of this machine learning model, to the requestor. The memory unit 110 stores a machine learning model (test target model M).

[0014] The test object model M is generated by various description methods such as a neural network, a support vector machine, a decision tree, etc. The neural network includes, for example, an autoencoder, a convolutional neural network (CNN), a recurrent neural network (RNN), etc.

[0015] The robustness measurement device 1 includes, for example, a control unit 10 and a storage unit 20. The control unit 10 includes, for example, an acquisition unit 11, a change image generation unit 12, an image input unit 13, an inference result acquisition unit 14, an inference result group generation unit 15, a robustness measurement unit 16, and a provision unit 17. Each functional unit of the control unit 10 is realized by a computer, such as a central processing unit (CPU) or a graphics processing unit (GPU), executing a program. Some or all of the functional units of the control unit 10 may be realized by hardware, such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), or may be realized by a combination of software and hardware. The program may be stored in advance in the storage unit 20 (a storage device with a non-transitory storage medium), or may be stored on a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

[0016] The acquisition unit 11 acquires a robustness measurement request transmitted from the terminal device T via the network NW. The robustness measurement request includes information on the input image D used for the inspection and the inspection target model M (inference device ID). The acquisition unit 11 may acquire the input image D directly from the terminal device T, or may acquire it from another device such as a storage device (not shown) or a camera based on the information included in the request. Furthermore, if the input image D is stored in advance in the storage unit 20, the acquisition unit 11 may acquire the input image D from the storage unit 20. The acquisition unit 11 is an example of an "acquisition unit."

[0017] The variation image generation unit 12 generates a variation image ND by varying the input image D. In other words, the variation image generation unit 12 generates at least one variation image ND to which noise has been added by adding any noise to the input image D. The variation image generation unit 12 adds image variations (noise) to the input image D by, for example, rotating, flipping, barrel-shaped, pincushion-shaped, or shear distortion, Gaussian noise, switching color channels, grayscaling, or histogram equalization of the input image D, as well as reproducing weather noise such as rain or snow. This makes it possible to measure the robustness of the inspection target model M against noise expected when actually operating it. For example, the variation image generation unit 12 adds each of multiple different types of noise (e.g., three types of noise A, noise B, and noise C) to one input image D to generate multiple variation images ND (e.g., three variation images ND). The multiple different types of noise are, for example, noises with different noise levels. Each noise condition can be a combination of the above-mentioned noises such as rotation, and it is possible to impart noise designed by the user U to suit the actual operating environment, such as adding rotation and snow noise, or adding inversion and rain noise. The variation image generation unit 12 is an example of a "variation image generation unit." The variation image generation unit 12 may also be called a "noise imparting unit," and the variation image ND may also be called a "noise-added image."

[0018] The image input unit 13 calls the inference unit 100 of the inference device ID via the communication network NW and inputs at least one of the input image D and the modified image ND to the inspection target model M via the inference unit 100. The inference unit 100 then inputs at least one of the input image D and the modified image ND to the inspection target model M and obtains a predetermined inference result as an output of the inspection target model M. For example, the inference result includes a detection result of an object contained in the image. The object detection result is represented, for example, by a predetermined rectangular boundary line (boundary box BB) surrounding the object and a class label indicating the classification of the object. Alternatively, a method of outputting an estimated probability distribution for all classes of the detection target instead of class labels, or semantic segmentation that estimates a class label for each pixel instead of a rectangle may be used. The image input unit 13 is an example of an "image input unit."

[0019] Fig. 2 is a diagram showing how a boundary box BB is set for an image according to the first embodiment. In the example shown in Fig. 2, on an input image D in which a target object (person) TA is captured, there are shown a boundary box BB0 of the target object TA obtained by inputting the input image D (without noise) into an inspection target model M, a boundary box BB1 of the target object TA obtained by inputting a modified image ND in which noise A is added to the input image D into the inspection target model M, a boundary box BB2 of the target object TA obtained by inputting a modified image ND in which noise B is added to the input image D into the inspection target model M, and a boundary box BB3 of the target object TA obtained by inputting a modified image ND in which noise C is added to the input image D into the inspection target model M.

[0020] The inference result acquisition unit 14 acquires and aggregates the inference results for each image output by the inference unit 100, and inputs them to the inference result group generation unit 15. The inference result acquisition unit 14 is an example of an "object detection result acquisition unit." The inference result acquisition unit 14 acquires, as the object detection result, the output result of the inspection target model M when at least one of the input image D and the modified image ND is input.

[0021] The inference result group generation unit 15 generates a group of inference results (a group of object region candidates) that are correlated with each other based on the inference results for each image (each noise condition). For example, the inference result group generation unit 15 compares the boundary boxes included in the inference results for each input image D and each modified image ND using an exhaustive search, and generates a group of inference results that are correlated with each other based on the degree of overlap. For example, the inference result group generation unit 15 calculates the overlap ratio between object detection regions (boundary boxes) based on the overlap rate IoU (Intersection over Union), compares the detection results for each noise condition, and groups the candidates with the highest overlap rate as object region candidates that indicate the same object, thereby generating an inference result group. For example, the inference result group generation unit 15 uses the boundary box BB0 of the input image D, which is a noise-free image, as a reference and generates a group of correlated inference results by comparing boundary boxes whose overlap rate IoU with this boundary box BB0 exceeds a predetermined threshold (for example, exceeds 0.5). Alternatively, the inference result group generation unit 15 may use, for example, the boundary box of the changed image ND with the lowest (or highest) noise level as a reference and generate, as a group of mutually associated inference results, boundary boxes whose overlap rate IoU with this boundary box exceeds a predetermined threshold (for example, exceeds 0.5). The inference result group generation unit 15 is an example of an "inference result group generation unit."

[0022] In the example shown in Figure 2, for example, using boundary box BB0 (no noise) as the reference, boundary boxes BB1, BB2, and BB3 (a total of four boundary boxes) whose overlap rate IoU with boundary box BB0 exceeds a predetermined threshold are grouped as an inference result group.

[0023] 3 is a diagram showing another example of how a boundary box BB is set for an image according to the first embodiment. In the example shown in FIG. 3, the overlap rate IoU of boundary box BB3 with boundary box BB0 is equal to or less than a predetermined threshold. In this case, boundary boxes BB1 and BB2 (a total of three boundary boxes excluding boundary box BB3) whose overlap rates IoU with boundary box BB0 exceed the predetermined threshold are grouped into an inference result group.

[0024] Fig. 4 is a diagram showing yet another example of how a boundary box BB is set for an image according to the embodiment. The example shown in Fig. 4 shows a boundary box BB1 based on a modified image ND in which noise A is added to input image D, and boundary boxes BB2-1 and BB2-2 based on a modified image ND in which noise B is added to input image D. Although both boundary boxes BB2-1 and BB2-2 overlap with boundary box BB1, boundary box BB2-1 has a higher overlap rate IoU (exceeds the threshold), and therefore boundary box BB1 and boundary box BB2-1 are grouped together as object region candidates indicating the same object, and are set as an inference result group.

[0025] The robustness measurement unit 16 measures the robustness of the test object model M based on the generated inference results. The robustness measurement unit 16 performs statistical processing on the inference results to calculate a robustness index. An object region can be represented by coordinates indicating the center of a rectangle, the width and height of the rectangle, the coordinates, width, and height of one point representing a corner of the rectangle, or the coordinates of two points representing the diagonal corners of the rectangle. Statistical processing can be applied to all of these measurements. By calculating at least one of the variance, maximum value, mean value, and median based on the rectangular coordinates for the same object under multiple noise conditions, it is possible to capture how the object detection region moves, expands, or shrinks due to the addition of noise. If the object region representing the same object does not change, the test object model M is considered to be robust to noise. Conversely, if there is a large change, it can be said to be vulnerable. In addition to the robustness for each detection region, the robustness for each image can also be considered. Since there are multiple object detection candidates in an image, the robustness of the entire image can be calculated by averaging the robustness for each detection region or using the maximum value. The reason for using the maximum value rather than the minimum value is that it is desirable to use the most vulnerable example as a representative in robustness assessment.

[0026] Furthermore, the object regions included in the inference result group have estimated class labels (estimated class labels). Object region candidates representing the same object are generally expected to have the same estimated class labels. However, the estimated class labels may change due to noise. A small variance in the estimated class labels is considered robust, while a large variance is considered fragile. In conventional object detection, the accuracy assessment based on comparison with training data (correct answer data) is only applied to object region candidates with the same class labels as the training data. In this embodiment, however, the same object is determined regardless of the estimated class labels. The robustness of the image can be calculated by averaging the robustness of each detected region for the estimated class labels or by using the maximum value. The maximum value, rather than the minimum value, is used because it is desirable to represent the weakest example in robustness assessment. That is, the robustness measurement unit 16 calculates the variance of the labels identifying objects included in the object detection results, and calculates the robustness of the object coordinates and the robustness of class identification based on the labels.

[0027] The inference result group generation unit 15 compares object detection results for each noise condition and generates an inference result group. Candidates with an IoU overlap rate above a threshold are grouped as object region candidates representing the same object. However, object regions not detected under other noise conditions have an IoU overlap rate below the threshold, resulting in no group being generated, or candidates being missing under some noise conditions. The inference result group generation unit 15 does not consider objects below a certain threshold to be the same object, and assigns an undetected label if there is a noise condition where no candidate in the group has an overlap rate above the threshold. In the inference result group representing the same object, the unit determines whether or not an overdetection has occurred based on the proportion of undetected labels assigned by the change image generation unit 12 to the noise. For example, if there are two noise conditions, one non-noise condition, and a threshold of 0.5, and two undetected conditions are present, the detection rate is 33.3%, below the threshold. If the result is below the threshold, it is considered a false over-detection (FP'; False Positive'), and conversely, if it is above the threshold, it is considered a false positive (TP'; True Positive').The false precision rate Precision' can be calculated as an over-detection index, and is expressed by the following formula (1).It is also possible to use the false over-detection rate per image (FPPI'; False Positive Per Image') as an over-detection index.

[0028]

number

[0029] Alternatively, robustness may be measured using false positives and false overdetections calculated by a method different from the above. In this case, the robustness measurement unit 16 first selects a reference image from measurement target data including an input image and a modified image, and uses the inference result of the selected reference image as a pseudo-positive label. Next, the robustness measurement unit 16 determines whether each object detection candidate included in the inference result group is a false positive (TP; True Positive) or a false overdetection (FP; False Positive) based on the pseudo-positive label. For a specific inference result group, if the pseudo-positive label indicates that the inference result group is grouped with the pseudo-positive label, the robustness measurement unit 16 determines that the object detection result grouped with the pseudo-positive label is a false positive. For example, the robustness measurement unit 16 determines that the object detection result for the modified image, whose overlap rate IoU with the pseudo-positive label exceeds a threshold, is a false positive. On the other hand, if the pseudo-ground truth label for a particular inference result group indicates that the inference result group is not grouped with the inference result group, the robustness measurement unit 16 determines that the object detection results grouped with the inference result group are pseudo-overdetection. For example, the robustness measurement unit 16 determines that an object detection result whose overlap rate IoU with the pseudo-ground truth label exceeds a threshold for an inference result in a transition image is pseudo-overdetection.

[0030] The robustness measurement unit 16 uses the above judgment results to calculate the false positive rate (TPR; True Positive Rate) and the false over-detection rate (FPR; False Positive Rate). The false positive rate (TPR) is the ratio of the number of false positive detections to the number of detections of the calculation target. The false over-detection rate (FPR) is the ratio of the number of false over-detections to the number of detections of the calculation target. The number of detections of the calculation target is, for example, the number of measurement target data other than the reference image. When the input image is the reference image, the number of change images is the number of detections of the calculation target.

[0031] The false positive rate (TPR) can be calculated, for example, by the following formula (2). The false false positive rate (FPR) can be calculated, for example, by the following formula (3). In formulas (2) and (3), "|ND|" indicates the number of changed images ND, and "i" indicates the identification number of the inference. Furthermore, "Gi" indicates the inference result group for inference "i" (a set of boundary boxes grouped as inference results for inference "i"), and "|Gi|" indicates the number of inference result groups for inference "i" (the number of boundary boxes grouped as inference results for inference "i"). Furthermore, "B(D)" indicates the inference result group for each inference for input image D (a set of boundary boxes grouped as inference results in input image D). Furthermore, "B(D)∩Gi≠φ" indicates that the pseudo-correct label B(D) of the reference image (input image D) contains the inference result of inference "i" (the inference result group Gi contains the inference result of input image D), and "B(D)∩Gi=φ" indicates that the pseudo-correct label B(D) of the reference image (input image D) does not contain the inference result of inference "i" (the inference result group Gi does not contain the inference result of input image D).

[0032]

number

[0033] The robustness measurement unit 16 calculates the robustness R of the test object model M using the false positive rate (TPR) and the false over-detection rate (FPR). The robustness R is an example of a robustness index and is the robustness (overall robustness) of the input image D itself. The robustness R can be calculated, for example, by the following formula (4). In formula (4), "|g|" indicates the number of grouped inference results. In formula (4), the robustness R is calculated by summing the false positive rate (TPR) and the value obtained by subtracting the false over-detection rate (FPR) from 1. The value obtained by subtracting the false over-detection rate (FPR) from 1 may also be called the false positive rate or false non-over-detection rate. The larger the false positive rate (TPR) and the smaller the false over-detection rate (FPR), the larger the robustness R. In equation (4), the robustness R (global robustness) is calculated by averaging the robustness (local robustness) for each boundary box included in the input image D.

[0034]

number

[0035] An example of measuring robustness using the above-mentioned formulas (2) to (4) will be described with reference to Figures 5A to 5C. Figure 5A shows inference results for four face groups (face A group to face D group) for an input image D, a modified image ND1 obtained by adding a change 1 to the input image D, a modified image ND2 obtained by adding a change 2 to the input image D, and a modified image ND3 obtained by adding a change 3 to the input image D. Each of the changes 1 to 3 is an image process that adds an image change. For example, change 1 is an image process that darkens the image, change 2 is an image process that brightens the image, and change 3 is an image process that adds noise to the image.

[0036] 5A to 5C, boundary boxes grouped into one of the face A group to face D group (grouped as a specific inference result) are indicated by solid or dashed lines. In the example shown in FIG. 5A, boundary boxes grouped into the face A group and boundary boxes grouped into the face C group are detected in input image D. In other words, faces A and C are detected as inference results (object region candidates) from input image D. Similarly, faces A, C, and D are detected as inference results from modified image ND1, faces A, B, and C are detected from modified image ND2, and faces A and D are detected from modified image ND3.

[0037] The robustness measurement unit 16 selects a reference image from the measurement target data including the input image D and the modified images ND1 to ND3, and uses the inference result of the selected reference image as the pseudo-ground truth label. Here, a case where the input image D is selected as the reference image will be described. The robustness measurement unit 16 uses the inference result for the input image D (detection results of faces A to D) as the pseudo-ground truth label.

[0038] Next, the robustness measurement unit 16 determines whether each object detection result in each modified image is a pseudo-positive detection or a pseudo-overdetection based on the pseudo-positive labels. In FIGS. 5A to 5C, object detection results determined to be pseudo-positive detections are indicated by solid lines, and object detection results determined to be pseudo-overdetections are indicated by dashed lines. In the example shown in FIG. 5A, faces A and C are detected from input image D, which is the reference image. Therefore, the detection results of face A in modified image ND1, the detection results of face A in modified image ND2, and the detection results of face A in modified image ND3 are determined to be pseudo-positive detections. Similarly, the detection results of face C in modified image ND1 and the detection results of face C in modified image ND2 are determined to be pseudo-positive detections.

[0039] Furthermore, faces B and D are not detected in input image D, which is the reference image. Therefore, the detection result of face B in modified image ND2 is determined to be a false overdetection. Similarly, the detection result of face D in modified image ND1 and the detection result of face D in modified image ND3 are determined to be false overdetection.

[0040] Next, the robustness measurement unit 16 calculates the false positive rate (TPR) and the false over-detection rate (FPR) using equations (2) and (3). In the example shown in FIG. 5A, the equation for calculating the false positive rate (TPR) is expressed by the following equation (5), and the equation for calculating the false over-detection rate (FPR) is expressed by the following equation (6). In the input image D, faces A and C are detected, but faces B and D are not detected, so the false positive rate (TPR) is calculated for the face A group and the face C group, and the false over-detection rate (FPR) is calculated for the face B group and the face D group.

[0041]

number

[0042] 5B is a diagram showing how the false positive detection rate (TPR) for face C group is calculated. In face C group, the number of false positive detections is 3, and the number of change images is 3. Therefore, the false positive detection rate (TPR) for face C group is calculated as shown in the following formula (7) and is 2 / 3 (=0.67). Similarly, the false positive detection rate (TPR) for face A group is calculated as shown in the following formula (8) and is 3 / 3 (=1.00) because the number of false positive detections is 4.

[0043]

number

[0044] FIG. 5C is a diagram showing how the false over-detection rate (FPR) for the face D group is calculated. In the face D group, the number of false over-detections is 3, and the number of change images is 3. Therefore, the false over-detection rate (FPR) for the face D group is calculated as shown in the following formula (9) and is 2 / 3 (=0.67). Similarly, the false over-detection rate (FPR) for the face B group is calculated as shown in the following formula (10) and is 1 / 3 (=0.33) because the number of false over-detections is 1.

[0045]

number

[0046] Next, the robustness measurement unit 16 calculates the robustness R using equation (4). In the example shown in Fig. 5A, the robustness R is calculated as shown in the following equation (11), which is 2 / 3 (=0.67). "Face A" to "Face D" in equation (11) represent "Face A group" to "Face D group," respectively.

[0047]

number

[0048] <Modification of the method for selecting the reference image> When selecting a reference image, it is preferable to select data that is likely to be correctly detected as the reference image. Generally, an input image is more likely to be correctly detected than a modified image that has been given an image change. For this reason, in the examples of FIGS. 5A to 5C, the input image was selected as the reference image. However, if a deformed image is more likely to be correctly detected than the input image, it is preferable to select one of the deformed images as the reference image. In this modified example, a method for selecting an appropriate image as the reference image from measurement target data that includes a reference image and a deformed image will be described.

[0049] In this modification, the robustness measurement unit 16 calculates a reference image candidate value indicating the difference between the input image and the modified image and a group of images for selecting a reference image, and selects a reference image based on the reference image candidate value. The group of images for selecting a reference image is a group of images prepared for selecting a reference image. The group of images for selecting a reference image is preferably data that allows highly accurate inference using an object detection model. For example, the group of images for selecting a reference image can be training data used in training the object detection model or testing data used in performance testing before deployment.

[0050] The reference image candidate value is an index showing the difference in tendency from the group of images for selecting a reference image. For example, the reference image candidate value can be the distance between the feature value output from the object detection model when the image for selecting a reference image is input to the object detection model and the feature value output from the object detection model when the input image or the changed image is input to the object detection model. The robustness measurement unit 16 selects, as the reference image, the image with the smallest reference image candidate value, which indicates the difference from the group of images for selecting a reference image, from the input image and the changed image.

[0051] For example, the robustness measurement unit 16 uses an object detection model to extract feature values ​​from each of the input image, the modified image, and the group of images for reference image selection, and calculates the distance between the feature values ​​of each of the input image and the modified image and the group of images for reference image selection as a reference image candidate value.The robustness measurement unit 16 then extracts the image with the smallest reference image candidate value from among the input image and the modified image as an image with a similar tendency to the group of images for reference image selection, and selects the extracted image as the reference image.This allows data that is more likely to have been correctly detected to be used as the reference image, thereby enabling more accurate robustness measurement.

[0052] In the above-described modified example, taking advantage of the fact that data with a similar trend to the training data is likely to be correctly detected, an object detection model is used to extract features from the training data and the measurement target data (input image and change image), and a reference image is selected using the distance between the features. Instead of the distance between the features, the difference in trend between the intermediate outputs when extracting features using the object detection model may be measured, and an image with a small difference in trend may be selected as the reference image. In this case, for example, a neuron firing pattern may be used as the intermediate output.

[0053] Furthermore, instead of measuring the relationship with the training data, an SA (Surprise Adequacy) value indicating the quality of the data may be calculated for each of the input image and the modified image, and an image with a high SA value may be selected as the reference image.

[0054] The providing unit 17 provides the measured robustness information to the requesting terminal device T, for example, via the communication network NW. The user U can check the robustness of the test target model M by looking at the robustness information displayed on the display unit or the like of the terminal device T.

[0055] The storage unit 20 stores, for example, an input image D, a modified image ND, etc. The storage unit 20 is, for example, a storage device such as a hard disk drive (HDD), a random access memory (RAM), a solid state drive (SSD), or a flash memory. Note that the storage unit 20 may be realized by another storage device connected via a communication network NW, such as a network attached storage (NAS) or an external storage server device.

[0056] [Processing flow] Next, a description will be given of the flow of robustness measurement processing by the robustness measurement device 1. Fig. 6 is a flowchart showing an example of robustness measurement processing according to the first embodiment.

[0057] First, the acquisition unit 11 acquires a robustness measurement request (input image D) transmitted from the terminal device T via the network NW (step S101). Note that the acquisition unit 11 may acquire the input image D from another device such as a storage device (not shown) or a camera, or may acquire it from the storage unit 20.

[0058] Next, the variation image generation unit 12 adds noise to the input image D to generate at least one variation image ND (step S103). The variation image generation unit 12 adds noise to the input image D by, for example, rotating, flipping, barrel, pincushion, or shear distortion, Gaussian noise, switching color channels, grayscaling, or histogram equalization of the input image D, or reproducing weather noise such as rain or snow.

[0059] Next, the image input unit 13 calls the inference unit 100 of the inference device ID via the communication network NW, and inputs at least one of the input image D and the modified image ND to the inspection target model M via the inference unit 100 (step S105). Next, the inference result acquisition unit 14 acquires the inference result (e.g., object detection result) for each image output by the inference unit 100 (step S107).

[0060] Next, the inference result group generation unit 15 generates an inference result group (object region candidate group) based on the inference result for each image (for each noise condition) (step S109). For example, the inference result group generation unit 15 compares the boundary boxes included in the inference results for each input image D and modified image ND by exhaustive search, and generates a group of inference results that are associated with each other based on the degree of overlap.

[0061] Next, the robustness measurement unit 16 measures the robustness of the test target model M based on the group of inference results (step S111). The robustness measurement unit 16 performs statistical processing on the group of inference results to calculate a robustness index. The robustness index is broadly divided into the robustness of each boundary box included in the input image D (local robustness) and the robustness of the input image D itself (global robustness). Furthermore, robustness is measured based on different methods depending on whether the test target model M corresponds to "supervised" or "unsupervised" model, whether it corresponds to "regression" or "classification", etc.

[0062] <Measurement of boundary box robustness (local robustness)> (1) Method 1 (Supervised and Regression) 7A is a flowchart showing an example of a first method of robustness measurement according to the first embodiment. First, the robustness measurement unit 16 calculates the overlap ratio IoU of the boundary boxes BB included in each of the input image D (noise-free) and the modified image ND with respect to the correct boundary box, which is correct data, based on the inference result group (step S201). Next, the robustness measurement unit 16 calculates the variance or mean of the calculated overlap ratio IoU and adopts it as a robustness value (step S203). This completes the processing of this flowchart.

[0063] (2) Method 2 (Supervised and Classification) 7B is a flowchart showing an example of a second method of robustness measurement according to the first embodiment. First, based on the inference result group, the robustness measurement unit 16 compares the correct class label, which is correct data, with the estimated class labels of the input image D (noise-free) and the modified image ND, and determines whether they match (step S211). Next, the robustness measurement unit 16 calculates the accuracy rate, which is the ratio of the number of estimated class labels that match the correct class label to the number of estimated class labels (equivalent to the number of images), and adopts this as a robustness value (step S213). This completes the processing of this flowchart. Note that the robustness measurement unit 16 may directly evaluate the estimated class labels (or the undetected class label if undetected) to calculate the Neighbor Accuracy and adopt this as a robustness value.

[0064] (3) Method 3 (Unsupervised and Regression) 7C is a flowchart showing an example of a third method of robustness measurement according to the first embodiment. First, the robustness measurement unit 16 calculates the variance based on the rectangular coordinates of the boundary boxes BB included in the same inference result group, and adopts this as the robustness value (step S221). This completes the processing of this flowchart.

[0065] (4) Method 4 (Unsupervised and Classification) 7D is a flowchart showing an example of the fourth method of robustness measurement according to the first embodiment. First, the robustness measurement unit 16 calculates the variance of the class labels of the boundary boxes BB included in the same inference result group, and adopts this as the robustness value (step S231). This completes the processing of this flowchart.

[0066] (5) Fifth method In the fifth method, robustness is measured using the method described in FIGS. 5A to 5C. FIG. 7E is a flowchart showing an example of the fifth method of robustness measurement according to the first embodiment. First, the robustness measurement unit 16 selects a reference image from the measurement target data (including the input image and the modified image) included in the same inference result group (step S241), calculates the false positive rate (TPR) and false over-detection rate (FPR) of each boundary box using the reference image as a reference (step S243), and calculates robustness using the false positive rate (TPR) and false over-detection rate (FPR) (step S245). This completes the processing of this flowchart.

[0067] <Measurement of overall image robustness (overall robustness)> Similar to the measurement of the robustness of the boundary box (local robustness) described above, the robustness of the entire image (global robustness) can also be measured using methods similar to the first to fifth methods described above. For example, similar to the first to fifth methods described above, the variance or mean (robustness) for each group of inference results is calculated, and the global robustness is measured by taking the average or maximum value of the multiple obtained robustness values.

[0068] Returning to Fig. 6, next, the providing unit 17 provides the measured robustness information to the requesting terminal device T, for example, via the communication network NW (step S113). The user U can check the robustness of the test target model M by looking at this robustness information displayed on the display unit or the like of the terminal device T. This completes the processing of this flowchart.

[0069] The robustness measurement device 1 of the first embodiment described above makes it possible to measure the robustness of a machine learning model simply and with high accuracy, regardless of whether it is supervised or unsupervised. Furthermore, the change image generation unit 12 can test the robustness of any machine learning model under any noise condition. Even when the inference result group generation unit 15 does not have correct answer data (unsupervised), the robustness measurement unit 16 can measure a robustness index from statistical information on the inference results included in the inference result group. Furthermore, a robustness index can be measured for both regression and classification inference processes.

[0070] Second Embodiment The second embodiment will be described below. The robustness measurement device of the second embodiment differs from the first embodiment in that the robustness index is calculated taking into account the importance of each noise condition. The robustness measurement device of the second embodiment will be described below, focusing on the differences from the first embodiment.

[0071] 8 is a functional block diagram showing an example of the functional configuration of a robustness measurement device 1A according to the second embodiment. A control unit 10A of the robustness measurement device 1A includes, for example, an acquisition unit 11, a transition image generation unit 12, an image input unit 13, an inference result acquisition unit 14, an inference result group generation unit 15, a robustness measurement unit 16, a provision unit 17, and a noise weight setting unit 18.

[0072] The noise weight setting unit 18 sets a weight (noise weight) for each noise condition, taking into consideration the importance of each noise condition assigned by the transition image generation unit 12. The noise weight is defined, for example, by a user U of the terminal device T, an administrator of the robustness measurement device, or the like. FIG. 9 is a diagram illustrating the process of setting a weight for each noise condition according to the second embodiment. As shown in FIG. 9, when three boundary boxes BB1 to BB3 are generated using three noises (noise a, noise b, and noise c), the center coordinates of the object detection candidate group are calculated using the following equation (12). In equation (12), (Xa, Ya), (Xb, Yb), and (Xc, Yc) are representative points (center coordinates, upper left coordinate, lower left coordinate, upper right coordinate, lower right coordinate, midpoint of each side, etc.) of the boundary box BB1, the boundary box BB2, and the boundary box BB3, and (Xcenter, Ycenter) are the center coordinates.

[0073]

number

[0074] As an example of noise weighting, if the noise weight for noise a is Ma = 0.5, the noise weight for noise b is Mb = 0.3, and the noise weight for noise c is Mc = 0.2, noise A will be prioritized. If the distances to the representative points (e.g., center coordinates) of each boundary box BB1, BB2, and BB3 are La, Lb, and Lc, respectively, based on the center coordinates (Xcenter, Ycenter), the weighted distances will be MaLa, MbLb, and McLc. The robustness measurement unit 16 measures the robustness of the inspection target model M, for example, by calculating the variance or average of the weighted distances MaLa, MbLb, and McLc. By statistically processing the weighted distances in this way, a robustness index corresponding to the set noise weights is calculated. For example, in the case of in-vehicle image processing, a noise simulating nighttime conditions may be prioritized in consideration of the high rate of nighttime accidents, while a noise simulating rain or snow, which are relatively unlikely to be encountered, may be prioritized.

[0075] Although the above description has been given taking an example where the number of noises is 3, the present embodiment is not limited to this. When the number of noises is n, the central coordinates are calculated by dividing the sum of the x coordinates and the sum of the y coordinates of the representative points of the boundary boxes corresponding to each noise by n, and the weighted distance is calculated as described above, thereby making it possible to measure the robustness of the inspection target model M.

[0076] The robustness measurement device 1A of the second embodiment described above makes it possible to measure the robustness of a machine learning model simply and with high accuracy, regardless of whether it is supervised or unsupervised. Furthermore, the change image generation unit 12 allows the robustness of any machine learning model to be tested under any noise condition. Even when the inference result group generation unit 15 does not have correct answer data (unsupervised), the robustness measurement unit 16 can measure a robustness index from statistical information on the inference results included in the inference result group. Furthermore, a robustness index can be measured for both regression and classification inference processes. Furthermore, the noise weight setting unit 18 enables robustness measurement that prioritizes desired noise conditions.

[0077] Third Embodiment The third embodiment will be described below. The robustness measurement device of the third embodiment differs from the first embodiment in that it is capable of determining overdetection and non-detection by generating a group of object region candidates based on ground-truth data of the object detection result. The robustness measurement device of the third embodiment will be described below, focusing on the differences from the first embodiment.

[0078] The acquisition unit 11 acquires correct data (correct values) of the object detection result (boundary box BB) corresponding to the input image D. The acquisition unit 11 acquires the correct data from, for example, the terminal device T, the storage unit 20, another storage device (not shown), etc. The inference result group generation unit 15 generates an object region candidate group based on the object detection result and correct values ​​for each input image D and the modified image ND. After generating the object region candidate group, the inference result group generation unit 15 calculates the overlap rate IoU of each boundary box BB included in the object region candidate group with respect to the correct data (object region) that serves as the teacher. If the overlap rate IoU exceeds a predetermined threshold (e.g., 0.5), the object region candidate group is considered to correspond to the teacher, and if it is equal to or less than the threshold, it can be said that there is no corresponding teacher. If there is no corresponding teacher, it is considered to be overdetection.

[0079] The robustness measurement unit 16 calculates the robustness of the regression index of the test target model M based on the overlap rate IoU for the correct answer data that serves as teacher. By calculating the variance and average of the overlap rate IoU for the correct answer data that serves as teacher, it is possible to know whether the estimated object detection area is stable. The robustness measurement unit 16 calculates the robustness of the classification index based on the accuracy rate by comparing it with the correct answer class label that serves as teacher. Regarding the robustness of the entire image (overall robustness), the robustness of the image can be calculated by averaging the robustness of the regression index for each detection area or by adopting the maximum value. Similarly, the average value or maximum value can be adopted for the classification index to calculate the robustness index for the entire image.

[0080] Whether an object is overdetected or not is determined based on the proportion of noises assigned by the variation image generation unit 12 that are labeled as not detected in the group of object detection candidates representing the same object. FIG. 10 illustrates an example of a supervised determination result of overdetection and not detected according to the third embodiment. FIG. 10 illustrates an example in which three images, two noise-added images (noise A, noise B) and one non-noise image (non-noise), are used, and the detection determination threshold is set to 0.5. Case 1 illustrates a case in which, for the three images in the group of object detection candidates representing the same object, the overlap rate IoU with the correct data for noise A and non-noise exceeds the overlap rate threshold and the estimated class label matches the correct data (◯: correct), while for noise B, the overlap rate IoU with the correct data is below the overlap rate threshold or the estimated class label does not match the correct data (×: incorrect). In this case, the detection rate is two-thirds (0.66), exceeding the threshold. Case 1 is determined as a "true positive (TP)." Case 2, on the other hand, shows a case where, for three images in the object detection candidate group showing the same object, the IoU overlap rate with the correct data exceeds the threshold and the estimated class label matches the correct data (○: correct), while for noise A and noise B, the IoU overlap rate with the correct data is below the threshold or the estimated class label does not match the correct data (×: incorrect). In this case, the detection rate is one-third (0.33), which is below the threshold. Case 2 is determined as a "false negative (FN)." That is, if the detection rate is below the threshold and a teacher is present, it is determined as a "false negative (FN)." Conversely, if the detection rate is above the threshold and a teacher is present, it is determined as a "false negative (TP)."

[0081] On the other hand, in cases 3 and 4, three images in the object detection candidate group showing the same object do not overlap with the correct data (the overlap rate IoU is zero) (i.e., cases where there is no link to the correct data). Here, in case 3, estimated class labels are assigned to noise A and non-noise, and in case 4, an estimated class label is assigned to noise A, but these estimated class labels are likely to be incorrect. For this reason, cases 3 and 4 are determined to be "overdetections (FP; false positives)." In other words, when an estimated class label is assigned to a case that is not linked to the correct data, it is treated as an overdetection (FP) regardless of the detection rate. Thus, detection performance indicators such as precision, recall, and F-measure can be calculated from the correct detection (TP), nondetection (FN), and overdetection (FP). The overdetection rate for each image (FPPI; false positives per image) can also be calculated in a similar manner.

[0082] Even when correct answer data does not exist (unsupervised), it is possible to determine whether a detection has occurred correctly or incorrectly as follows. FIG. 11 is a diagram showing an example of the result of the correct answer and overdetection determination (unsupervised) according to the third embodiment. FIG. 11 illustrates an example in which three images, two noise-added images (noise A, noise B) and one non-noise image (non-noise), are used, and the detection determination threshold is set to 0.5. In Case 5, noise A and non-noise are set as an object detection candidate group indicating the same object, and their estimated class labels match, but noise B is not included in the object detection candidate group for noise A and non-noise. In this case, the detection rate is two-thirds (0.66), exceeding the threshold. Case 5 is determined as a "pseudo correct detection (TP')." The word "pseudo" is used because there is no correct answer data in the unsupervised case, and there is no certainty that the result is correct. On the other hand, in Case 6, an estimated class label is assigned to the non-noise image, but undetected labels are assigned to noise A and B. In this case, the detection rate is one-third (0.33), which exceeds the threshold. Case 6 is determined to be a "pseudo over-detection (FP')."

[0083] That is, the robustness measurement unit 16 calculates the detection rate within the object region candidate group based on the undetected labels assigned to object detection results that do not overlap with other object detection results in the object region candidate group that indicate the same object, and calculates an overdetection index based on the calculated detection rate.

[0084] The robustness measurement device 1 of the third embodiment described above makes it possible to measure the robustness of a machine learning model simply and with high accuracy, regardless of whether it is supervised or unsupervised. Furthermore, the change image generation unit 12 can test the robustness of any machine learning model under any noise condition. Even when the inference result group generation unit 15 does not have correct answer data (unsupervised), the robustness measurement unit 16 can measure a robustness index from statistical information on the inference results included in the inference result group. Furthermore, a robustness index can be measured for both regression and classification inference processes.

[0085] It also makes it possible to obtain an index of overdetection, which is important in object detection. In addition to the robustness of the detection results, the possibility that the detection itself is an overdetection can be suggested, which makes it possible to take measures when re-learning based on data with low robustness, such as adjusting the detection threshold for object detection or not using results with a certain level of overdetection potential for training. Furthermore, it becomes possible to measure robustness more accurately by utilizing training data (ground truth data).

[0086] <Modification of the method for generating a group of inference results> In the above embodiment, an example has been described in which an inference result group is generated using the overlap rate IoU between the object detection region (boundary box) of the input image and the object detection region (boundary box) of the changed image. In this modification, the inference result group generation unit 15 identifies the inference basis locations used as the basis for the inference, and generates an inference result group using the inference basis locations. In this case, the inference result group generation unit 15 generates an inference result group based on the overlap rate between the inference basis locations. As the inference basis locations, for example, a heat map indicating the locations used as the basis for inference when making the inference can be used. Note that this modification is particularly effective when the image change is a geometric transformation.

[0087] 12 is a diagram showing how an inference result group is generated in this modified example. As shown in FIG. 12, in this modified example, a modified image RD is generated by applying a rotation to an input image D, and a boundary box BB0 is detected as an object detection region by inputting the input image D to an inspection target model M. A boundary box BB1 is detected as an object detection region by inputting the modified image RD to the inspection target model M. The inference result group generation unit 15 acquires, from the inspection target model M, an inference basis location that was used as an inference basis in the input image D when generating the boundary box BB0, and generates an inference basis heat map I based on the acquired inference basis location. Similarly, the inference result group generation unit 15 acquires, from the inspection target model M, an inference basis location that was used as an inference basis in the modified image RD when generating the boundary box BB1, and generates an inference basis heat map J based on the acquired inference basis location.

[0088] Next, the inference result group generation unit 15 performs an inverse transformation of the added image change on the heat map J to generate an inversely transformed heat map J. For example, if image rotation is added as an image change, the inference result group generation unit 15 performs an image rotation in the reverse direction on the heat map J.

[0089] Next, the inference result group generation unit 15 calculates the overlap rate between the heat map I and the heat map J after inverse transformation, and generates an inference result group based on the calculated overlap rate. For example, areas where the overlap rate between the heat maps is greater than a threshold are grouped as object area candidates indicating the same object.

[0090] The overlap rate of heat map I and heat map J can be expressed, for example, by the following formula (13). Formula (13) shows the overlap rate of object detection areas detected for a specific inference. In formula (13), "I(x)" is the pixel value at position x in heat map I, "J(x)" is the pixel value at position x in heat map J, and "N" is the number of pixels in heat map I and heat map J. In formula (13), the smaller pixel value of heat map I or heat map J is used at each position and the used pixel values ​​are summed up, so the more the areas used as the basis for inference overlap, the larger the overlap rate value.

[0091]

number

[0092] In this modification, by comparing heat maps indicating the basis for inference, the accuracy of generating a group of inference results can be improved even when the image change is a geometric transformation. FIG. 13 shows a comparative example in which boundary boxes are compared for a modified image ND generated by geometric transformation. Simply comparing the boundary box BB0 of the input image D with the boundary box BB1 of the modified image ND generated by image rotation reduces the accuracy of generating a group of inference results, even if they would be appropriately grouped as the same object region candidate. Image rotation also changes the position of the boundary box, reducing the accuracy of generating a group of inference results. Furthermore, even when comparing boundary boxes after performing reverse rotation on the boundary box BB1 of the modified image ND, the shape of the boundary box (detection rectangle) is distorted by the rotation, reducing the accuracy of generating a group of inference results. In contrast, in this modification, by using a heat map that exhibits minimal shape change when reverse rotation is applied instead of a rectangular boundary box, object detection regions that would be appropriately grouped as the same object region candidate are appropriately grouped as the same object region candidate, improving the accuracy of generating a group of inference results.

[0093] <Example of use> One possible application of the robustness index measured as described above is as a measure of reliability. A commonly used method is to combine deep neural networks (DNNs) with Bayesian inference to output the reliability of the results. Since high robustness is considered to be high reliability, robustness measurements can be applied to machine learning models that do not have a reliability output and used as a substitute for reliability. Even for methods that do have a reliability output, it is possible to compare the reliability output from Bayesian inference with the results of robustness measurements to determine the degree to which it correlates with practical noise tolerance, thereby enabling the reliability of the reliability output to be measured.

[0094] As an example of its use as a measure of reliability, if a single piece of vulnerable data is found, it may be possible to determine that the inference results are unreliable. For example, in an onboard surveillance system such as an autonomous driving assistance system, if camera images are unreliable, measures can be taken such as prioritizing the detection results of other sensors such as LiDAR. In this way, robustness measurement can be effective in applications where data is updated constantly and reliability must be measured at any time. This also applies when using AI to track wanted criminals using surveillance cameras or to detect the presence of trouble. It also applies to systems such as those using cameras mounted on trains to capture and detect the intrusion of suspicious objects onto the tracks.

[0095] On the other hand, reliability can also be utilized in cases where specific scenes are important when analyzing data later, rather than in real time. For example, this applies to accident footage recorded by a dashcam, where only a few frames of the moment of the accident are focused on and object detection results in the image are important. Surveillance cameras may also be analyzed later in some cases, such as when crime scenes are filmed and then inspected later. Furthermore, in sports video recordings, it is necessary to focus on specific scenes, such as when a goal is scored or a foul is committed, which can be useful when reviewing team play. In sports cases, a system that assists referees in real time is also conceivable.

[0096] In addition to being used as a measure of reliability, robustness measurements can also be used for retraining. By collecting data determined to be vulnerable based on the results of robustness measurements and retraining the object detection model that was being tested, it is possible to build a model that is more robust against vulnerable data. Rather than simply using data determined to be vulnerable, it is also possible to focus on the vulnerability of the class labels and consider additional training for the class to which the vulnerable data belongs. For example, if the tested object detection model is determined to be weak against motorcycles, scenes containing motorcycles can be collected and retrained. If it is difficult to obtain new training data, it is possible to adjust the division of training data and test data so that the training data contains more images of motorcycles.

[0097] There are two types of robustness indices: robustness for each detected object region (boundary box) and robustness for each image. When using a robustness indices as a measure of reliability, it is sufficient to measure the robustness for each boundary box. On the other hand, from the perspective of re-training, re-training is performed using vulnerable images, so robustness for each image can be adopted. However, when adding categories during re-training, information is collected for each boundary box, and vulnerable labels are identified and used.

[0098] Furthermore, object detection can be considered a combination of two tasks: regression of detected coordinates and classification of detected classes, and robustness indices can be calculated independently as regression indices and classification indices. For example, when detecting a vehicle and combining 3D measurement technology to estimate its position and distance, and using this in autonomous driving, a regression indices is adopted because the location of obstacles that need to be avoided is important. On the other hand, when using surveillance cameras to find wanted suspects, the classification labels of the detection results are considered important, and the indices to be adopted can vary depending on the application. When both regression and classification are important, it is possible to integrate the two indices, regression indices and classification indices, into a single robustness indices by linearly combining them, for example.

[0099] Overdetection metrics, such as pseudo-precision for unsupervised examples and F-score for supervised examples, can be considered a type of robustness for each image. The average overdetection metric across the entire dataset can be monitored, and if it exceeds a threshold, measures such as adjusting the object detection threshold of the object detection model can be taken. Furthermore, to classify these metrics into two categories—absolutely robust or weak—a threshold for the robustness metric must be established. This requires using multiple datasets and establishing a threshold for the domain being tested. However, it is easy to compare robustness metrics within the dataset being tested and identify relatively weak data points. For example, one possible measure is to extract data in the bottom 10% of robustness metrics and retrain the model.

[0100] Thus, according to any of the above-described embodiments, it is possible to provide a robustness measurement device, a robustness measurement method, and a program that can measure the robustness of a machine learning model simply and with high accuracy, regardless of whether it is supervised or unsupervised.

[0101] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0102] 1, 1A... robustness measurement device, 10... control unit, 11... acquisition unit, 12... change image generation unit, 13... image input unit, 14... inference result acquisition unit, 15... inference result group generation unit, 16... robustness measurement unit, 17... provision unit, 18... noise weight setting unit, 20... memory unit, 100... inference unit, 110... memory unit, T... terminal device, NW... communication network, ID... inference device

Claims

1. an acquisition unit that acquires an input image; a modified image generating unit that generates at least one modified image by modifying the input image; an inference result group generation unit that generates an inference result group by grouping inference results that are mutually associated among the inference results for at least one of the input image and the modified image; a robustness measurement unit that measures the robustness of the model to be tested based on the group of inference results; Equipped with the inference result is an object detection result in at least one of the input image and the transition image; the inference result group is a group of object region candidates of the object detection result, the robustness measurement unit measures the robustness of the inspection target model by performing statistical processing on the object detection results included in the object region candidate group; the robustness measurement unit determines, as the statistical processing, whether the object region candidate group is a false positive detection defined as a correct detection or a false over-detection defined as an erroneous detection, calculates a false positive detection rate and a false over-detection rate based on the determination result, and measures the robustness of the inspection target model based on the false positive detection rate and the false over-detection rate. Robustness measurement device.

2. the inference result group generation unit generates an inference result group by grouping the inference results based on an overlap rate of the object detection results. The robustness measurement device according to claim 1 .

3. the inference result group generation unit identifies an inference basis portion used as a basis for the inference for each of the inference results, and generates an inference result group by grouping the inference results based on the inference basis portion. The robustness measurement device according to claim 1 .

4. the inference result group generation unit performs the grouping based on an overlap rate of the inference basis portions. The robustness measurement device according to claim 3 .

5. the robustness measurement unit selects a reference image from the input image and the modified image, and determines whether the object detection result is the false positive detection or the false over-detection based on the object detection result in the reference image.

5. A robustness measurement device according to any one of claims 1 to 4.

6. the robustness measurement unit calculates a reference image candidate value indicating a difference between the input image and the modified image and a group of images for selecting a reference image, and selects the reference image based on the reference image candidate value; 6. The robustness measurement device according to claim 5.

7. the robustness measurement unit selects, as the reference image, an image from the input image and the modified image that is small in difference from a group of images for selecting a reference image, based on the reference image candidate value; 7. The robustness measurement device according to claim 6.

8. an image input unit that inputs at least one of the input image and the modified image into the inspection object model; an object detection result acquisition unit that acquires, as the object detection result, an output result of the inspection target model when at least one of the input image and the modified image is input; Further provided with The robustness measurement device according to claim 1 .

9. the robustness measurement unit compares the object detection results for the input image and the modified image, and groups the object detection results with the largest overlap ratio as the object region candidate group indicating the same object. The robustness measurement device according to claim 2 .

10. The acquisition unit further acquires a correct answer value of the object detection result, the inference result group generation unit generates the object region candidate group based on the object detection results and the correct answer values ​​for each of the input image and the modified image. The robustness measurement device according to claim 1 .

11. The computer Take an input image, generating at least one transformed image by transforming the input image; generating a group of inference results by grouping together inference results that are mutually associated among the inference results for at least one of the input image and the modified image; measuring the robustness of the model to be tested based on the group of inference results; 1. A method comprising: the inference result is an object detection result in at least one of the input image and the transition image; the inference result group is a group of object region candidates of the object detection result, measuring the robustness includes measuring the robustness of the inspection target model by performing statistical processing on the object detection results included in the object region candidate group; The statistical processing includes determining whether the object region candidate group is a false positive detection defined as a correct detection or a false over-detection defined as an erroneous detection, calculating a false positive detection rate and a false over-detection rate based on the determination result, and measuring robustness of the inspection target model based on the false positive detection rate and the false over-detection rate. Robustness measurement methods.

12. On the computer, Acquire the input image, generating at least one transformation image by transforming the input image; generating a group of inference results by grouping together inference results that are mutually associated among the inference results for at least one of the input image and the modified image; measuring the robustness of the model to be tested based on the group of inference results; A program, the inference result is an object detection result in at least one of the input image and the transition image; the inference result group is a group of object region candidates of the object detection result, measuring the robustness includes measuring the robustness of the inspection target model by performing statistical processing on the object detection results included in the object region candidate group; The statistical processing includes determining whether the object region candidate group is a false positive detection defined as a correct detection or a false over-detection defined as an erroneous detection, calculating a false positive detection rate and a false over-detection rate based on the determination result, and measuring robustness of the inspection target model based on the false positive detection rate and the false over-detection rate. program.

Images