Test result analysis device, test result analysis method, and test result analysis program product

Abstract

The evaluation value calculation section (22) calculates an evaluation value of each of a plurality of layers of an object detection model configured using a neural network, which detects an object in the detection image data, as an object layer, based on a heat map and a detection region in which the object is detected, the heat map indicating an activity degree of each pixel in the image data obtained based on an output result of the object layer. The layer selection section (23) selects at least a part of the plurality of layers based on the evaluation value.

Description

Technical Field

[0001] This application relates to techniques for analyzing the detection results of object detection models constructed using neural networks. Background Technology

[0002] The following action is performed: the detection results of the object detection model that detects objects from image data are analyzed to determine the missing learning data in the object detection model (see Patent Document 1).

[0003] The following action is performed: when analyzing the detection results, the parts that serve as the basis for the object detection model's judgment are visualized. GradCAM (Gradient-weighted Class Activation Mapping) is a technique used to visualize these parts.

[0004] Object detection models can be categorized into two-stage and one-stage models. A two-stage model identifies the object by narrowing down the area represented by the Region of Interest (RoI). A one-stage model determines the object and its location using a set of bounding boxes of specific sizes, called anchor boxes.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2019-192082 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] Regarding two-stage models, it is said that pooling layers after scaling down the RoI are suitable for GradCAM-based visualization. In contrast, for one-stage models, the layers suitable for GradCAM-based visualization vary depending on conditions such as the type of object and the size of the detected object.

[0010] Additionally, for two-stage models, the pooling layer after reducing the RoI may not be the most suitable for GradCAM-based visualization.

[0011] The purpose of this application is to identify layers suitable for visualizing the parts that serve as the basis for judgment in an object detection model.

[0012] Methods for solving problems

[0013] The detection result analysis device of this application has:

[0014] The evaluation value calculation unit takes multiple layers of an object detection model constructed using a neural network, which detects objects contained in the image data, as object layers, and calculates the evaluation value of each object layer based on a heatmap and the detection region of the detected object. The heatmap represents the activity level of each pixel in the image data obtained based on the output of the object layer.

[0015] The layer selection unit selects at least a portion of the layers from the plurality of layers based on the evaluation values ​​calculated by the evaluation value calculation unit.

[0016] The evaluation value calculation unit calculates the evaluation value based on the ratio of the activity to the inner and outer sides of the detection area as represented by the heat map.

[0017] The evaluation value calculation unit calculates the evaluation value based on the ratio of the total activity value inside the detection area to the total activity value outside the detection area.

[0018] When the activity level is higher than the activity threshold, the evaluation value calculation unit converts the activity level into a conversion activity level corresponding to the activity threshold. When the activity level is lower than the activity threshold, the unit converts the activity level into a conversion activity level corresponding to an activity threshold one level lower than the activity threshold. Based on this, the evaluation value is calculated.

[0019] The layer selection unit selects the benchmark number of layers among those whose evaluation values ​​are higher than the evaluation threshold.

[0020] The detection result analysis device also has a synthesis unit that synthesizes the heat map related to the layer selected by the layer selection unit to generate a composite map.

[0021] The compositing unit takes each pixel of the image data as an object pixel, and sets the highest activity among the activity of the object pixels represented by the heatmaps that are respectively related to the selected partial layers as the activity of the object pixels in the compositing image, thereby generating the compositing image.

[0022] In the test result analysis method of this application,

[0023] The evaluation value calculation unit takes multiple layers of an object detection model constructed using a neural network, which detects objects contained in the image data, as object layers. Based on the heatmap and the detection region of the detected object, it calculates the evaluation value of the object layer. The heatmap represents the activity of each pixel in the image data obtained based on the output of the object layer.

[0024] The layer selection unit selects at least a portion of the layers from the plurality of layers based on the evaluation values.

[0025] The test result analysis program of this application enables a computer to function as a test result analysis device to perform the following processes:

[0026] The evaluation value calculation process treats multiple layers of a neural network-based object detection model containing detected objects in the image data as object layers. Based on heatmaps and the detection regions of the detected objects, an evaluation value for each object layer is calculated. The heatmap represents the activity level of each pixel in the image data, obtained from the output of the object layers.

[0027] The layer selection process selects at least a subset of layers from the plurality of layers based on the evaluation values ​​calculated through the evaluation value calculation process.

[0028] Invention Effects

[0029] In this application, an evaluation value for a layer is calculated based on a heatmap representing the activity of each pixel in the image data and the detection region of the detected object. At least a subset of layers is then selected from multiple layers based on these evaluation values. This allows for the determination of layers suitable for visualization. Attached Figure Description

[0030] Figure 1 This is a structural diagram of the detection result analysis device 10 of Embodiment 1.

[0031] Figure 2 This is a flowchart illustrating the overall operation of the detection result analysis device 10 according to Embodiment 1.

[0032] Figure 3 This is a flowchart of the evaluation value calculation process in Implementation Method 1.

[0033] Figure 4 This is an explanatory diagram of the evaluation value calculation and processing in Implementation Method 1.

[0034] Figure 5 This is an explanatory diagram of the layer selection process and synthesis process in Implementation Method 1.

[0035] Figure 6 This is an explanatory diagram of the synthesis process in Implementation Method 1.

[0036] Figure 7 This is an explanatory diagram illustrating an example of the analysis based on the composite diagram 34 in Implementation Method 1.

[0037] Figure 8 This is an explanatory diagram regarding the case where there are two or more preferred criteria.

[0038] Figure 9This is an explanatory diagram regarding the case where there are two or more preferred criteria.

[0039] Figure 10 This is a structural diagram of the detection result analysis device 10 of modified example 1.

[0040] Figure 11 This is a flowchart of the evaluation value calculation process in Implementation Method 2.

[0041] Figure 12 This is an explanatory diagram of the binarization process in Implementation Method 2.

[0042] Figure 13 This is an explanatory diagram of the binarization effect of implementation method 2.

[0043] Figure 14 This is an explanatory diagram of the ternary processing of variation example 3.

[0044] Figure 15 This is an illustration of the n-value transformation process in variation example 3. Detailed Implementation

[0045] Implementation Method 1

[0046] ***Structure Description***

[0047] Reference Figure 1 The structure of the detection result analysis device 10 of Embodiment 1 is explained.

[0048] The detection result analysis device 10 is a computer that determines the layer suitable for visualizing the parts that serve as the basis for judging the object detection model.

[0049] The detection result analysis device 10 includes hardware such as a processor 11, memory 12, storage 13, and communication interface 14. The processor 11 is connected to other hardware via signal lines and controls these other hardware components.

[0050] Processor 11 is an IC (Integrated Circuit) that performs processing. As a specific example, processor 11 is a CPU (Central Processing Unit), DSP (Digital Signal Processor), or GPU (Graphics Processing Unit).

[0051] Memory 12 is a storage device for temporary data storage. As a specific example, memory 12 is SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory).

[0052] Memory 13 is a storage device for storing data. As a specific example, memory 13 is an HDD (Hard Disk Drive). In addition, memory 13 can also be removable recording media such as SD (Secure Digital) memory cards, CF (Compact Flash), NAND flash memory, floppy disks, optical discs, compact discs, Blu-ray discs, and DVDs (Digital Versatile Disk).

[0053] Communication interface 14 is an interface used for communicating with external devices. As a specific example, communication interface 14 is a port for Ethernet (registered trademark), USB (Universal Serial Bus), or HDMI (registered trademark, High-Definition Multimedia Interface).

[0054] As functional structural elements, the detection result analysis device 10 includes an image acquisition unit 21, an evaluation value calculation unit 22, a layer selection unit 23, and a synthesis unit 24. The functions of each functional structural element of the detection result analysis device 10 are implemented by software.

[0055] The memory 13 stores programs that implement the functions of each functional structural element of the detection result analysis device 10. These programs are read into the memory 12 by the processor 11 and executed by the processor 11. Thus, the functions of each functional structural element of the detection result analysis device 10 are implemented.

[0056] exist Figure 1 In this diagram, only one processor 11 is shown. However, there can be multiple processors 11, and multiple processors 11 can cooperate to execute programs that implement various functions.

[0057] ***Instructions for Action***

[0058] Reference Figures 2-6 This explains the operation of the detection result analysis device 10 in Implementation Method 1.

[0059] The operation process of the detection result analysis device 10 in Embodiment 1 is equivalent to the detection result analysis method in Embodiment 1. Furthermore, the program for implementing the operation of the detection result analysis device 10 in Embodiment 1 is equivalent to the detection result analysis program in Embodiment 1.

[0060] Reference Figure 2 This explains the overall operation of the detection result analysis device 10 in Implementation Method 1.

[0061] An object detection model is a model that detects objects contained in image data. Object detection models are constructed using neural networks. Because they use neural networks, object detection models consist of multiple layers.

[0062] ( Figure 2 Step S11: Image Acquisition and Processing

[0063] The image acquisition unit 21 acquires image data 31, which is the object of processing.

[0064] Specifically, the image acquisition unit 21 acquires image data 31 by reading image data 31 set in memory 13 by the user of the detection result analysis device 10.

[0065] ( Figure 2 Step S12: Evaluation value calculation and processing)

[0066] The evaluation value calculation unit 22 treats the multiple layers constituting the object detection model as object layers and calculates the evaluation value of the object layers.

[0067] At this time, the evaluation value calculation unit 22 calculates the evaluation value based on the heat map 33 and the detection area 32 of the object detected from the image data 31 obtained in step S11. The heat map 33 represents the activity of each pixel in the image data 31 obtained based on the output result of the object layer.

[0068] Reference Figure 3 This explains the evaluation value calculation process of Implementation Method 1.

[0069] ( Figure 3 Step S21: Object detection processing

[0070] The evaluation value calculation unit 22 takes the image data 31 obtained in step S11 as input and detects the target object using the object detection model. Here, the type of target object can be pre-specified or specified by the user. Figure 4 The text indicates the type of object specified as the bottle.

[0071] ( Figure 3 Step S22: Detection area determination process)

[0072] The evaluation value calculation unit 22 determines the area where the object was detected, i.e., the detection area 32, which was determined by detecting the object in step S21.

[0073] ( Figure 3 Step S23: Heatmap generation and processing)

[0074] The evaluation value calculation unit 22 uses the multiple layers that constitute the object detection model as object layers and generates a heatmap 33.

[0075] Specifically, the evaluation value calculation unit 22 generates a heatmap 33 related to the object layer based on the output result of the object layer obtained when detecting the object in step S21. The heatmap 33 represents the activity of each pixel in the image data 31 obtained in step S11. Here, it is assumed that the higher the activity of a pixel, the larger the value shown in the heatmap 33.

[0076] The object layer among the multiple layers constituting the object detection model consists only of convolutional and pooling layers that can generate heatmaps 33. Specifically, a layer capable of generating heatmaps 33 refers to a layer that can calculate gradients with multiple channels having more than two pixels in both the vertical and horizontal directions.

[0077] In Implementation 1, it is assumed that all layers capable of generating heatmap 33 are designated as object layers. However, it is also possible to designate only a portion of the layers capable of generating heatmap 33 as object layers. For example, it is also possible to designate only the layers after a certain layer among all the layers capable of generating heatmap 33 as object layers.

[0078] Furthermore, the later the layer in the multiple layers constituting the object detection model, the fewer pixels in the image data that are the object being processed. However, the evaluation value calculation unit 22 expands the output of the object layer to generate a heatmap 33 representing the activity of each pixel in the image data 31.

[0079] For example, in a certain layer, the number of pixels is sometimes 1 / 4 of the image data 31 obtained in step S11. In this case, one pixel represents four pixels of image data 31, so the evaluation value calculation unit 22 assumes that one pixel represents four pixels of image data 31 and generates a heatmap 33.

[0080] Furthermore, even in later-processed layers, the number of pixels in the output image data sometimes increases depending on the object detection model. The number of pixels in the output may vary from layer to layer, but in any case, the number of pixels in the output can be scaled up or down in a manner consistent with the number of pixels in the image data 31.

[0081] ( Figure 3Step S24: Evaluation Processing

[0082] The evaluation value calculation unit 22 takes the multiple layers that generated the heat map 33 as object layers, and calculates the evaluation value based on the heat map 33 generated in step S23 and the detection area 32 determined in step S22 for each object layer.

[0083] Specifically, the evaluation value calculation unit 22 calculates the evaluation value based on the ratio of activity shown in the heat map related to the target layer to that on the inner and outer sides of the detection area 32. In Embodiment 1, the evaluation value calculation unit 22 calculates the ratio of the total activity value on the inner side of the detection area 32 to the total activity value on the outer side of the detection area 32 as the evaluation value.

[0084] exist Figure 4 In this model, pixels with deeper shadows are considered to have higher activity. Figure 4 In the detection area 32, there are multiple pixels with darker shadow lines inside the detection area 32, and a small number of pixels with lighter shadow lines outside the detection area 32. Therefore, compared with the total activity value outside the detection area 32, the total activity value inside the detection area 32 is a larger value, with an evaluation value of 1 or higher.

[0085] Here, the ratio of the total activity value inside the detection region 32 to the total activity value outside the detection region 32 is used as the evaluation value. However, this is not a limitation; for example, the ratio of the total activity value inside the detection region 32 to the total activity value of the entire image data 31 can also be used as the evaluation value.

[0086] ( Figure 2 Step S13: Layer selection processing)

[0087] The layer selection unit 23 selects at least a portion of the layers constituting the object detection model based on the evaluation values ​​calculated in step S12.

[0088] Specifically, the layer selection unit 23 selects layers with a benchmark number of evaluation values ​​higher than the evaluation threshold. Therefore, if there are more than one benchmark number of layers with evaluation values ​​higher than the evaluation threshold, the benchmark number of layers is selected. On the other hand, if there are only fewer than one benchmark number of layers with evaluation values ​​higher than the evaluation threshold, all layers with evaluation values ​​higher than the evaluation threshold are selected.

[0089] The evaluation threshold is a pre-set value based on the degree to which a layer contributes to the detection of an object will be designated as an analysis object. The number of reference layers is a pre-set value based on the number of layers constituting the object detection model.

[0090] exist Figure 5In the dataset, the layers from layer conv0 to layer conv17_1 that received evaluation values ​​higher than the evaluation threshold of 1.0 are layers conv11, conv13, conv14_1, and conv14_2. Additionally, in... Figure 5 In this context, the evaluation value of layer conv11 is represented as inf. Inf indicates the following situation: all activated pixels are located inside the detection region 32, and the evaluation value is infinitely large.

[0091] exist Figure 5 In this example, assuming there are 5 baselines, all of the four layers conv11, conv13, conv14_1, and conv14_2 were selected.

[0092] ( Figure 2 Step S14: Synthesis Processing

[0093] The synthesis unit 24 synthesizes the heat map 33 related to the layer selected in step S13 to generate a composite map 34.

[0094] Specifically, the compositing unit 24 uses each pixel of the image data 31 as the object pixel, and sets the highest activity among the activity values ​​of the object pixels represented by the heatmaps related to the multiple layers selected in step S13 as the activity value of the object pixel in the compositing image 34, thereby generating the compositing image 34.

[0095] For example, such as Figure 6 As shown, regarding a certain pixel p i，j Let the activity levels of the heatmaps associated with the selected layers be 15, 239, and 76, respectively. In this case, the activity levels of pixel p in the synthesized image 34 are... i，j The highest activity level among 15, 239, and 76 was 239.

[0096] exist Figure 5 In this process, the heatmaps 33 related to the four layers conv11, conv13, conv14_1, and conv14_2 are synthesized to generate a composite image 34. Therefore, the activity level represented by each pixel in the composite image 34 represents the highest activity level among the layers conv11, conv13, conv14_1, and conv14_2.

[0097] ***Effects of Implementation Method 1***

[0098] As described above, the detection result analysis device 10 of Embodiment 1 calculates an evaluation value for each layer based on the heatmap 33 and the detection area 32, and selects a layer based on the evaluation value. This allows for the determination of layers suitable for visualization.

[0099] For a given layer, a higher total activity value within the inner side of detection region 32 indicates a higher likelihood that the layer's results contribute significantly to the detection of the target object. Specifically, a higher ratio of the total activity value within the inner side of detection region 32 to the total activity value outside the outer side of detection region 32 indicates that the layer's results are strongly reflected in the target object detection results. Furthermore, the ratio of the total activity value within the inner side of detection region 32 to the total activity value outside the outer side of detection region 32 is used as an evaluation metric in Embodiment 1.

[0100] Therefore, selecting layers with higher evaluation values ​​means selecting layers that are suitable for visualizing the parts that serve as the basis for judgment in object detection models.

[0101] The detection result analysis device 10 of Embodiment 1 selects a layer suitable for visualization and generates a composite image 34. Therefore, it is possible to generate a composite image 34 that appropriately represents the parts used as the basis for judging the object detection model. As a result, appropriate analysis can be performed on the object detection model.

[0102] Reference Figure 7 This illustrates an example of the analysis based on the synthetic diagram 34 in Implementation 1.

[0103] exist Figure 7 In the image, the type of object is a bicycle. Figure 7 In the analysis, the entire bicycle was not identified as detection area 32; only the area from the front wheel to near the sprocket was identified as detection area 32. That is, it cannot be said that the bicycle's state was properly detected by the object detection model.

[0104] In the composite image 34, the pedals, crank, a portion of the frame, and a portion of the front wheel are used as criteria for judgment. Furthermore, it is known that the handlebars, saddle, and rear wheel are not frequently used as criteria. Therefore, based on this result, for example, we could consider providing learning data related to the handlebars, saddle, and rear wheel—which are not frequently used as criteria—to the object detection model for learning.

[0105] In implementation method 1, Figure 2 In step S13, layers with a number of benchmarks among the layers whose evaluation values ​​are higher than the evaluation threshold are selected. Here, the number of benchmarks can also be 1. However, the number of benchmarks is preferably 2 or more.

[0106] When the number of reference layers is 1, the synthesis unit 24 can directly output the heat map 33 related to the selected layer as the synthesis map 34.

[0107] Reference Figure 8 and Figure 9 This indicates that the preferred number of references is 2 or more.

[0108] exist Figure 8 In the text, the type of object is a motorcycle. Figure 8 In the example, layers conv7, conv9, conv12, conv13, conv14_1, conv14_2, and conv15_1 obtained evaluation values ​​higher than the evaluation threshold of 1.0. Furthermore, in... Figure 8 In this example, assuming a baseline of 5, layers conv9, conv12, conv14_1, conv14_2, and conv15_1 are selected to generate composite image 34. The result, based on composite image 34, shows that the motorcycle's front wheel, its surrounding area, and part of the handlebars are the parts used for judgment. In the case of a motorcycle, it is evident that the exposed wheel is frequently used as a judgment criterion. Based on composite image 34, the object detection model appropriately makes the determination.

[0109] Here, the layer with the highest evaluation value is layer conv9. Therefore, with only one reference layer selected, i.e., only layer conv9 is selected when only the layer with the highest evaluation value is chosen. Thus, according to the composite diagram 34, a portion of the handlebar appears to be the part used as the basis for judgment. In this way, the evaluation value of a layer that only focuses on a narrow area becomes the highest, and it is possible that only a layer that cannot be said to properly represent the part used as the basis for judgment was selected.

[0110] exist Figure 9 In the image, the type of object is a bicycle. Figure 9 Image data 31 contains two bicycles.

[0111] exist Figure 9 In this context, the layer with the highest evaluation value is layer conv15_1. Therefore, when the number of references is 1, i.e., when only the layer with the highest evaluation value is selected, only layer conv15_1 is chosen. However, in layer conv15_1, only the area near the nearest bicycle is activated, and the area near the bicycles inside is not activated. Therefore, in the case of multiple objects, the evaluation value of a layer that only focuses on a portion of the objects becomes the highest, and it is possible that a layer that cannot be said to properly represent the parts used as the basis for judgment is selected.

[0112] ***Other Structures***

[0113] <Variation Example 1>

[0114] In Implementation Example 1, each functional structural element is implemented in software. However, as a variation 1, each functional structural element can also be implemented in hardware. Regarding this variation 1, aspects that differ from Implementation Example 1 will be explained.

[0115] Reference Figure 10This explains the structure of the detection result analysis device 10 in modified example 1.

[0116] When each functional structural element is implemented in hardware, the detection result analysis device 10 has an electronic circuit 15 to replace the processor 11, memory 12, and storage 13. The electronic circuit 15 is a dedicated circuit that implements the functions of each functional structural element, memory 12, and storage 13.

[0117] As for electronic circuit 15, it is assumed to be a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).

[0118] Each functional structural element can be implemented by one electronic circuit 15, or the functional structural elements can be distributed among multiple electronic circuits 15.

[0119] <Variation Example 2>

[0120] As a variation 2, some of the functional structural elements can be implemented in hardware, while the other functional structural elements can be implemented in software.

[0121] The processor 11, memory 12, storage 13, and electronic circuit 15 are collectively referred to as the processing circuit. That is, the functions of each functional structural element are implemented by the processing circuit.

[0122] Implementation Method 2

[0123] The difference between Embodiment 2 and Embodiment 1 is that, after the activity of each pixel represented by heatmap 33 is converted to an n-value, the evaluation value is calculated. Here, n is an integer greater than or equal to 2. In Embodiment 2, this difference will be explained, while the explanation of the similarities will be omitted.

[0124] ***Instructions for Action***

[0125] Reference Figure 11 and Figure 12 This explains the operation of the detection result analysis device 10 in Implementation Method 2.

[0126] The operation process of the detection result analysis device 10 in Embodiment 2 is equivalent to the detection result analysis method in Embodiment 2. Furthermore, the program for implementing the operation of the detection result analysis device 10 in Embodiment 2 is equivalent to the detection result analysis program in Embodiment 2.

[0127] Reference Figure 11This explains the evaluation value calculation process in Implementation Method 2.

[0128] Processing of steps S31 to S33 Figure 3 The processing of steps S21 to S23 is the same.

[0129] ( Figure 11 Step S34: n-value conversion)

[0130] The evaluation value calculation unit 22 performs n-value conversion on the heatmaps 33 of each layer generated in step S33. In embodiment 2, the evaluation value calculation unit 22 performs binarization on the heatmaps 33 of each layer.

[0131] Specifically, the evaluation value calculation unit 22 takes each pixel in the heatmap 33 as an object pixel. If the activity level of the object pixel is higher than the activity threshold, the activity level of the object pixel is converted to 1; if the activity level of the object pixel is lower than the activity threshold, the activity level of the object pixel is converted to 0. The result is, as follows: Figure 12 As shown, in heatmap 33, pixels with darker shaded lines have an activity level of 1, while other pixels have an activity level of 0. Figure 12 In the image, white represents pixels with an activity level of 1, and black represents pixels with an activity level of 0.

[0132] ( Figure 11 Step S35: Evaluation Processing

[0133] The evaluation value calculation unit 22 uses the activity converted in step S34 to calculate the ratio of the total activity value inside the detection area 32 to the total activity value outside the detection area 32 as the evaluation value.

[0134] exist Figure 12 In the detection area 32, there are 864 pixels with an activity level of 1 on the outer side and 2944 pixels with an activity level of 1 on the inner side. Therefore, the evaluation value is 2944 / 864 ≈ 3.4.

[0135] ***Effects of Implementation Method 2***

[0136] As described above, the detection result analysis device 10 of Embodiment 2 calculates an evaluation value after binarizing the activity of each pixel in the heatmap 33. By binarizing the activity, the evaluation value of important layers becomes higher, and the evaluation value of unimportant layers becomes lower. As a result, layers suitable for visualization can be determined more appropriately.

[0137] In Embodiment 2, as an example of binarization, when the activity is higher than the activity threshold, the converted activity is set to 1, and when the activity is below the activity threshold, the converted activity is set to 0. The converted activity for each activity threshold can be set to any number.

[0138] For example, as Figure 13 shown, in the case where binarization is not performed, it is a state where the evaluation values of multiple layers slightly exceed the evaluation threshold of 1.0. In contrast, in the case where binarization is performed, for some layers where the evaluation value slightly exceeds the evaluation threshold, the evaluation value becomes lower and is less than the evaluation threshold of 1.0. On the other hand, the evaluation values of the layers that originally had relatively high evaluation values become even higher. As a result, the number of layers whose evaluation values exceed the evaluation threshold of 1.0 decreases.

[0139] ***Other structures***

[0140] <Variant Example 3>

[0141] In Embodiment 2, the heat map 33 is binarized. However, not limited to binarization, n - value conversion can also be performed.

[0142] For example, in the case of ternarization, the evaluation value calculation unit 22 uses two thresholds, threshold X and threshold Y, as activity thresholds. In this case, as Figure 14 shown, for pixels whose activity is below threshold X, the converted activity is set to 0. For pixels whose activity is higher than threshold X and below threshold Y, the converted activity is set to 0.5. For pixels whose activity is higher than threshold Y, the converted activity is set to 1.

[0143] In this way, in the case of n - value conversion, n - 1 activity thresholds are set, and the converted activity for each activity threshold is set. For the n - value conversion process, the converted activity is determined for each range that divides the activity by i (0 < i < n) to i - 1. When the activity is higher than the n - 1 activity threshold, it is converted to the converted activity corresponding to the n - 1 activity threshold. When the activity is below the n - 1 activity threshold and higher than the n - 2 activity threshold, it is converted to the converted activity corresponding to the n - 2 activity threshold.

[0144] To simplify the calculation, for example, the converted activity corresponding to an activity threshold higher than n - 1 is set to 1, the converted activity corresponding to an activity threshold where the activity is below n - n + 1 (=1) is converted to 0, and for n - 2 to 2, the converted activity is converted to a value between 1 and 0 to perform n - value conversion. By setting the lower limit value of the converted activity to 0, the calculation can be simplified. In addition, the upper limit value of the converted activity is not limited to 1. It can also be set to other values.

[0145] Alternatively, the ReLU function, which shifts the threshold, can be used for n-value conversion.

[0146] Specifically, such as Figure 15 As shown, for pixels with an activity level below the evaluation threshold X, the activity level is set to 0. For pixels with an activity level higher than the evaluation threshold X, the activity level is maintained.

[0147] <Variation Example 4>

[0148] The detection result analysis device 10 in each embodiment can also be applied to object detection models used in automated guided vehicles (AGVs). In AGVs that use image recognition as a guidance method, the position of the vehicle is determined by reading marks drawn on the floor and ceiling. By applying the detection result analysis device 10 of each embodiment to the object detection model used in AGVs, the accuracy of the object detection model used in AGVs can be improved. As a result, AGVs capable of moving with higher precision can be provided.

[0149] The embodiments and variations of the present invention have been described above. Several of these embodiments and variations may also be implemented in combination. Furthermore, any one or more of these embodiments may be implemented partially. In addition, the present invention is not limited to the above embodiments and variations, and various modifications can be made as needed.

[0150] Label Explanation

[0151] 10: Detection result analysis device; 11: Processor; 12: Memory; 13: Storage; 14: Communication interface; 15: Electronic circuit; 21: Image acquisition unit; 22: Evaluation value calculation unit; 23: Layer selection unit; 24: Synthesis unit; 31: Image data; 32: Detection area; 33: Heat map; 34: Synthetic image.

Claims

1. A detection result analysis device, the detection result analysis device having: The evaluation value calculation unit takes multiple layers of an object detection model constructed using a neural network, which detects objects contained in the image data, as object layers. Based on a heatmap and the detection region of the detected object, it calculates the evaluation value of the object layer according to the ratio of the activity represented by the heatmap inside to outside the detection region. The heatmap represents the activity of each pixel in the image data obtained based on the output of the object layer. The evaluation value calculation unit calculates the evaluation value based on the ratio of the total activity value inside the detection region to the total activity value outside the detection region. A layer selection unit, based on the evaluation value calculated by the evaluation value calculation unit, selects at least a subset of layers from the plurality of layers suitable for visualizing the parts used as the judgment basis of the object detection model, wherein the layer selection unit selects a base number of layers among the layers whose evaluation value is higher than an evaluation threshold; and The compositing unit synthesizes the heatmaps related to the layers selected by the layer selection unit to generate a composite image representing the parts that serve as the judgment criteria for the object detection model. The compositing unit takes each pixel of the image data as an object pixel and sets the highest activity among the activity of the object pixels represented by the heatmaps related to the selected partial layers as the activity of the object pixel in the composite image, thereby generating the composite image.

2. The detection result analysis device according to claim 1, wherein, When the activity level is higher than the activity threshold, the evaluation value calculation unit converts the activity level into a conversion activity level corresponding to the activity threshold. When the activity level is lower than the activity threshold, the unit converts the activity level into a conversion activity level corresponding to an activity threshold one level lower than the activity threshold. Based on this, the evaluation value is calculated.

3. A method for analyzing test results, wherein, The evaluation value calculation unit uses multiple layers of an object detection model constructed using a neural network to detect objects contained in the image data, each as an object layer. Based on a heatmap and the detection region of the detected object, it calculates the evaluation value of the object layer according to the ratio of the activity represented by the heatmap inside to outside the detection region. The heatmap represents the activity of each pixel in the image data obtained based on the output of the object layer. The evaluation value calculation unit calculates the evaluation based on the ratio of the total activity value inside the detection region to the total activity value outside the detection region. The layer selection unit selects at least a subset of layers from the plurality of layers that are suitable for visualizing the parts used as the judgment basis of the object detection model, based on the evaluation value. The layer selection unit selects a base number of layers from the layers whose evaluation value is higher than an evaluation threshold. The compositing unit composites the heatmaps related to the layers selected by the layer selection unit to generate a composite image representing the parts that serve as the judgment criteria for the object detection model. The compositing unit takes each pixel of the image data as an object pixel and sets the highest activity among the activity of the object pixels represented by the heatmaps related to the selected partial layers as the activity of the object pixel in the composite image, thereby generating the composite image.

4. A test result analysis program product, comprising a test result analysis program that enables a computer to function as a test result analysis device for performing the following processes: The evaluation value calculation process involves treating multiple layers of a neural network-based object detection model containing objects in the detected image data as object layers. Based on heatmaps and the detection regions of the detected objects, the evaluation value of each object layer is calculated according to the ratio of the activity represented by the heatmap inside to outside the detection region. The heatmap represents the activity of each pixel in the image data obtained based on the output of the object layer. In the evaluation value calculation process, the evaluation is calculated based on the ratio of the total activity value inside the detection region to the total activity value outside the detection region. Layer selection processing, based on the evaluation value calculated through the evaluation value calculation process, selects at least a subset of layers from the plurality of layers suitable for visualizing the parts used as the judgment basis of the object detection model, wherein the selected layers are selected from the base number of layers whose evaluation values ​​are higher than the evaluation threshold; and The compositing process synthesizes the heatmaps related to the layers selected in the layer selection process to generate a composite image representing the parts that serve as the judgment criteria for the object detection model. In this process, each pixel of the image data is taken as an object pixel, and the highest activity among the activity of the object pixels represented by the heatmaps related to the selected partial layers is set as the activity of the object pixel in the composite image, thereby generating the composite image.

Images