Object detection device and object detection method

Abstract

To provide a technology capable of reducing non-detection and erroneous detection of an object.SOLUTION: An exemplary object detection device includes a processing unit and a storage unit. The processing unit performs object detection processing for detecting an object from input images. The storage unit stores a threshold value used for the object detection processing. The processing unit extracts candidate regions of the object from the input images, calculates an overlapping ratio between the candidate regions, and determines the candidate regions to be left from the candidate regions that overlap with each other based on the overlapping ratio and the threshold value. The threshold value is set for each type of the object.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a technique for detecting an object from an image. [Background technology]

[0002] Conventionally, a technique is known in which a plurality of region candidates are extracted from an input image, and then some of the plurality of region candidates are removed to determine a recognition target region (see, for example, Patent Document 1). In Patent Document 1, among a plurality of bounding boxes having mutually similar object regions, only the one with the highest class probability is retained and determined as the recognition target region. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-61505 Summary of the Invention [Problem to be solved by the invention]

[0004] If excessive removal is performed on multiple bounding boxes that are candidate regions for objects extracted from an image, the object will not be detected, while if the removal of bounding boxes that should be removed is not performed, the object will be falsely detected.

[0005] In view of the above, an object of the present invention is to provide a technology that can reduce non-detection or erroneous detection of an object. [Means for solving the problem]

[0006] An exemplary object detection device of the present invention includes a processing unit and a storage unit. The processing unit performs an object detection process to detect an object from an input image. The storage unit stores a threshold value used in the object detection process. The processing unit extracts candidate regions for the object from the input image, calculates an overlap ratio between the candidate regions, and determines which candidate regions to keep from among the candidate regions that overlap with each other based on the overlap ratio and the threshold value. The threshold value is set for each type of object. [Effects of the Invention]

[0007] According to the exemplary embodiment of the present invention, it is possible to reduce non-detection and erroneous detection of objects, and improve object detection performance. [Brief explanation of the drawings]

[0008] [Figure 1] Block diagram showing the hardware configuration of an object detection device [Figure 2] A block diagram showing the functional configuration of a processing unit included in an object detection device. [Figure 3] Schematic diagram to explain IoU [Figure 4] FIG. 10 is a diagram illustrating an example of a threshold table. [Figure 5] 1 is a flowchart illustrating an example of an object detection process executed by an object detection device. [Figure 6] FIG. 10 is a schematic diagram showing an example of a result of inference processing by an inference unit; [Figure 7] Schematic diagram showing an example of the results of NMS processing by the NMS unit [Figure 8] FIG. 10 is a diagram showing an example of a threshold value table according to a modified example; DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0010] <1. Object detection device> (1-1. Hardware configuration) Fig. 1 is a block diagram showing the hardware configuration of an object detection device 1 according to an embodiment of the present invention. For the purpose of making the explanation easier to understand, Fig. 1 also shows a camera 2, which is a device separate from the object detection device 1.

[0011] As shown in FIG. 1, the object detection device 1 includes a processing unit 11 and a storage unit 12. The object detection device 1 may be, for example, a device mounted on a moving object. The moving object may include, for example, a vehicle, a ship, an aircraft, or a robot. The object detection device 1 may also be built into, for example, a device worn by a person. The object detection device 1 may also be included in, for example, a server device that provides services to other devices, such as in-vehicle devices, via a network. The server device here may be a cloud server. The object detection device 1 may also be composed of, for example, a device mounted on a moving object and a server device. The object detection device 1 may also be a device installed on real estate, or may be configured to be included in a surveillance system installed in a large facility, for example.

[0012] The processing unit 11 includes a processor that performs arithmetic processing and the like. The processor may be configured to include, for example, a CPU (Central Processing Unit). The processor may also be configured to include, for example, a CPU and a GPU (Graphics Processing Unit). The processing unit 11 may be configured with one processor or multiple processors. When configured with multiple processors, the processors only need to be connected to each other so that they can communicate with each other.

[0013] An image (more specifically, image data) is input to the processing unit 11. For example, an image is input to the processing unit 11 from outside the object detection device 1. In the example shown in FIG. 1, the image is input directly from a camera 2 that is communicatively connected to the object detection device 1. However, the image may also be input to the processing unit 11 via a portable storage medium such as an SD card, a USB memory, or an optical disk. The image may also be input to the processing unit 11 via a network such as the Internet. The processing unit 11 performs object detection processing to detect an object from the input image (input image). Details of the object detection processing will be described later.

[0014] The storage unit 12 non-temporarily stores or memorizes computer-readable programs, data, etc. The storage unit 12 has a non-volatile storage medium. The non-volatile storage medium of the storage unit 12 may be composed of at least one of a semiconductor memory, a magnetic medium, an optical medium, etc.

[0015] In this embodiment, the storage unit 12 stores a threshold value table 121. The threshold value table 121 includes threshold values ​​used in the object detection process. That is, the storage unit 12 stores the threshold values ​​used in the object detection process. The details of the threshold value table 121 will be described later.

[0016] (1-2. Processing section) 2 is a block diagram showing the functional configuration of processing unit 11 included in object detection device 1 according to the embodiment of the present invention. The functions of processing unit 11 are realized by a processing device such as a CPU executing arithmetic processing in accordance with a program stored in storage unit 12. As shown in FIG. 2, processing unit 11 includes, as its functions, an acquisition unit 111, an inference unit 112, an NMS (Non Maximum Suppression) unit 113, and an output unit 114.

[0017] At least one of the acquisition unit 111, the inference unit 112, the NMS unit 113, and the output unit 114 included in the processing unit 11 may be configured with hardware such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPU. The acquisition unit 111, the inference unit 112, the NMS unit 113, and the output unit 114 are conceptual components. The function performed by one component may be distributed among multiple components, or the functions of multiple components may be integrated into one component.

[0018] The acquisition unit 111 acquires an input image. For example, the input image may be an image captured by a camera 2 capturing an image of the periphery of a vehicle. As will be described later, the object detection device 1 of this embodiment can improve object detection performance. As a result, it is possible to appropriately detect objects from camera images captured by an on-board camera, and to appropriately control the vehicle using the camera images.

[0019] The inference unit 112 performs inference processing. More specifically, the inference unit 112 performs object detection using a trained model based on machine learning. The inference unit 112 performs processing to identify the position (area) and type of a predetermined object from an input image using a known object detection algorithm. The type of object to be detected is determined appropriately depending on the intended use of the object detection device 1. For example, if the object detection device 1 is installed in an automobile, such as when used to control the automatic driving of the automobile, the detection target of the object detection device 1 will be an object visible from the automobile. More specifically, the detection targets include automobiles, motorcycles, people, bicycles, traffic lights, etc.

[0020] Examples of well-known object detection algorithms include Fast R-CNN, Fast R-CNN, Faster R-CNN, YOLO, and SSD. In this embodiment, the inference unit 112 uses YOLO as the object detection algorithm. The results obtained by the inference process by the inference unit 112 include the type of object, a bounding box indicating the area of ​​the object, and a confidence score indicating the probability of the object's existence. The confidence score is, for example, a numerical value between 0 and 1, and the closer the value is to "1," the higher the confidence.

[0021] Note that the bounding box obtained by the inference unit 112 is a result obtained at a stage prior to the final output of the object detection result, and is not a definitive result. For this reason, in this specification, the bounding box obtained by the inference unit 112 is referred to as a candidate area. As can be seen from the above, the processing unit 11 extracts candidate areas of an object from the input image. The object detection method for detecting an object from an input image includes an extraction step of extracting candidate areas of the object from the input image. There is not necessarily only one candidate area for each object, and multiple candidate areas may be extracted.

[0022] The NMS unit 113 performs NMS processing, which is processing for removing some of the overlapping candidate areas. More specifically, overlapping candidate areas refer to multiple candidate areas that partially overlap each other. By the NMS processing, some of the multiple candidate areas that overlap each other are removed. In the NMS processing, an Intersection over Union (IoU), which is the overlap ratio between the candidate areas, is calculated. That is, the processing unit 11 calculates the overlap ratio between the candidate areas. The object detection method includes a calculation step for calculating the overlap ratio between the candidate areas.

[0023] IoU is calculated using the following formula (1). IoU = X / Y (1) X: The area where the two candidate regions overlap Y: The total area of ​​the two overlapping candidate regions IoU is an index that indicates the degree of overlap between candidate regions, and the closer the value is to "1", the greater the overlap.

[0024] Fig. 3 is a schematic diagram for explaining IoU. In Fig. 3, it is assumed that an automobile 100, which is the detection target, is included in an image, and a first candidate region 200 and a second candidate region 300 have been extracted for the automobile 100. In this example, the first candidate region 200 and the second candidate region 300 are both rectangular regions. Note that the shape of the candidate regions is not limited to a rectangle and may be other shapes.

[0025] 3, X in the above formula (1) is the area of ​​the overlapping portion (hatched portion) of the first candidate region 200 and the second candidate region 300. Y in formula (1) is obtained by adding the area of ​​the first candidate region 200 and the area of ​​the second candidate region 300 together and subtracting the area of ​​the overlapping portion of the first candidate region 200 and the second candidate region 300 (the area calculated as X above) from the sum.

[0026] More specifically, the NMS unit 113 determines which candidate areas to keep from among the overlapping candidate areas based on the calculated IoU value and a threshold value included in the threshold value table 121 (see FIG. 1). That is, the processing unit 11 determines which candidate areas to keep from among the overlapping candidate areas based on the overlap rate between the candidate areas and a threshold value stored in the storage unit 12. The object detection method includes a determination step of determining which candidate areas to keep from among the overlapping candidate areas based on the overlap rate and a preset threshold value. The candidate areas determined to be kept are areas that are ultimately output as object areas. The areas that are ultimately output as object areas are areas that are displayed as bounding boxes, etc.

[0027] In detail, if the calculated IoU value exceeds a threshold included in the threshold table 121, part of the candidate area is removed. The candidate area that is not removed is the candidate area to be retained as described above. That is, the processing unit 11 removes part of the candidate area if the overlap rate between the candidate areas exceeds a threshold stored in the storage unit 12. With this configuration, it is possible to remove part of multiple candidate areas that have a large overlap rate and are likely to be candidate areas for the same object. That is, unnecessary candidate areas can be appropriately removed, allowing the object to be appropriately detected.

[0028] More specifically, among the overlapping candidate regions, the candidate region with the highest reliability score is selected as the candidate region to be retained, and the rest are removed. That is, the processing unit 11 removes all but the candidate region with the highest reliability score from among the overlapping candidate regions. With this configuration, it is possible to retain the most reliable region from among the regions estimated to be candidate regions for the same object, thereby improving the reliability of object detection.

[0029] In this embodiment, the process of removing a portion of overlapping candidate areas is performed on candidate areas estimated to be the same type of object. Iou is calculated for candidate areas that are estimated to be the same type of object among candidate areas that are determined to overlap with each other. IoU is not calculated for candidate areas that are estimated to be different types of object even if they overlap with each other. That is, in this embodiment, the processing unit 11 calculates the overlap ratio for candidate areas of the same type of object. This configuration makes it possible to remove unnecessary candidate areas while simplifying the NMS process as much as possible.

[0030] In the example shown in FIG. 3, it is assumed that the object type of both the first candidate region 200 and the second candidate region 300 is estimated to be an automobile. In other words, the first candidate region 200 and the second candidate region 300 are the targets for which IoU calculation is performed. It is also assumed that the IoU values ​​of the first candidate region 200 and the second candidate region 300 exceed a predetermined threshold. It is also assumed that the reliability score of the first candidate region 200 is 0.9, and the reliability score of the second candidate region 300 is 0.5. In this case, of the extracted first candidate region 200 and second candidate region 300, the first candidate region 200 with the highest reliability score is retained, and the second candidate region 300 is removed.

[0031] Figure 3 shows an example in which two candidate regions of the same type overlap, but this is merely an example. There may also be candidate regions in which no overlapping candidate regions exist. A candidate region that does not have any overlapping candidate regions is determined to be the candidate region to be retained as described above, and is not removed.

[0032] It is also possible that three or more candidate regions of the same type overlap. In this case, two candidate regions are selected from all overlapping candidate regions, and it is determined whether their IoU values ​​exceed a threshold. If they exceed the threshold, one candidate region is removed according to its reliability score. If they do not exceed the threshold, both candidate regions are retained. Next, two candidate regions are selected from the remaining candidate regions in a combination different from the previously selected combination, and the IoU values ​​are compared with the threshold and the candidate regions are removed as necessary. Subsequently, the remaining candidate regions are subjected to the same processing as above. The above processing is repeated for candidate regions with IoU values ​​exceeding the threshold until only one candidate region remains. This narrows down the number of candidate regions targeting the same object to one. Furthermore, the remaining candidate regions can be retained as regions targeting different objects.

[0033] The output unit 114 performs processing to output the results obtained by the above-mentioned inference processing and NMS processing. The output unit 114 outputs the final results of the position and type of the detected object to, for example, a display device or a control device that controls the vehicle.

[0034] (1-3. Threshold) Next, the thresholds stored in the storage unit 12 will be described in detail. In this embodiment, the thresholds are stored in the form of a threshold table 121. FIG. 4 is a diagram showing an example of the threshold table 121. In the example shown in FIG. 4, the object detection device 1 detects cars (automobiles), people, bicycles, and traffic lights from camera images captured by a camera 2 that captures the surroundings of the host vehicle. As shown in FIG. 4, thresholds are set for each type of object. In the example shown in FIG. 4, separate thresholds are set for cars, people, bicycles, and traffic lights.

[0035] It is conceivable to use the same threshold value for all objects regardless of their type. However, such a configuration would make it difficult to set an appropriate threshold value for all objects due to differences in size, shape, and other factors depending on the object type. As a result, the NMS process may remove too many candidate regions, resulting in undetected objects. Alternatively, the NMS process may not remove candidate regions that should have been removed, resulting in false detection of objects. In this regard, if the threshold value is set for each object type, as in this embodiment, the threshold value can be appropriately determined according to the object type, reducing the possibility of undetection or false detection of objects. In other words, object detection performance can be improved.

[0036] In this embodiment, since candidate areas estimated to be the same type of object are targeted and some of the overlapping candidate areas are removed, the threshold is set for objects of the same type, but not for objects of different types. For example, a threshold is set for cars, but not for cars and people.

[0037] The threshold value is determined in advance, for example, by using the object detection device 1 or by using a device other than the object detection device 1, and is stored in the storage unit 12. For example, the configuration may be such that the processing unit 11 of the object detection device 1 determines the threshold value and stores it in the storage unit 12. Alternatively, for example, the threshold value may be determined by another device and distributed from the other device to the object detection device 1. Instead of distribution, the threshold value determined by the other device may be stored in the storage unit 12 using a portable recording medium. Alternatively, for example, the threshold value may be determined by the other device and stored in the storage unit 12 by manual input.

[0038] For example, the threshold value may be determined by experimentally varying a number of values, performing an object detection process for each value, and calculating an evaluation index for the results of each object detection process. The object detection process is a process that includes the above-described inference process and NMS process. The evaluation index for the object detection process may be mAP (mean average precision). For example, the threshold value may be determined so that the mAP is maximized. By using this method, an appropriate threshold value can be obtained for each type of object, thereby improving object detection performance.

[0039] The threshold may be determined by, for example, a known technique such as hyperparameter tuning. The threshold may be determined by, for example, Bayesian optimization, which is a type of hyperparameter tuning. In this embodiment, the threshold is a value ranging from 0 to 1. In Bayesian optimization, a plurality of values ​​(for example, 10 values) are randomly selected from the range of 0 to 1, and an object detection process is performed on these values ​​to calculate an evaluation index (mAP). Depending on the result of the evaluation index, a process of determining the next value to be examined is repeated until an optimal threshold is determined.

[0040] <2. Object detection processing> FIG. 5 is a flowchart showing an example of the object detection process executed by the object detection device 1 according to the embodiment of the present invention.

[0041] In step S1, the processing unit 11 initializes the object detection model. The object detection model is the trained model described above. For example, the processing unit 11 reads out weight parameters stored in the storage unit 12. Once the initialization of the object detection model is complete, the process proceeds to the next step S2.

[0042] In step S2, the processing unit 11 reads out the threshold value table 121 stored in the storage unit 12. When the reading out of the threshold value table 121 is completed, the process proceeds to the next step S3.

[0043] In step S3, the processing unit 11 starts an inference processing loop, which includes a process in step S31, a process in step S32, a process in step S33, a process in step S34, and a process in step S35.

[0044] In step S31, the acquisition unit 111 acquires an input image input from the camera 2. In detail, the acquisition unit 111 acquires the input image at a predetermined cycle. The predetermined cycle is determined appropriately depending on the purpose of object detection, etc. Once the input image is acquired, the process proceeds to the next step S32.

[0045] In step S32, the inference unit 112 performs inference processing using an object detection model on the input image. Through the inference processing, candidate regions of the object being detected from the input image, a confidence score, and the type of the object are obtained. FIG. 6 is a schematic diagram showing an example of the results of the inference processing by the inference unit 112. In the example shown in FIG. 6, cars and traffic lights are included in the detection target objects, and cars and traffic lights are detected as object types. In detail, two candidate regions 400a and 400b classified as cars are extracted. Furthermore, two candidate regions 500a and 500b classified as traffic lights are extracted. When the inference processing by the inference unit 112 is completed, the process proceeds to the next step S33.

[0046] Hereinafter, to make it easier to distinguish between the names, one of the two candidate areas 400a, 400b classified as cars will be referred to as the first vehicle candidate area 400a, and the other as the second vehicle candidate area 400b. Also, one of the two candidate areas 500a, 500b classified as traffic lights will be referred to as the first traffic light candidate area 500a, and the other as the second traffic light candidate area 500b.

[0047] In step S33, the NMS unit 113 performs the above-described NMS processing on the inference result obtained in step S32. Here, it is assumed that the inference result shown in Fig. 6 above is obtained, and the threshold value shown in Fig. 4 is used in the NMS processing.

[0048] The NMS unit 113 calculates the IoU for the first and second vehicle candidate areas 400a and 400b, which are classified as vehicles, and determines whether the IoU calculation result exceeds 0.61, which is the threshold value for vehicles obtained from the threshold table 121. If the IoU value exceeds 0.61, the NMS unit 113 determines that the two vehicle candidate areas 400a and 400b represent the same vehicle 3. In this case, the NMS unit 113 removes the one with the smaller reliability score from the first vehicle candidate area 400a or the second vehicle candidate area 400b. If the IoU value is 0.61 or less, the NMS unit 113 determines that the first vehicle candidate area 400a and the second vehicle candidate area 400b represent different vehicle areas, and decides to retain the two vehicle candidate areas 400a and 400b.

[0049] The NMS unit 113 also calculates the IoU for the first traffic light candidate area 500a and the second traffic light candidate area 500b, which are classified as traffic lights, and determines whether the calculated IoU exceeds 0.62, which is the threshold value for traffic lights obtained from the threshold table 121. If the IoU value exceeds 0.62, the NMS unit 113 determines that the two traffic light candidate areas 500a and 500b represent the same traffic light 4. In this case, the NMS unit 113 removes the one with the smaller reliability score from the first traffic light candidate area 500a or the second traffic light candidate area 500b. If the IoU value is 0.62 or less, the NMS unit 113 determines that the first traffic light candidate area 500a and the second traffic light candidate area 500b represent areas of different traffic lights, and decides to retain the two traffic light candidate areas 500a and 500b.

[0050] Fig. 7 is a schematic diagram showing an example of the results of NMS processing by the NMS unit 113. Fig. 7 is based on the assumption that the inference results shown in Fig. 6 are obtained. The IoU value calculated for the first vehicle candidate area 400a and the second vehicle candidate area 400b exceeds the threshold value between vehicles, 0.61. The IoU value calculated for the first traffic light candidate area 500a and the second traffic light candidate area 500b exceeds the threshold value between traffic lights, 0.62.

[0051] 7, as a result of the NMS processing, of the two vehicle candidate areas 400a, 400b estimated to represent the area of ​​the same vehicle 3, the first vehicle candidate area 400a, which has the highest reliability score, remains. Also, as a result of the NMS processing, of the two traffic light candidate areas 500a, 500b estimated to represent the area of ​​the same traffic light 4, the first traffic light candidate area 500a, which has the highest reliability score, remains. When the NMS processing is complete, the process proceeds to the next step S34.

[0052] In step S34, the output unit 114 outputs the final result of the object detection process performed on the input image. In the examples shown in Figures 6 and 7, the final result of the object detection process is a result in which one bounding box is attached to each of the car 3 and the traffic light 4. When the output process is completed, the process proceeds to the next step, S35.

[0053] In step S35, the processing unit 11 checks whether processing has been performed on the final frame. That is, the processing unit 11 checks whether processing has been completed on all input images. If processing has been completed on all input images (Yes in step S35), the object detection processing shown in Fig. 5 ends. On the other hand, if processing has not been completed on all input images (No in step S35), the inference processing loop including the processing of steps S31 to S35 is repeated.

[0054] <3. Modifications> In the embodiment described above, the process of removing a portion of overlapping candidate areas is configured to target candidate areas estimated to be the same type of object. However, this is an example, and the process of removing a portion of overlapping candidate areas may also be performed on candidate areas estimated to be different types of objects. In other words, Iou may be calculated for candidate areas that are determined to overlap with each other and are estimated to be different types of objects.

[0055] In other words, the processing unit 11 may calculate the overlap ratio for candidate regions of different types of objects. With this configuration, for example, in a situation where multiple types of objects overlap, it is possible to appropriately remove objects with low reliability. As a result, it is possible to reduce the possibility of false object detection. Furthermore, it is possible to make it easier for a person viewing the detection results on a screen to understand the detected objects.

[0056] FIG. 8 is a diagram showing an example of a threshold value table 121A of a modified example. In the example shown in FIG. 8, it is assumed that the object detection device detects cars, people, bicycles, trucks, and traffic lights. Cars are smaller automobiles than trucks and are considered to be a different type of object from trucks. In the example shown in FIG. 8, as in the above-described embodiment, threshold values ​​are set for each type of object. In the example shown in FIG. 8, separate threshold values ​​are set for cars, people, bicycles, trucks, and traffic lights.

[0057] However, in this modification, thresholds are set not only between objects of the same type, such as between cars or between people, but also between objects of different types: between cars and people, between cars and bicycles, between cars and trucks, between cars and traffic lights, between people and bicycles, between people and trucks, between people and traffic lights, between bicycles and trucks, between bicycles and traffic lights, and between trucks and traffic lights.

[0058] Note that, similarly to the above-described embodiment, each threshold value may be obtained by using a hyperparameter tuning method such as Bayesian optimization. The threshold value table 121A has a configuration in which various threshold values ​​are organized into a table, and is stored in advance in the storage unit 12.

[0059] In this modification, for example, if a candidate region for a car and a candidate region for a person are extracted from the input image so as to overlap, the IoU between these two candidate regions is calculated, and if the calculated IoU value exceeds a threshold (0.54 in the example of FIG. 8), the region with the smaller reliability score is eliminated. Also, for example, if a candidate region for a person and a candidate region for a bicycle are extracted from the input image so as to overlap, the IoU between these two candidate regions is calculated, and if the calculated IoU value exceeds a threshold (0.66 in the example of FIG. 8), the region with the smaller reliability score is eliminated.

[0060] <4. Things to keep in mind> In addition to the above embodiments, the various technical features disclosed in this specification can be modified in various ways without departing from the spirit of the technical creation. In other words, the above embodiments should be considered to be illustrative in all respects and not limiting. The technical scope of the present invention is defined by the claims, not by the description of the above embodiments, and should be understood to include all modifications that fall within the meaning and scope of the claims. Furthermore, the multiple embodiments and modifications shown in this specification may be combined as appropriate to the extent possible.

[0061] The scope of this embodiment includes a computer program that causes a computer to implement the above-described object detection device 1, and a computer-readable non-volatile recording medium on which the program is recorded. Also, the scope of this embodiment includes a computer-readable non-volatile recording medium on which a computer program that causes a computer to implement the above-described object detection method is recorded. [Explanation of symbols]

[0062] 1. Object detection device 2. Camera 11 Processing section 12...Storage section 200, 300, 400a, 400b, 500a, 500b... Candidate regions

Claims

1. a processing unit that performs object detection processing to detect an object from an input image; a storage unit that stores threshold values ​​used in the object detection process and set for each type of object; Equipped with The processing unit extracting a candidate region of the object from the input image; Detecting a type of object in the candidate region from the input image; calculating an overlap rate between the candidate regions, including cases where the candidate regions are of different types of objects; The object detection device determines which candidate regions to leave out of the overlapping candidate regions based on the overlap ratio and the threshold.

2. The object detection device according to claim 1 , wherein the processing unit removes a part of the candidate region when the overlap rate exceeds the threshold value.

3. The processing unit Detecting a confidence level in the candidate region from the input image; The object detection device according to claim 2 , wherein the candidate regions that overlap each other are removed except for the candidate region with the highest reliability.

4. The object detection device according to claim 1 , wherein the processing unit calculates the overlap ratio for the candidate regions of the same type of object.

5. An object detection device as described in any one of claims 1 to 4, wherein the threshold is set not only between objects of the same type but also between objects of different types.

6. The object detection device according to claim 1 , wherein the input image is a photographed image taken by a camera that photographs the periphery of a vehicle.

7. An object detection method for detecting an object from an input image, an inference step of extracting a candidate region of the object from the input image and detecting the type of the object in the candidate region; a calculation step of calculating an overlap ratio between the candidate regions, including when the candidate regions are of different types of objects; a determining step of determining, based on the overlapping ratio and a threshold value previously set for each type of object, the candidate areas to be retained from among the candidate areas that overlap with each other; An object detection method comprising:

8. The object detection method according to claim 7 , wherein the threshold is set not only for objects of the same type but also for objects of different types.

Images