OBJECT DETECTION DEVICE, OBJECT DETECTION METHOD, AND OBJECT DETECTION PROGRAM

Abstract

To improve accuracy in detecting a detection object.SOLUTION: An object detection device relating to an embodiment is an object detection device which detects detection objects from an image captured by a camera. A control unit divides the image into a plurality of areas on the basis of distribution information generated by using a plurality of images from which a distribution state of appearance positions within the image is preliminarily acquired for each of sizes of the detection objects, and sets thresholds for inference results of a learning model trained on feature quantities of the detection objects, to each of the plurality of divided areas.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an object detection device, an object detection method, and an object detection program. [Background technology]

[0002] Conventionally, there is a technology for detecting traffic lights from a captured image of the area ahead of a vehicle. For example, in such a technology, the installation positions of traffic lights registered in advance in map information are set as a detection target area, and then the traffic lights are detected (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-27328 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for improvement, for example, in terms of improving the detection accuracy of the detection target.

[0005] The present invention has been made in view of the above, and aims to provide an object detection device, an object detection method, and an object detection program that can improve the detection accuracy of an object to be detected. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objective, the object detection device of the present invention is an object detection device that detects a detection target from an image captured by a camera, and a control unit divides areas in the image based on distribution information generated using multiple images previously acquired to show the distribution of appearance positions within the image for each size of the detection target, and sets a threshold for each divided area for the inference results of a learning model that has learned the features of the detection target. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the detection accuracy of the detection target. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of an object detection device being installed. [Figure 2] FIG. 2 is a diagram showing an outline of the object detection method. [Figure 3] FIG. 3 is a diagram showing an outline of the object detection method. [Figure 4] FIG. 4 is a block diagram of the object detection device. [Figure 5] FIG. 5 is a diagram illustrating an example of the threshold information storage unit. [Figure 6] FIG. 6 is a flowchart showing a processing procedure executed by the object detection device. [Figure 7] FIG. 7 is a flowchart showing a processing procedure executed by the object detection device. [Figure 8] FIG. 8 is a diagram showing an application example of the object detection method. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments for carrying out an object detection device, an object detection method, and an object detection program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the object detection device, the object detection method, and the object detection program according to the present application are not limited to these embodiments.

[0010] First, an overview of an object detection device, an object detection method, and an object detection program according to an embodiment will be described with reference to Figures 1 to 3. Figure 1 is a diagram showing an example of an installation of an object detection device. Figures 2 and 3 are diagrams showing an overview of an object detection method.

[0011] As shown in Fig. 1, an object detection device 50 according to the embodiment is mounted on, for example, a vehicle 100, and detects a detection target from an image input from a camera installed on the vehicle 100. In the example shown in Fig. 1, the object detection device 50 is a drive recorder that captures an image of the area in front of the vehicle 100. Note that the object detection device 50 may be housed in a housing separate from the drive recorder, and may be configured to detect a detection target from an image of the area around the vehicle 100, including the area behind and to the sides of the vehicle 100 in addition to the area in front of the vehicle 100.

[0012] For example, the object detection device 50 uses deep learning technology to detect a traffic light, which is a detection target, from an image captured in front of the vehicle 100. More specifically, the object detection device 50 uses a learning model that has learned the characteristics of the detection target, and detects, as the detection target, an object for which the confidence level, which is the inference result of the learning model, exceeds a preset threshold.

[0013] In such an object detection method using machine learning, the larger the detection target object shown in the image, the more accurate the detection, and the smaller the detection target object shown in the image, the lower the detection accuracy tends to be. In other words, the farther the detection target object is from the vehicle 100, the smaller the detection target object shown in the image, and the lower the detection accuracy tends to be.

[0014] More specifically, as shown in Figure 2, the larger the traffic light in the image, the higher the confidence level tends to be, making it easier to detect the traffic light in the image. However, if the traffic light in the image is small, for example, the confidence level may be low and the traffic light may not be detected.

[0015] On the other hand, depending on the application specifications, it may be necessary to detect distant objects, i.e., small objects that appear in an image. In such cases, if the threshold for the confidence level is a uniform value, there is a risk that a nearby object may be mistakenly detected as a traffic light when attempting to detect a distant traffic light.

[0016] Therefore, in the object detection method according to the embodiment, an image is divided into a plurality of regions, and a threshold is set for each divided region.

[0017] Specifically, as shown in FIG. 3, in the object detection method according to the embodiment, an image is divided into multiple regions based on distribution information generated using multiple images previously acquired to show the distribution of appearance positions within the image for each size of the object to be detected.

[0018] Fig. 3 shows a case where the distribution information is information that maps the relationship between the size of the detected object and its appearance position. In the distribution information shown in Fig. 3, the size of one sphere corresponds to the size of the detected object in the image, and the sphere that overlaps the detected coordinates is displayed as a bubble-like mapping.

[0019] For example, by using such distribution information, it is possible to see that there is a correlation between the size of the detected object and its appearance position, and that the larger the size, the more likely it is to appear towards the top of the image, and the smaller the size, the more likely it is to appear towards the center of the image.

[0020] For example, in the object detection method, an image is divided into multiple regions according to the appearance size of the detection target based on distribution information, and a threshold value for the confidence level is set for each divided region. In the example shown in the figure, the image is divided by boundaries B1 to B3.

[0021] Area A1, located above boundary B1, is an area where relatively large detection objects appear, and area A2, sandwiched between boundary B1 and boundary B2, is an area where medium-sized detection objects appear.

[0022] In addition, the area A3 between the boundary B2 and the boundary B3 is an area where relatively small detection objects appear, and the area below the boundary B3 is an area where almost no detection objects are detected.

[0023] For example, when the vehicle 100 passes a traffic light, the traffic light gradually becomes larger and then disappears as the vehicle 100 travels. In such driving data, the collected traffic light image can be divided into regions according to the size of the detected object by making the boundaries of each region (see, for example, B1 and B2) radial.

[0024] In other words, by dividing the area radially, it is possible to make the appearance size of the detection target included in each area appropriately uniform, which allows a threshold value to be set for each area according to the appearance size, as will be described later, thereby improving the detection accuracy of the detection target in each area.

[0025] For example, in the object detection method, a high threshold for the confidence level is set in region A1 where the detected object in the image is large, and the threshold is gradually lowered in order of region A2 and region A3. As a result, in region A1, only objects with a high confidence level can be detected as detected objects, and in regions A2 and A3, only objects with a confidence level exceeding the threshold can be detected as detected objects.

[0026] In other words, in the object detection method, by dividing an image into multiple regions and setting a threshold for the confidence level for each divided region, it is possible to detect traffic lights located close to the vehicle 100 while suppressing false detections in which objects other than traffic lights located close to the vehicle 100 are detected as the detection target.

[0027] Therefore, the object detection method according to the embodiment can improve the detection accuracy of the detection target.

[0028] Next, an example of the configuration of an object detection device 50 according to an embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram of the object detection device 50. Note that Fig. 4 also shows a camera 10.

[0029] The camera 10 is an imaging device equipped with an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). For example, the camera 10 captures an image ahead of the vehicle 100 and inputs the captured image to the object detection device 50.

[0030] 4, the object detection device 50 includes a storage unit 20 and a control unit 30. The storage unit 20 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example of FIG. 4, the storage unit 20 includes a distribution information storage unit 21, a threshold information storage unit 22, and a learning model storage unit 23.

[0031] The distribution information storage unit 21 stores distribution information. The distribution information here is, for example, information indicating the distribution of appearance positions within an image for each size of detection target object, as shown in FIG. 3. For example, the distribution information storage unit 21 stores distribution information generated based on images captured by other vehicles 100. In other words, by sharing and using the distribution information among multiple object detection devices 50, initial setup in each object detection device 50 can be performed efficiently.

[0032] The threshold information storage unit 22 stores information about thresholds for the inference results of the learning model. Fig. 5 is a diagram showing an example of the threshold information storage unit 22. As shown in Fig. 5, the threshold information storage unit 22 stores information on items such as "area ID," "boundary," and "threshold" in association with each other.

[0033] The "area ID" is an identifier for identifying each area. The "boundary" indicates the boundary of each area, and is, for example, a function that indicates the area. For example, it is preferable that the boundary is a quadratic curve. This makes it easy to calculate, for example, which area an object in the image is included in.

[0034] In other words, if the boundary is expressed as a complex function, the area in which the object in the image is contained must be determined through complex calculations, which may increase the processing load.In contrast, by expressing the boundary as a quadratic function, it is possible to determine the area in which the object in the image is contained while reducing the processing load.

[0035] "Threshold" indicates the threshold for the confidence level in the corresponding region. In the example shown in the figure, the boundary of the region identified by region ID "A1" is ">B1" and the threshold is "th1".

[0036] Returning to the explanation of FIG. 4, the learning model storage unit 23 will be described. The learning model storage unit 23 stores a learning model. A learning model is a model that learns the feature amount of a detection object by deep learning. For example, a model that has been learned using an image of a traffic light as training data is stored in the learning model storage unit 23 as the learning model.

[0037] The control unit 30 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (not shown) stored in the storage unit 20 using RAM as a work area. The control unit 30 can also be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0038] 4, the control unit 30 includes a dividing unit 31, a setting unit 32, and a detection unit 33. The dividing unit 31 divides the image into a plurality of regions based on distribution information generated using a plurality of images previously acquired to show the distribution of appearance positions within the image for each size of the detection target object.

[0039] For example, during initial setup of the object detection device 50, the division unit 31 divides the image into a plurality of regions. For example, the division unit 31 displays distribution information read from the distribution information storage unit 21 on a display unit (not shown), and divides the image into a plurality of regions based on an operation of the initial setup user on the distribution information.

[0040] In this case, it is preferable that the dividing unit 31 provides distribution information in which the size of each sphere is expressed in a display size according to the appearance size of the detection object, as shown in Fig. 3. This allows the initial setting user to easily grasp the correlation between the appearance size and the appearance position in the image.

[0041] Thereafter, the dividing unit 31 divides the image into a plurality of regions in accordance with the operation of the initial setting user for the presented distribution information. For example, the dividing unit 31 approximates the boundaries set by the initial setting user to quadratic curves or straight lines, and writes the approximated boundaries into the threshold information storage unit 22.

[0042] The dividing unit 31 may, for example, automatically divide the image into a plurality of regions using a predetermined algorithm based on the distribution information. Also, for example, the dividing unit 31 may display candidate boundaries for dividing each region based on the distribution information, and provide a UI that allows the initial setting user to select a boundary from the candidate boundaries.

[0043] The setting unit 32 sets a threshold value for the inference result of the learning model for each of the multiple regions divided by the dividing unit 31. For example, the setting unit 32 sets the threshold value set by the initial setting person for each of the regions set by the initial setting person.

[0044] Alternatively, the setting unit 32 may set a threshold for each region according to the appearance size of the detection target object in each region divided by the dividing unit 31. For example, the setting unit 32 may calculate an average value or the like of the appearance size in each region, and then set a threshold according to the average value or the like.

[0045] The threshold information set by the dividing unit 31 and the setting unit 32 may be information executed outside the object detection device 50. In other words, the configuration of the dividing unit 31 and the setting unit 32 may be regarded as the object detection device 50. In this case, for example, threshold information that has been set in advance on a server or the like owned by the manufacturer may be installed in the object detection device 50 mounted on each vehicle 100.

[0046] The detection unit 33 detects a detection target from an image captured by the camera 10 according to the result of comparison between the inference result for the image and a threshold value. Specifically, first, each time an image is acquired from the camera 10, the detection unit 33 uses the learning model stored in the learning model storage unit 23 to obtain information on the confidence level, which is the inference result of the learning model, and the detected position of the target.

[0047] Next, the detection unit 33 refers to the threshold information storage unit 22 and calculates which region the detected position of the object is included in. The detection unit 33 calculates, for example, the region that includes the center of gravity of the object or the coordinates of a predetermined position of the object (for example, the upper left corner) as the region that includes the object.

[0048] Next, the detection unit 33 compares the threshold of the area including the object with the certainty of the object, and if the certainty is greater than the threshold, detects the object as a detection object. That is, if the area including the object is area A1, the detection unit 33 compares the certainty with a threshold th1 (see FIG. 5) to detect the detection object, while if the area including the object is area A2, the detection unit 33 compares the certainty with a threshold th2 to detect the detection object.

[0049] That is, the detection unit 33 selects a threshold value according to the area in which the object is captured and compares it with the confidence level, thereby improving the accuracy of detecting the object. The detection unit 33 also notifies a predetermined application of information about the object detected as the detection object.

[0050] Next, a processing procedure executed by the object detection device 50 according to the embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 and Fig. 7 are flowcharts showing the processing procedure executed by the object detection device 50.

[0051] First, the processing procedure up to setting the threshold will be described with reference to Fig. 6. As shown in Fig. 6, first, the object detection device 50 generates the distribution information shown in Fig. 3 (step S101).

[0052] Next, the object detection device 50 divides the image into a plurality of regions based on the distribution information (step S102). Next, the object detection device 50 sets a threshold for each region (step S103). Then, the object detection device 50 ends the process.

[0053] Next, a processing procedure for object detection processing will be described with reference to Fig. 7. As shown in Fig. 7, the object detection device 50 acquires an image from the camera 10 (step S111). Next, the object detection device 50 inputs the acquired image into the learned learning model and acquires an inference result (step S112).

[0054] Next, the object detection device 50 selects a threshold based on which region the target object is included in (step S113), and compares the confidence level with the threshold (step S114). Next, the object detection device 50 determines whether the confidence level is greater than the threshold (step S115).

[0055] If the certainty factor is greater than the threshold (step S115: Yes), the object detection device 50 detects the object as a detection target (step S116) and ends the process. On the other hand, if the certainty factor is less than the threshold (step S115; No), the object detection device 50 skips the process of step S116, that is, does not detect the object as a detection target and ends the process.

[0056] As described above, the object detection device 50 according to the embodiment is an object detection device that detects a detection target from an image captured by the camera 10, and the control unit 30 divides the image into multiple regions based on distribution information generated using multiple images previously acquired to show the distribution of appearance positions within the image for each size of the detection target, and sets a threshold for each of the multiple divided regions for the inference results of a learning model that has learned the feature quantities of the detection target. Therefore, the object detection device 50 according to the embodiment can improve the detection accuracy of the detection target.

[0057] In the above embodiment, the detection object is a traffic light, but the present invention is not limited to this. For example, the detection object may be another vehicle.

[0058] Fig. 8 is a diagram showing an application example of the object detection method. As shown in Fig. 8, for example, when the detection target is another vehicle, distribution information indicates a distribution state that spreads radially from the center of the image.

[0059] Therefore, the example in Fig. 8 illustrates a case where an image is radially divided by boundaries B1 and B2 according to the distribution of sphere sizes. For example, in the example shown in Fig. 8, the area above boundary B1 is divided into area A1, area A2 between boundaries B1 and B2, and area below boundary B2 into area A3, and different thresholds are set for each of areas A1 to A3, and other vehicles, which are detection targets, are detected from the image as shown in the lower part of the figure.

[0060] Even in such cases, other vehicles can be detected based on the comparison results between the threshold set for each area and the confidence level, which is the inference result of the learning model, thereby reducing false detections and improving detection accuracy.

[0061] In the above-described embodiment, for example, a case where one threshold value is set for each region has been described, but the present invention is not limited to this. For example, when the detection results of the object detection device 50 are output to multiple applications, multiple threshold values ​​may be set for each region, and a threshold value may be selected depending on the application to which the results are output.

[0062] That is, multiple thresholds may be set for each region depending on the detection purpose. In this case, the region may be divided into different regions depending on the application of the output destination. This allows the object detection device 50 to obtain detection results suited to the application of the output destination, thereby contributing to improved convenience.

[0063] In the above-described embodiment, the detection object is described as a traffic light or another vehicle, but the present invention is not limited to this. That is, the detection object may be a white line on the road, or any other object. In the above-described embodiment, the camera 10 is described as a camera fixed to the vehicle 100, but the present invention is not limited to this. That is, the camera 10 may be, for example, a security camera or the like, installed on the road or in a store.

[0064] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0065] 10 Camera 21 Distribution information storage unit 22 Threshold information storage unit 23 Learning model memory unit 30 Control Unit 31 Split part 32 Setting section 33 Detection unit 50 Object detection device 100 vehicles A1~A3 area B1~B3 boundary

Claims

1. An object detection device that detects a detection target from an image captured by a camera, the object detection device having a control unit, The control unit Dividing the image radially into a plurality of regions based on distribution information generated using a plurality of images previously acquired to show a distribution state of appearance positions of the detection target objects in the image for each size of the detection target objects; a threshold value for an inference result of a learning model that has learned the feature amount of the detection object is set for each of the divided regions; Object detection device.

2. An object detection device that detects a detection target from an image captured by a camera, the object detection device having a control unit, The control unit Dividing the image into a plurality of regions based on distribution information generated using a plurality of images previously acquired to show a distribution state of appearance positions of the detection target objects in the image for each size of the detection target objects; setting a threshold value for an inference result of a learning model that has learned the feature amount of the detection object for each of the divided regions; approximating the boundaries of the plurality of regions with quadratic curves; Object detection device.

3. The control unit Detecting the detection target from the image captured by the camera according to a comparison result between the inference result for the image and the threshold value. The object detection device according to claim 1 or 2.

4. The control unit Detecting the detection object from the image captured by a camera fixed to the vehicle; The object detection device according to any one of claims 1 to 3.

5. The control unit Dividing the image into the plurality of regions based on the distribution information generated based on images captured by cameras fixed to the other vehicles. The object detection device according to claim 4 .

6. The control unit Dividing the image into the plurality of regions according to the size distribution based on the distribution information; The larger the size distributed in the region, the higher the threshold is set. The object detection device according to any one of claims 1 to 5.

7. The control unit Dividing the image into the plurality of regions according to a detection purpose of the detection object; setting the threshold value for each of the plurality of regions according to the detection purpose; The object detection device according to any one of claims 1 to 6.

8. An object detection method for detecting a detection target from an image captured by a camera, comprising: Dividing the image radially into a plurality of regions based on distribution information generated using a plurality of images previously acquired to show a distribution state of appearance positions of the detection target objects in the image for each size of the detection target objects; a threshold value for an inference result of a learning model that has learned the feature amount of the detection object is set for each of the divided regions; Object detection methods.

9. An object detection program that detects a detection target from an image captured by a camera, Dividing the image radially into a plurality of regions based on distribution information generated using a plurality of images previously acquired to show a distribution state of appearance positions of the detection target objects in the image for each size of the detection target objects; a threshold value for an inference result of a learning model that has learned the feature amount of the detection object is set for each of the divided regions; Object detection program.

Images