Object detection apparatus, image processing method, image processing program, and onboard object detection system

Abstract

To provide a technique for suppressing reduction in accuracy of detecting a target object and reducing processing load, with balance.SOLUTION: An object detection apparatus includes a controller for detecting a target object included in an image to be processed. The controller is configured to: estimate a segmentation state of the target object obtained by segmenting the image to be processed; determine an overlap region in the image segmentation based on the estimated result of the segmentation state; divide the image to be processed into a plurality of segmented images having the overlap region; execute processing to detect the target object on the segmented images; and integrate the results of the detection processing on the segmented images, to detect the target object included in the image to be processed.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an image processing technique for detecting an object in an image. [Background technology]

[0002] Conventionally, many models such as SSD, YOLO, and Faster R-CNN are known as object detection algorithms using deep neural networks (DNNs). In these models, input images are usually resized to a predetermined resolution (e.g., 640 x 640 pixels) to reduce computational costs. Then, the resized images are used to perform inference for object detection.

[0003] For example, when the input image is an HD (resolution 1280 x 720 pixels) or Full HD (resolution 1920 x 1080 pixels) image, the resolution of the image is reduced by the above-mentioned resizing process. This causes a problem that the detection accuracy of small objects in the original image before resizing is likely to decrease during inference using the model.

[0004] Conventionally, a method is known in which an original image is divided into multiple images instead of being resized (see, for example, Patent Document 1). This method can improve the accuracy of detecting small objects in the original image compared to when the image is resized. Patent Document 1 also discloses that dividing an original image into multiple overlapping regions, and dividing the image with overlapping areas in this manner allows for a later synthesis process to be performed with high accuracy. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-144589 A Summary of the Invention [Problem to be solved by the invention]

[0006] When an original image is divided into a plurality of divided images and object detection is performed using a model, there is a possibility that the detection target in the original image may be cut off during division, and if such a cut occurs, there is a concern that the accuracy of object detection may decrease. When the detection target in the original image is small, there is a particular concern that the accuracy of object detection may decrease.

[0007] In response to the above concerns, for example, it is conceivable to set the overlap rate of the overlapping portion described above to a large value. By setting the overlap rate to a large value, the possibility that an uncut detection target object is included in at least one of the multiple divided images increases, so that it is possible to suppress a decrease in the accuracy of object detection. However, if the overlap rate is simply set to a large value, there is a concern that the image size of each divided image will increase, resulting in an increase in the processing load (calculation load).

[0008] In view of the above, an object of the present invention is to provide a technique that can achieve a good balance between suppressing a decrease in the detection accuracy of a target object and reducing the processing load. [Means for solving the problem]

[0009] An exemplary object detection device of the present invention is an object detection device having a controller that detects a target object included in an image to be processed. The controller estimates a cutting state of the target object when the image to be processed is divided into images, determines an overlapping area in the image division based on the estimated cutting state, divides the image to be processed into a plurality of divided images having the overlapping areas, executes a detection process of the target object for each of the divided images, and detects the target object included in the image to be processed by integrating the results of the detection process for the plurality of divided images. Effect of the Invention

[0010] According to the exemplary embodiment of the present invention, it is possible to achieve a good balance between suppressing a decrease in the detection accuracy of an object and reducing the processing load. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an object detection system. [Diagram 2] A block diagram showing a functional configuration of a controller included in an object detection device. [Diagram 3] 1 is a flowchart showing an outline of an object detection process executed by an object detection device. [Figure 4A] A schematic diagram of an image before division [Figure 4B] FIG. 4B is a schematic diagram for explaining the division process of the image shown in FIG. 4A; [Figure 4C] FIG. 4C is a diagram showing a schematic diagram of an inference result for each divided image divided by the division process shown in FIG. [Figure 4D] FIG. 4D is a schematic diagram showing the result of integration processing performed based on the inference results of each divided image shown in FIG. [Diagram 5] Schematic diagram to explain IoU [Figure 6A] A schematic diagram showing how the overlapping area is reduced to divide the image. [Figure 6B] FIG. 6B is a diagram showing a schematic diagram of an inference result for each divided image divided in the state shown in FIG. 6A. [Figure 7] FIG. 6B is a schematic diagram showing a state in which the image shown in FIG. 6A is divided by enlarging the overlapping area. [Figure 8] FIG. 1 is a diagram for explaining variations in width of overlapping regions; [Figure 9] 1 is a flowchart illustrating an example of a flow of an image division process. [Figure 10] A diagram showing an example of the relationship between density, bounding box (Bbox) size, and overlap rate. [Figure 11] A flowchart showing a modified example of image division processing [Figure 12A] FIG. 13 is a schematic diagram for explaining a third modified example; [Figure 12B] FIG. 13 is a schematic diagram for explaining a third modified example; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0013] <1. Object detection system> Fig. 1 is a diagram showing a schematic configuration of an object detection system 100 according to an embodiment of the present invention. As shown in Fig. 1, the object detection system 100 includes a camera 1 and an object detection device 2. Specifically, the object detection system 100 is an in-vehicle object detection system mounted on a vehicle. However, the object detection system of the present invention is not limited to in-vehicle use, and can be applied to various uses such as road facilities, commercial facilities, factories, and offices.

[0014] The camera 1 is provided so as to be able to communicate with the object detection device 2 via wired or wireless communication. The camera 1 outputs a captured image (more specifically, image data; the same applies below) to the object detection device 2. In this embodiment, the camera 1 is an on-board camera mounted on a vehicle. The camera 1 captures images of the surroundings of the vehicle. The number of cameras 1 may be one or more. The camera 1 is, for example, at least one of a front camera that captures images in front of the vehicle, a rear camera that captures images behind the vehicle, and a side camera that captures images to the sides of the vehicle.

[0015] The object detection device 2 receives an input of an image captured by the camera 1. The object detection device 2 detects a target object (hereinafter, may be simply referred to as a target object) included in the input image (image to be processed) using a known object detection algorithm. Examples of known object detection algorithms include SSD, YOLO, and Faster R-CNN. The object detection device 2 of this embodiment uses YOLO as the object detection algorithm. The object detection device 2 is configured to process an image and detect a target object in the image, and is an example of an image processing device.

[0016] In this embodiment, the object detection device 2 is an in-vehicle device mounted on a vehicle. However, the object detection device is not limited to an in-vehicle device, and may be, for example, a server device that is provided so as to be able to communicate with the in-vehicle device. Note that the in-vehicle device configured as the object detection device 2 may include a portable device (for example, a mobile device) that can be taken out of the vehicle and brought into the vehicle.

[0017] The object detection device 2 is a computer device including a controller 21 and a memory 22. The controller 21 includes an arithmetic circuit that performs arithmetic processing. The arithmetic circuit is specifically configured with a processor. The processor includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The controller 21 may be configured with one processor or multiple processors. When configured with multiple processors, it is sufficient that those processors are arranged to be able to communicate with each other.

[0018] The memory 22 includes a volatile memory and a non-volatile memory. The volatile memory is specifically a RAM (Random Access Memory). The non-volatile memory is specifically a ROM (Read Only Memory). The non-volatile memory may also be a flash memory, a hard disk drive, or the like. The non-volatile memory stores programs and data that can be read by a computer.

[0019] In this embodiment, the memory 22 stores a trained model 22a that has been trained by machine learning. In this embodiment, the machine learning is deep learning. The model 22a is a trained model that has been trained by a known learning method such as an error backpropagation method using a training data set that is a collection of teacher data. The controller 21 performs inference using the model 22a on the input image and detects an object that has been set in advance as a detection target. The object may be, for example, at least one of a vehicle, a person, a traffic light, and a road sign. The object to be detected is not limited to one type, and multiple types may be set.

[0020] The object detection system 100 further includes a driving control device 3. The driving control device 3 controls, for example, at least one of the brakes, the steering wheel, and the accelerator of the vehicle. The driving control device 3 is provided so as to be able to communicate with the object detection device 2 via wire or wirelessly. The driving control device 3 performs driving control according to the detection result of the object detection device 2. For example, when it is determined that another vehicle or a person is present near the host vehicle according to the detection result of the object detection device 2, the driving control device 3 performs one or more of the brake control, the steering wheel control, and the accelerator control.

[0021] The driving control device 3 does not necessarily have to be provided. Instead of the driving control device 3, or in addition to the driving control device 3, another device such as a notification device that is capable of communicating with the object detection device 2 may be provided. The notification device may be configured to notify the driver of the vehicle of an alarm or the like by voice, screen display, or the like, for example.

[0022] <2. Object detection device> Fig. 2 is a block diagram showing a functional configuration of the controller 21 included in the object detection device 2. As shown in Fig. 2, the controller 21 includes, as its functions, an acquisition unit 211, a division unit 212, an inference unit 213, an integration unit 214, and an output unit 215. The functions of the controller 21 are realized by a processor executing arithmetic processing in accordance with a program stored in the memory 22. The number of programs that realize the functions of the controller 21 may be one or more.

[0023] The program stored in the memory 22 is a computer program that causes a computer to realize the functions of the controller 21 (in other words, the object detection device 2). Such a computer program may be provided, for example, by a computer-readable nonvolatile recording medium. The nonvolatile recording medium may be, for example, an optical recording medium (for example, an optical disk), a magneto-optical recording medium (for example, a magneto-optical disk), a USB memory, or an SD card, in addition to the above-mentioned nonvolatile memory. As another example, the computer program may be provided from a program providing server via a communication line such as the Internet, or provided by so-called downloading.

[0024] Further, each of the functional units 211 to 215 may be realized by one program, but may be configured to be realized by separate programs for each functional unit. Further, each of the functional units 211 to 215 may be realized by having a processor execute a program, that is, by software, as described above, but may be realized by other methods. Each of the functional units 211 to 215 may be realized by using, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). That is, each of the functional units 211 to 215 may be realized by hardware using a dedicated IC or the like. Further, each of the functional units 211 to 215 may be realized by using both software and hardware. Further, each of the functional units 211 to 215 is a conceptual component. A function executed by one component may be distributed to multiple components. Further, functions possessed by multiple components may be integrated into one component.

[0025] The acquisition unit 211 acquires information (data) from the outside. In detail, the acquisition unit 211 sequentially acquires images periodically transmitted from the camera 1. In other words, images are input to the controller 21. The acquisition unit 211 may acquire information other than images. For example, the acquisition unit 211 may acquire operation information of a vehicle (host vehicle) in which the host device 2 is mounted, by using CAN (Controller Area Network) communication. The operation information of the host vehicle may be, for example, speed information, steering angle information, or blinker information.

[0026] The division unit 212 divides the image acquired by the acquisition unit 211 into a plurality of divided images while providing overlapping areas. That is, the controller 21 divides the input image into a plurality of divided images while providing overlapping areas. The input image is a photographed image captured by the camera 1 mounted on the vehicle. When dividing one image into a plurality of divided images, the division unit 212 judges how to divide the image, and divides the image according to the judgment result. In other words, when dividing one image into a plurality of divided images, the division unit 212 determines the division conditions of the image, and divides the image under the determined division conditions. The division conditions include a judgment related to the overlapping areas. Details of the division image generation process by the division unit 212 will be described later.

[0027] The inference unit 213 performs inference using the model (trained model) 22a. In this embodiment, as described above, the model 22a uses YOLO, which is known as an object detection algorithm. The model 22a performs processing to identify the position (area) and type (class) of an object previously determined as a detection target from an input image. The type of object (object) to be detected may be appropriately determined according to the purpose of object detection. The object is, for example, a car, a motorcycle, a bicycle, or a person. The type of object set as a detection target may be single or multiple.

[0028] Images acquired sequentially by the acquisition unit 211 are input in order to the model 22a. However, in this embodiment, each image acquired by the acquisition unit 211 is, in principle, divided into a plurality of divided images by the division unit 212 before being input to the model 22a. Therefore, in principle, a plurality of divided images are input in order to the model 22a for each image acquired by the acquisition unit 211. Note that, in this embodiment, there are exceptional cases where an image acquired by the acquisition unit 211 is not divided into a plurality of divided images by the division unit 212, but this will be described later.

[0029] For each of the multiple divided images, inference is performed using the model 22a. Therefore, for each image acquired by the acquisition unit 211, an inference result is obtained for each divided image. That is, for each image acquired by the acquisition unit 211, an inference result is obtained for each divided part (area). In a configuration in which one image is divided into multiple divided images, the size of each divided image becomes smaller than the size of the original image due to the division of the image. For this reason, it is not necessary to perform image resizing (downsampling), which causes a decrease in the detection accuracy of small objects in the image. Even when the divided images are resized, the reduction ratio (degree of downsampling) can be made smaller than when the image is resized without being divided. That is, in a configuration in which one image is divided into multiple divided images and inference is performed for each divided image, a decrease in the detection accuracy of the object can be suppressed compared to when the original (before division) image is resized.

[0030] The integration unit 214 performs integration processing to integrate the inference results for each divided image, obtained by dividing one image into a plurality of divided images, into an inference result for one image. This integration processing results in an object detection result for one image acquired by the acquisition unit 211. In other words, the controller 21 detects an object in an image (original image) based on the processing results for each of the plurality of divided images.

[0031] In the vicinity of the boundary between adjacent divided images, objects to be detected may be detected in an overlapping manner. In the integration process, such overlapping detection results are deleted, etc. Details of the integration process will be described later. In this embodiment, as described above, when an image is divided, the division is performed while providing overlapping areas. This makes it easy to align the divided images with each other, and makes it easier to perform the integration process.

[0032] The output unit 215 outputs to the outside the detection result (object detection result) of the object for each image acquired by the acquisition unit 211. In this embodiment, the output unit 215 outputs the object detection result to the driving control device 3. As described above, the driving control device 3 performs driving control of the vehicle according to the acquired object detection result.

[0033] 3. Object detection method The object detection method (object detection process) executed by the object detection device 2 will be described with a concrete example.

[0034] [3-1. Overview of object detection processing] First, a rough flow of the object detection process will be described. Fig. 3 is a flowchart showing a rough flow of the object detection process executed by the object detection device 2. The object detection process by the object detection device 2 is started, for example, when various devices constituting the object detection system 100 are started. Specifically, for example, when a power switch such as an ACC power supply of the vehicle is turned on, the object detection process by the object detection device 2 is started.

[0035] In step S1, the acquisition unit 211 starts acquiring images. The images acquired by the acquisition unit 211 are sequentially processed in steps S2 to S7 described below in the order of acquisition. Once the images are acquired, the process proceeds to the next step S2.

[0036] In step S2, processing of the image with the highest priority in acquisition order is started, and the dividing unit 212 divides the image to be processed according to the determined division conditions. The process of deciding the division conditions will be described later.

[0037] FIG. 4A is a diagram that shows a schematic diagram of an image 200 before division. FIG. 4B is a schematic diagram for explaining a division process of the image 200 shown in FIG. 4A. In the example shown in FIG. 4B, the image 200 is divided into four divided images 201-204. All of the four divided images 201-204 are rectangular. Moreover, the four divided images 201-204 have the same size (area). Note that these are merely examples, and the number of divisions of an image may be other than four. Moreover, the sizes of the multiple divided images may be different from each other. Moreover, the divided images may have a shape other than a rectangle (for example, a triangular shape).

[0038] In the example shown in Fig. 4B, a first divided image 201 shown in a solid rectangular frame is mainly an image of the upper left part of the image 200 before division. A second divided image 202 shown in a dashed-dotted rectangular frame is mainly an image of the upper right part of the image 200 before division. A third divided image 203 shown in a thick dashed rectangular frame is mainly an image of the lower left part of the image 200 before division. A fourth divided image 204 shown in a dashed-dotted rectangular frame is mainly an image of the lower right part of the image 200 before division.

[0039] The right end of the first divided image 201 is located to the right of the first bisector 301 that bisects the image 200 before division in the left-right direction by a length W1 / 2 (meaning half of W1). On the other hand, the left end of the second divided image 202 is located to the left of the first bisector 301 by a length W1 / 2. For this reason, the first divided image 201 and the second divided image 202 overlap and include a rectangular region R1 with a width W1 (=W1 / 2×2) in the left-right direction that is located above the center of the image 200 before division. The region R1 thus included in the multiple divided images 201 and 202 corresponds to an overlap region.

[0040] The lower end of the first divided image 201 is located a length W2 / 2 (meaning half of W2) below the second bisector 302 that vertically bisects the image 200 before division. On the other hand, the upper end of the third divided image 203 is located a length W2 / 2 above the second bisector 302. For this reason, the first divided image 201 and the third divided image 203 overlap and include a rectangular region R2 of vertical width W2 (=W2 / 2×2) located to the left of the vertical center of the image 200 before division. The region R2 thus included in the multiple divided images 201 and 203 corresponds to an overlap region.

[0041] The bottom end of the second divided image 202 is located below the second bisector 302 by a length W2 / 2. Meanwhile, the top end of the fourth divided image 204 is located above the second bisector 302 by a length W2 / 2. For this reason, the second divided image 202 and the fourth divided image 204 overlap and include a rectangular region R3 of width W2 in the vertical direction that is located to the right of the vertical center of the image 200 before division. The region R3 thus included in the multiple divided images 202 and 204 corresponds to an overlap region.

[0042] The right end of the third divided image 203 is located to the right of the first bisector 301 by a length W1 / 2. Meanwhile, the left end of the fourth divided image 204 is located to the left of the first bisector 301 by a length W1 / 2. For this reason, the third divided image 203 and the fourth divided image 204 overlap with each other and include a rectangular region R4 of width W1 in the left-right direction that is located below the center in the left-right direction of the image 200 before division. The region R4 thus included in the multiple divided images 203 and 204 corresponds to an overlap region.

[0043] As can be seen from the above, in the division process, the image 200 is divided into a plurality of divided images 201 to 204 while providing overlapping regions R1 to R4. Returning to Fig. 3, when the division process is completed, the process proceeds to the next step S3.

[0044] In step S3, the inference unit 213 sets a variable N to "1." The variable N is a variable that indicates the number of inferences. When the variable N is set to "1," the process proceeds to the next step S4.

[0045] In step S4, the inference unit 213 performs inference on the Nth divided image (here, the first divided image corresponds since N=1). As described above, in this embodiment, YOLO is used as the object detection algorithm, so information in the following format is output as the inference result. When the inference is completed, the process proceeds to the next step S5. format [x,y,w,h,conf ] x, y : center coordinates of the bounding box w : The horizontal (x) length of the bounding box h : The vertical (y) length of the bounding box conf : Inference value (confidence value)

[0046] In the format, x, y, w, and h are information that specifies the area (position) of the detection target. The inference value (confidence value) is a confidence score (confidence information) that indicates the probability of the existence of an object. The confidence score is, for example, a numerical value between 0 and 1, and the closer the value is to "1", the higher the confidence level.

[0047] If there are multiple types of objects to be detected, "conf" is output separately for each type of object. For example, if the types of objects to be detected are "car", "truck", and "bus", the format will be as follows: format [x,y,w,h,car_conf,track_conf,bus_conf ] car_conf : The inference value (confidence value) of the class "car" track_conf: Inference value (confidence value) of class "track" bus_conf : Inference value (confidence value) of class "bus"

[0048] In step S5, the inference unit 213 adds "1" to the variable N. When the addition process for the variable is completed, the process proceeds to the next step S6.

[0049] In step S6, the inference unit 213 judges whether or not inference has been completed for all the divided images obtained in step S2. Whether or not inference has been completed for all the divided images is judged based on whether or not the variable N has exceeded the number of divided images. If the variable N exceeds the number of divided images, it is judged that inference has been completed for all the divided images. If the variable N does not exceed the number of divided images, it is judged that inference has not been completed for all the divided images. For example, in the case where the number of divided images is four, if the variable N is 5, it is judged that inference has been completed for all the divided images, and if the variable N is less than 5, it is judged that inference has not been completed for all the divided images. If it is judged that inference has been completed for all the divided images (Yes in step S6), the process proceeds to the next step S7. On the other hand, if it is judged that inference has not been completed for all the divided images (No in step S6), the process returns to step S4, and the processes after step S4 are performed. In step S4, for example, if the variable N is 2, inference is performed for the second divided image, and if the variable N is 3, inference is performed for the third divided image.

[0050] Fig. 4C is a diagram showing a schematic diagram of the inference results of each divided image 201-204 divided by the division process shown in Fig. 4B. As shown in Fig. 4C, in each divided image 201-204, a bounding box BX showing the position of an object detected as a result of inference is displayed superimposed on the image. Note that the inference result displayed superimposed on the image may include a class and an inference value in addition to the bounding box BX.

[0051] Returning to FIG. 3, in step S7, the integration unit 214 performs integration processing to make the inference results obtained for each divided image into the inference result for the original image. The same object may be detected overlapping between divided images near the boundaries of multiple divided images. In the integration processing, it is determined whether the object detected in each divided image is the same. If it is determined that it is the same object, a process is performed to combine the detection results into one. If it is determined that it is not the same object, it is determined that each detection result detects a different object, and the detection results are left without being combined into one.

[0052] FIG. 4D is a schematic diagram showing the result of the integration process performed based on the inference results of the divided images 201 to 204 shown in FIG. 4C. It is determined that the object detected in the first divided image 201 (indicated by a bounding box BX1) and the object detected in the third divided image 203 (indicated by a bounding box BX3) are the same object. For this reason, in FIG. 4D showing the result of the integration process, one bounding box BX5 obtained by integrating the two bounding boxes BX1 and BX3 is displayed superimposed on the original image (image before division) 200. It is also determined that the object detected in the second divided image 202 (indicated by a bounding box BX2) and the object detected in the fourth divided image 204 (indicated by a bounding box BX4) are the same object. For this reason, in FIG. 4D showing the result of the integration process, one bounding box BX6 obtained by integrating the two bounding boxes BX2 and BX4 is displayed superimposed on the original image 200.

[0053] Whether or not they are the same object is determined, for example, by comparing the detection result of the object in each divided image with the detection result of the object in another divided image. The comparison of the detection results is performed after performing coordinate alignment between divided images having overlapping regions. Specifically, the comparison of the detection results may be a comparison between bounding boxes. The degree of overlap or continuity between the bounding boxes may be determined to determine whether or not they are the same object. For example, if the degree of overlap between the bounding boxes is large, they are determined to be the same object. Also, for example, if it is determined that the bounding boxes are connected, they are determined to be the same object.

[0054] The integration process by the integration unit 214 may be performed using a known NMS (Non Maximum Suppression) process. The NMS process is a process for removing a part of overlapping candidate regions. In this example, when coordinate matching is performed between divided images having overlapping regions, multiple bounding boxes, some of which overlap with each other, correspond to the candidate regions. The NMS process may remove some of the multiple candidate regions that overlap with each other.

[0055] In the NMS process, the intersection over union (IoU), which is the overlap rate between candidate regions, is calculated. IoU is calculated using the following formula (1). IoU = X / Y (1) X: The area of ​​the overlap between the two candidate regions Y: The total area of ​​the two overlapping candidate regions IoU is an index that indicates the degree of overlap between candidate regions, and the closer the value is to "1", the greater the overlap.

[0056] Fig. 5 is a schematic diagram for explaining IoU. In Fig. 5, a vehicle VE (object) to be detected is included in an image, and a first candidate area BX1 and a second candidate area BX3 are extracted from the vehicle VE. The first candidate area BX1 and the second candidate area BX3 correspond to the bounding boxes BX1 and BX3 shown in Fig. 4C, respectively.

[0057] 5, X in the above formula (1) is the area of ​​the overlapping portion (hatched portion) of first candidate region BX1 and second candidate region BX3. Y in formula (1) is obtained by adding the area of ​​first candidate region BX1 and the area of ​​second candidate region BX3 together and subtracting the area of ​​the overlapping portion of first candidate region BX1 and second candidate region BX3 (the area obtained as X above) from the sum.

[0058] In the NMS process, if the calculated IoU value exceeds a preset threshold, a part of the candidate region is removed. When a part of the candidate region is removed, the candidate regions that overlap with each other are removed except for the one with the highest inference value. If the calculated IoU value is equal to or less than a preset threshold, each candidate region is left without being removed.

[0059] Returning to Fig. 3, in step S8, the division unit 212 determines whether or not to perform object detection processing for the next image. If there is no special circumstance such as a command to stop object detection processing, a command to turn off the power, or the occurrence of an error, it is determined that the next image will be processed. If it is determined that the next image will be processed (Yes in step S8), the process returns to step S2, and the processes from step S2 onwards are performed. If it is determined that the next image will not be processed (No in step S8), the process shown in Fig. 3 is terminated.

[0060] [3-2. Details of division process] Next, a detailed description will be given of the division process executed by the division unit 212. Note that the division process referred to here corresponds to the process of step S2 in FIG.

[0061] As described above, in the segmentation process, an image is segmented into multiple parts while providing overlapping areas. However, if the overlapping areas are made too large, the size of each segmented image obtained after segmentation becomes large, and the burden (computational cost burden) of the inference process in the model 22a increases. For this reason, from the perspective of computational cost, it is preferable to make the overlapping areas as small as possible. However, making the overlapping areas small gives rise to the following concerns.

[0062] Fig. 6A is a schematic diagram showing a state in which an image 200 is divided by reducing the overlapping area. Fig. 6B is a schematic diagram showing the inference results of each of the divided images 201-204 divided in the state shown in Fig. 6A. Note that in the examples shown in Fig. 6A and Fig. 6B, the overlapping areas R1, R4 between the left and right divided images are made small (narrow), but the overlapping areas R2, R3 between the top and bottom divided images are not made particularly small.

[0063] When dividing image 200, an object in image 200 may be cut off. Even if an object is cut off by division, if the object is detected in at least one of the divided images 201 to 204, then basically, when an integration process is performed, the object cut off by division is also correctly detected in image 200 (original image).

[0064] However, as shown in FIG. 6A and FIG. 6B, when a small object (small object) 500 in the image 200 is cut, the cut small object 500 may be too small to be properly detected in each divided image 201-204. In FIG. 6B, for example, in the third divided image 203, the small object 500 is not detected, and a bounding box BX indicating detection is not displayed. If the small object 500 cannot be detected in all divided images 201-204, the small object 500 is not detected. Note that a cut that adversely affects the detection of a small object is a state in which the small object 500 is cut in all divided images, and none of the divided images has a full image or a partial image (an image that occupies a large proportion of the full image) of the small object 500 that allows the small object 500 to be identified.

[0065] In addition, as shown in Fig. 6A, when the overlapping area is reduced and the small object 500 is cut off, the accuracy of the judgment of the degree of overlap and continuity during the integration process is likely to decrease. For this reason, for example, when the small object 500 is detected in multiple divided images 201, 202, and 204 as in the example shown in Fig. 6B, the integration process cannot be performed appropriately, and a situation may occur in which a single object is actually detected as multiple objects.

[0066] As can be seen from the above, when dividing the image 200 into a plurality of divided images 201 to 204, it is preferable to divide the image 200 so as to prevent cutting of small objects as much as possible. In other words, it is preferable to divide the image 200 so as to reduce the probability that small objects are cut.

[0067] Fig. 7 is a schematic diagram showing a state in which the image 200 shown in Fig. 6A is divided by enlarging the overlapping regions R1 and R4. As shown in Fig. 7, when the image 200 is divided by enlarging the overlapping regions R1 and R4, the probability that the small object 500 will fit within the overlapping regions R1 and R4 without being divided increases. In other words, when the image 200 is divided by enlarging the overlapping regions R1 and R4, the probability that the small object 500 will be cut off can be reduced. For this reason, from the viewpoint of detection accuracy, it is preferable to make the overlapping regions as large as possible.

[0068] Therefore, in this embodiment, the division unit 212 (controller 21) is configured to estimate the situation in which the object in the image 200 is cut and vary the overlapping regions R1 to R4 when dividing the image 200. With this configuration, it is possible to achieve a good balance between suppressing the decrease in the detection accuracy of the object and reducing the processing load for detecting the object. For example, when it is determined that the small object is highly likely to be cut based on the estimation result of the cutting situation, the overlapping region is enlarged as shown in FIG. 6B. This makes it possible to suppress the decrease in the detection accuracy of the object. On the other hand, when it is determined that the small object is less likely to be cut based on the estimation result of the cutting situation, the overlapping region is reduced as shown in FIG. 6A. This makes it possible to reduce the size of the divided image input to the model 22a and reduce the processing load during inference.

[0069] (3-2-1. Estimation of the circumstances under which the object is cut) As described above, the overlap region is varied by estimating the situation in which an object in an image is cut. In this embodiment, the division unit 212 (controller 21) estimates the situation in which an object in an image is cut based on the density of the object. If the density of the object in an image is high, the probability that the object will be cut by dividing the image increases. On the other hand, if the density of the object in an image is low, the probability that the object will be cut by dividing the image decreases. In other words, the situation in which an object will be cut can be estimated by determining the density of the object in an image.

[0070] In this embodiment, the division unit 212 (controller 21) estimates the situation in which an object in an image is cut based on the processing result of a previous image in which processing to detect an object was performed prior to the image in question. In such a configuration, the situation in which an object in an image is cut can be estimated by successively using the detection results of past objects, and therefore the probability of appropriately estimating the situation in which an object is cut can be improved.

[0071] In detail, the division unit 212 (controller 21) obtains the density based on the processing result of a previous image in which processing for detecting objects was performed before the current processing image. The previous image is preferably an image immediately before the current processing image in which processing for detecting objects was performed. By using a previous image in which object detection was performed earlier (especially immediately before), it is possible to improve the estimation accuracy of the density of objects in the current processing target image.

[0072] The density of the object in the image may be calculated in a numerical format, such as the ratio of the object to a predetermined region of the image. However, since it is only necessary to estimate the situation in which the object is cut, the density of the object in the image may be calculated qualitatively. In other words, the configuration may simply determine whether the density of the predetermined region of the image is high or low. The predetermined region of the image may be, for example, the entire image, or a part of the image, such as a region where the object is likely to exist or a region where the object may be cut by dividing the image. The predetermined region of the image may be, for example, the center of the image or a strip-shaped region extending to the left and right including the center. The predetermined region is not limited to one, and may be multiple.

[0073] For example, if the number of objects in a specified area exceeds a preset threshold, the density may be determined to be high. If the density is determined to be high, the possibility of small objects being cut off may be high, and therefore the overlap area may be set large to suppress a decrease in detection accuracy. If the number of objects in a specified area is equal to or less than a preset threshold, the density may be determined to be low. If the density is determined to be low, the possibility of small objects being cut off may be low, and therefore the overlap area may be set small to reduce the processing load.

[0074] In addition, when there are multiple types of objects to be detected, the density may be estimated including all of them, but the density may be calculated by focusing on only some of the objects. For example, when there is an object that is unlikely to be cut by dividing the image, the density may be calculated by excluding such an object.

[0075] Furthermore, the division unit 212 (controller 21) may obtain the density based on shooting location information related to a shooting location other than the image (a location where the image to be processed is shot). When using external information in this manner, the density can be obtained with reduced processing load. For example, the shooting location information other than the image (external information) may include at least one of traffic information, map information, and intersection information at the shooting location. The traffic information is, for example, VICS (Vehicle Information and Communication System (registered trademark)) information. The map information is, for example, map information stored for navigation. The intersection information may be, for example, information acquired from the map information.

[0076] Here, it is assumed that the object is a vehicle. In this case, if the traffic information determines that the road on which the vehicle is traveling is congested, the density of the object may be determined to be high. Also, for example, if the map information determines that the area on which the vehicle is traveling is an urban area, the density of the object may be determined to be high. Conversely, if the map information determines that the area on which the vehicle is traveling is a suburban area, the density of the object may be determined to be low. Also, for example, if the object is a traffic light, the density of the object may be determined from intersection information. For example, if it is determined that the vehicle is about to pass an intersection or is passing through an intersection, the density of the traffic light, which is the object, may be determined to be high. Also, for example, if the object is a car or a person, the density of the object may be determined using past accumulated information (accumulated information related to congestion, etc.) and time period information. For example, if it is determined that it is a commuting time such as morning or evening, the density of the object may be determined to be high.

[0077] (3-2-2. Example of overlap area fluctuation) In this embodiment, the dividing unit 212 (controller 21) varies the overlap area by changing the width of the overlap area, which allows the overlap area to be varied by a simple process.

[0078] FIG. 8 is a diagram for explaining the variation in width of the overlap region. In FIG. 8, as in FIG. 4B, the rectangular portion of the thick solid line indicates the first divided image 201, the rectangular portion of the one-dot chain line indicates the second divided image 202, the rectangular portion of the thick dashed line indicates the third divided image 203, and the rectangular portion of the two-dot chain line indicates the fourth divided image 204. The first divided image 201 and the second divided image 202 overlap in the left-right direction to form a rectangular overlap region R1 with a left-right width W1. The first divided image 201 and the third divided image 203 overlap in the up-down direction to form a rectangular overlap region R2 with a up-down width W2. The second divided image 202 and the fourth divided image 204 overlap in the up-down direction to form a rectangular overlap region R3 with a up-down width W2. The third divided image 203 and the fourth divided image 204 overlap in the left-right direction to form a rectangular overlap region R4 with a left-right width W1.

[0079] In Fig. 8, thin dashed line 301 is a first bisector that bisects pre-division image 200 in the left-right direction, and thin dashed line 302 is a second bisector that bisects pre-division image 200 in the top-bottom direction. In the description using Fig. 8, the four divided images 201 to 204 are assumed to be the same size.

[0080] As shown in FIG. 8, in this embodiment, the width of the overlap region includes a width W1 in the left-right direction and a width W2 in the up-down direction. When the width W1 in the left-right direction is changed, the overlap rate between the first divided image 201 and the second divided image 202 adjacent to each other in the left-right direction is changed. Similarly, when the width W1 in the left-right direction is changed, the overlap rate between the third divided image 203 and the fourth divided image 204 adjacent to each other in the left-right direction is changed. Furthermore, when the width W2 in the up-down direction is changed, the overlap rate between the first divided image 201 and the third divided image 203 adjacent to each other in the up-down direction is changed. Similarly, when the width W2 in the up-down direction is changed, the overlap rate between the second divided image 202 and the fourth divided image 204 adjacent to each other in the up-down direction is changed.

[0081] That is, as a change in the width of the overlap region to vary the overlap region, at least one of a change in the width W1 in the left-right direction and a change in the width W2 in the up-down direction is assumed. Which of the multiple assumed modes (three in detail) to adopt may be determined, for example, by conducting an experiment or a simulation. As another example, the multiple modes may be selectable by the user by switching settings.

[0082] When changing the horizontal width W1 of the overlap area, for example, the following process is performed. The width W1 / 2 by which the right ends of the first divided image 201 and the third divided image 203 protrude rightward from the first bisector 301 and the width W1 / 2 by which the left ends of the second divided image 202 and the fourth divided image 204 protrude leftward from the first bisector 301 are simultaneously changed. Increasing the protruding width W1 / 2 increases the width of the overlap areas R1 and R4. Reducing the protruding width W1 / 2 decreases the width of the overlap areas R1 and R4.

[0083] Furthermore, when changing the vertical width W2 of the overlapping area, for example, the following process is performed. The width W2 / 2 by which the lower ends of the first divided image 201 and the second divided image 202 protrude downward from the second bisector 302 and the width W2 / 2 by which the upper ends of the third divided image 203 and the fourth divided image 204 protrude upward from the second bisector 302 are simultaneously changed. Increasing the protruding width W2 / 2 increases the width of the overlapping areas R2 and R3. Reducing the protruding width W2 / 2 decreases the width of the overlapping areas R2 and R3.

[0084] In this embodiment, the division unit 212 (controller 21) widens the width of the overlap region when it is determined that the density of the objects is high, and narrows the width of the overlap region when it is determined that the density of the objects is low. This reduces the possibility that small objects in the image are cut off by division. When varying the width of the overlap region, the width may be changed between two settings, "wide" and "narrow" (two-step change). However, the width of the overlap region may be changed in three or more steps.

[0085] The reason for widening the width of the overlap region when it is determined that the density of objects is high is to reduce the possibility that small objects are cut off. As a means for reducing the possibility that small objects are cut off, instead of or in addition to widening the width of the overlap region, the number of divided images may be reduced. That is, when dividing an image, the controller 21 may vary the number of divided images by estimating the situation in which an object in the image is cut off.

[0086] (3-2-3. Flow of division process) 9 is a flowchart illustrating an example of the flow of image division processing, and is a flowchart showing the detailed flow of the processing in step S2 in FIG.

[0087] In step S21, the division unit 212 determines whether the image being processed is a first image. The first image is the image acquired by the acquisition unit 211 first from the camera 1 that captures images continuously in time. For example, it is the image acquired by the acquisition unit 211 first from the camera 1 when the vehicle is turned on. If it is determined to be the first image (Yes in step S21), the process proceeds to the next step S22. If it is determined not to be the first image (No in step S21), the process proceeds to step S24.

[0088] In step S22, the division unit 212 performs a resizing process to reduce the resolution of the image to a predetermined resolution. For example, an image with a resolution of 1280 x 720 pixels is resized to a resolution of 640 x 640 pixels by the resizing process. When the resizing process is completed, the process proceeds to the next step S23.

[0089] In step S23, an object detection model is used to perform inference for detecting an object in the previously resized image. The object detection model may be the above-mentioned model 22a, but in some cases may be a separately prepared model (a machine-learned model). The inference outputs the class and position of the object. When the inference is completed, the process proceeds to the next step S24.

[0090] In step S24, the division unit 212 estimates the state in which the image to be processed is cut by dividing the image. The estimation is performed using the density of the object obtained from the detection result of the object in the immediately preceding image. Since there is no immediately preceding image in the initial image, the density of the object is obtained according to the result of the inference performed in step S23. For example, when the density of the object is high, it is estimated that there is a high possibility that a small image will be cut by dividing the image. When the density of the object is low, it is estimated that there is a low possibility that a small image will be cut by dividing the image. When the cutting state of the object by dividing the image is estimated, the process proceeds to the next step S25.

[0091] In step S25, the division unit 212 sets the overlap area when dividing the image according to the estimation result of the cutting state of the object. That is, the overlap area in the image division is determined based on the estimation result of the cutting state. When the division unit 212 determines that the possibility of cutting a small object is high from the estimation result of the cutting state of the object, the division unit 212 selects a setting in which the width of the overlap area is wide (for example, see the setting of the overlap area shown in FIG. 7) in order to reduce the possibility of cutting the small object. When the division unit 212 determines that the possibility of cutting a small object is low from the estimation result of the cutting state of the object, the division unit 212 selects a setting in which the width of the overlap area is narrow (for example, see the setting of the overlap area shown in FIG. 6B) in order to prioritize reduction of the processing load. Note that here, it is assumed that two settings, "wide width" and "narrow width", can be selected for the setting of the overlap area. When the setting of the overlap area is completed, the process proceeds to the next step S26.

[0092] In step S26, the division unit 212 cuts the image at positions according to the setting of the overlap area to generate a plurality of divided images. Each divided image has an overlap area. When the plurality of divided images are generated, the process from step S3 in FIG. 3 is performed.

[0093] In the processing example shown in FIG. 9, in order to estimate the situation in which the object is cut by dividing the image, the first image is resized and inference is performed, but this is merely an example. For example, instead of obtaining the density in the first image using the immediately preceding image, the density may be estimated using shooting location information other than the image (for example, VICS information, map information, etc.). In this case, the first image may not be resized and inference using the resized image may not be performed. In addition, the first image may be configured to have a setting in which the width of the overlap area is widened without estimating the situation in which the object is cut, with a priority given to suppressing the decrease in the detection accuracy of the object. In this case, the first image may not be resized and inference using the resized image may not be performed. In addition, inference using the resized image may be performed periodically, not just for the first image.

[0094] <4. Modifications> [4-1. First modified example] In the embodiment described above, the situation in which an object in an image is cut is estimated based on the density of the object. However, this is an example, and for example, the division unit 212 (controller 21) may estimate the situation in which an object in an image is cut based on the size of the object. That is, the division unit 212 may estimate the situation in which an object in an image is cut based on the density and size of the object. This makes it possible to more appropriately estimate the situation in which an object in an image is cut.

[0095] The division unit 212 (controller 21) may obtain the size of the object based on the processing result of a previous image in which processing to detect the object was performed before the current image to be processed. This makes it possible to properly obtain the size of the object in the image to be divided before dividing the image. Furthermore, in this configuration, since the density and size of the object can be obtained from the same previous image, it is possible to suppress an increase in the processing load. Note that the previous image is preferably an image (immediately preceding image) immediately before the image to be processed. The size of the object may be, for example, the size of a bounding box. The size of the bounding box may be, for example, the area of ​​the bounding box.

[0096] Fig. 10 is a diagram showing an example of the relationship between the density, the size of the bounding box (Bbox), and the overlap rate. In the example shown in Fig. 10, a "small" overlap rate means that the width of the overlap area is set to be narrow. Also, a "large" overlap rate means that the width of the overlap area is set to be wide.

[0097] 10, a "high" density level means that the density level calculated from the immediately preceding image, etc. is greater than a preset density threshold value, and a "low" density level means that the density level calculated from the immediately preceding image, etc. is equal to or less than a preset density threshold value.

[0098] In the example shown in FIG. 10, a bounding box being "large" means that the size of the bounding box calculated from the immediately preceding image, etc. is larger than a preset size threshold. A bounding box being "small" means that the size of the bounding box calculated from the immediately preceding image, etc. is equal to or smaller than a preset size threshold. If there are multiple objects in an image, the size of the bounding box may be determined to be "large" or "small" by, for example, comparing the average value of the areas of the bounding boxes of the multiple objects with the above-mentioned size threshold. Instead of the average value, the size of the bounding box may be determined to be "large" or "small" as follows. If there are a number of bounding boxes whose size exceeds a preset size threshold that exceeds a preset number threshold, the size of the bounding box is determined to be "large", and otherwise the size of the bounding box is determined to be "small".

[0099] In the example shown in Fig. 10, if the density of objects in the image to be segmented is high and the size of the bounding box of the object is large, the overlap rate is reduced. This is because, although the density of objects is high, they are likely to be large in size and likely to contain many characteristic parts for detecting the object even if cut by segmentation, so it is considered that the detection accuracy is unlikely to decrease. In such a case, a setting that reduces the overlap area is selected to prioritize reduction in processing load.

[0100] In the example shown in Fig. 10, if the density of objects in the image to be segmented is high and the size of the bounding box of the object is small, the overlap rate is increased. This is because it is estimated that the density of small objects is high and that these small objects are likely to be cut off by segmentation. In such a case, a setting that increases the overlap area is selected so that small objects are not cut off as much as possible, that is, by prioritizing suppression of a decrease in detection accuracy.

[0101] In the example shown in Fig. 10, if the density of objects in the image to be segmented is low and the bounding box of the object is large, the overlap rate is reduced. This is because it is estimated that small objects are unlikely to be cut off, and even if a large object is cut off, the detection accuracy is unlikely to decrease. In such a case, a setting that reduces the overlap area is selected, prioritizing reduction in processing load.

[0102] In addition, if the density in the image to be divided is low and the size of the bounding box is large, it is estimated that there are a small number of very large objects (possibly only one) in the image. In such a case, even if a resizing process is performed to reduce the resolution of the image, there is a low possibility that the detection accuracy will decrease. For this reason, it may be decided to perform a resizing process without dividing the image.

[0103] In the example shown in Figure 10, if the density of objects in the image to be segmented is low and the size of the bounding box is small, the overlap rate is reduced. This is because it is estimated that the possibility of cutting off the entire object, including small objects, is low. In such a case, a setting that reduces the overlap area is selected to prioritize reduction in processing load.

[0104] [4-2. Second modified example] In the embodiment described above, the overlap area is varied according to the estimated result of the situation in which the object in the image is cut off, but this is merely an example. The division unit 212 (controller 21) may vary the overlap area by further taking into account vehicle operation information. The vehicle operation information is operation information of the host vehicle that is equipped with the object detection device 2. With such a configuration, it is possible to achieve a better balance between suppressing the deterioration of the detection accuracy of the object and reducing the processing load.

[0105] In this modified example, as a preferred embodiment, the operation information includes the speed of the vehicle. For example, when the speed of the vehicle is slow, even if the detection accuracy of a small object that exists far from the vehicle and appears small in the image is reduced, the effect is not necessarily large. For this reason, even if the density of objects is determined to be high, if the speed of the vehicle is equal to or less than a predetermined first speed threshold (for example, 30 km / h or less), the overlap area may be made small by prioritizing reduction of the processing load. Conversely, when the speed of the vehicle is fast, even if the object exists far from the vehicle and appears small in the image, the approach speed is fast, so that a decrease in the detection accuracy of the object is not preferable. For this reason, even if the density of objects is determined to be low, if the speed of the vehicle exceeds a predetermined second speed threshold (for example, 50 km / h), the overlap area may be made large so that the small object is less likely to be cut off. Note that the first speed threshold and the second speed threshold may be different or the same.

[0106] The controller 21 can obtain the speed of the vehicle by using, for example, CAN communication. The controller 21 may obtain the speed of the vehicle from the amount of change (amount of movement) per unit time of the position of the vehicle obtained from GPS (Global Positioning System) information.

[0107] Fig. 11 is a flowchart showing a modified example of the image division process. Fig. 11 is a modified example of the flowchart shown in Fig. 9. In Fig. 11, the process of step S27 is performed between the process of step S23 and the process of step S24 in Fig. 9. The process of step S27 is a process in which the division unit 212 acquires operation information of the host vehicle (a detailed example is speed information). Note that the process of step 27 may be configured to be performed, for example, between the process of step S24 and the process of step S25 in Fig. 9.

[0108] In the case of the modified example shown in FIG. 11, in the overlap area setting process in step S25, the overlap area is set by adding the operation information of the host vehicle to the estimated result of the cutting state of the object. For example, when it is determined that the density of the object is high and the speed of the host vehicle exceeds a predetermined first speed threshold, the width of the overlap area is set wide. Even when it is determined that the density of the object is high, the width of the overlap area is set narrow when the speed of the host vehicle is equal to or lower than the predetermined first speed threshold. When it is determined that the density of the object is low and the speed of the host vehicle is equal to or lower than the predetermined second speed threshold, the width of the overlap area is set narrow. Even when it is determined that the density of the object is low, the width of the overlap area is set wide when the speed of the host vehicle exceeds the predetermined second speed threshold.

[0109] [4-3.Third modified example] In the embodiment described above, the situation in which an object in an image is cut off is estimated based on the density of the object, and the overlap area is varied based on the estimation result, but this is merely an example.

[0110] The estimation of the situation in which the object in the image is cut off may be performed based on the traveling situation of the vehicle. The traveling situation of the vehicle may include, for example, the traveling environment in which the vehicle is traveling and the driving situation of the vehicle. The traveling environment of the vehicle may include, for example, the shape of the traveling road. Furthermore, the driving situation of the vehicle may include a situation such as a planned left turn or a planned right turn. The shape of the traveling road can be acquired from, for example, map information. Furthermore, the driving situation such as a planned right turn can be acquired from, for example, CAN information.

[0111] 12A and 12B are schematic diagrams for explaining the third modified example. Here, as an example, the situation in which an object in an image is cut off is assumed to be estimated based on the shape of the road RO. In FIG. 12A and FIG. 12B, the shape of the road RO on which the vehicle is traveling is curved to the right. When the road RO on which the vehicle is traveling is determined to be curved to the right based on map information or the like, the probability that the object vehicle exists on the left side of the image 200A is small. For this reason, the possibility that the object OB is divided can be reduced by arranging the overlap area RA between the left and right divided images toward the left side of the image (corresponding to FIG. 12B) rather than arranging it in the center of the image (corresponding to FIG. 12A).

[0112] That is, the division unit 212 (controller 21) may vary the overlap area based on the cut-off state of the object in the image estimated based on the traveling state of the vehicle. Then, the division unit 212 (controller 21) may vary the overlap area by changing the position of the overlap area. With this configuration, it is possible to reduce the possibility that the object will be cut off.

[0113] In a configuration in which the position of the overlap region is changed, the width of the overlap region may be changed together with the change in the position of the overlap region.

[0114] <5. Things to keep in mind> Various technical features disclosed in the description of the invention may be modified in various ways without departing from the spirit of the technical creation. Furthermore, the embodiments and modifications disclosed in the description of the invention may be combined to the extent possible. [Explanation of symbols]

[0115] 1. Camera 2. Object detection device (on-vehicle device) 21. Controller 100 Object detection system (vehicle-mounted object detection system) 200, 200A...Image 201...First divided image 202...Second divided image 203...Third divided image 204... 4th divided image R1, R2, R3, R4, RA... overlapping area

Claims

1. An object detection device having a controller for detecting a target object included in an image to be processed, The controller: Estimating a cutting state of the target object when the processing target image is divided into images; determining an overlap region in the image division based on the estimation result of the cutting state; Dividing the processing target image into a plurality of divided images having the overlapping regions; performing a detection process of the target object for each of the divided images; detecting a target object included in the processing target image by integrating results of the detection process for the plurality of divided images; Object detection device.

2. The object detection device according to claim 1 , wherein the controller estimates the cutting state of the target object based on an object detection processing result of a previous processing target image in which processing to detect the target object was performed prior to the processing target image.

3. The object detection device according to claim 1 , wherein the controller estimates the cutting state of the target object based on a density of the target object.

4. The object detection device according to claim 3 , wherein the controller calculates the density based on a detection result of the target object in a previous processing target image in which processing for detecting the target object was performed prior to the current processing target image.

5. The object detection device according to claim 3 , wherein the controller calculates the density based on shooting location information relating to a shooting location of the image to be processed.

6. The object detection device according to claim 5 , wherein the image-capturing location information includes at least one of traffic information, map information, and intersection information at the image-capturing location.

7. The object detection device according to claim 3 , wherein the controller estimates the cutting state of the target object further based on a size of the target object.

8. The object detection device according to claim 7 , wherein the controller determines the size of the target object based on a detection result of the target object in a previous processing target image in which processing for detecting the target object was performed prior to the current processing target image.

9. the processing target image is an image captured by a camera mounted on a vehicle, The object detection device according to claim 1 , wherein the controller varies the overlap region by taking into account operation information of the vehicle.

10. The object detection device according to claim 9 , wherein the vehicle operation information includes a speed of the vehicle.

11. The object detection device according to claim 1 , wherein the controller varies the overlap region by changing a width of the overlap region.

12. The object detection device according to claim 3 , wherein the controller increases a width of the overlap region when the density is determined to be high, and decreases a width of the overlap region when the density is determined to be low.

13. The object detection device according to claim 1 , wherein the controller changes a position of the overlap region to vary the overlap region.

14. 1. An image processing method for detecting a target object included in a processing target image, comprising: Estimating a cutting state of the target object when the processing target image is divided into images; determining an overlap area in the image division based on the result of the division state estimation; Dividing the processing target image into a plurality of divided images having the overlapping regions; performing a detection process of the target object for each of the divided images; detecting a target object included in the processing target image by integrating results of the detection process for the plurality of divided images; Image processing methods.

15. An image processing program for detecting a target object included in a processing target image, estimating a cutting state of the target object when the processing target image is divided into images; determining an overlap region in the image division based on the estimation result of the cutting state; Dividing the processing target image into a plurality of divided images having the overlapping regions; performing a detection process of the target object for each of the divided images; detecting a target object included in the processing target image by integrating results of the detection process in the plurality of divided images; An image processing program that causes a computer to execute the above.

16. A camera that captures the surroundings of the vehicle; an in-vehicle device to which the processing target image captured by the camera is input; Equipped with The in-vehicle device includes: Estimating a cutting state of the target object when the processing target image is divided into images; determining the overlap region in the image division based on the estimation result of the cutting state; Dividing the processing target image into a plurality of divided images having the overlapping regions; performing a detection process of the target object for each of the divided images; detecting a target object included in the processing target image by integrating results of the detection process for the plurality of divided images; In-vehicle object detection system.

Images