Object detection device and object detection method

Abstract

To provide an object detection device and an object detection method capable of avoiding delay of end timing of processing of object detection.SOLUTION: In an object detection device, first detection means 72 specifies a label of an object appearing in an input frame and specifies a position of a bounding box of the object, and predicts a region of the bounding box in a frame input after the frame on the basis of a history of the position of the bounding box of the same object. The second detection means 73 specifies the label of the imaged object and specifies the position of the bounding box of the object with respect to the region of the predicted bounding box in the frame input later. The first detection means 72 and the second detection means 73 share hardware resources. The arbitration means 76 controls on / off of a start prohibition flag for prohibiting the start of the processing of the first detection means.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an object detection device, an object detection method, and an object detection program. [Background technology]

[0002] Patent Document 1 describes a monitoring device equipped with a camera control means having at least two control modes according to the moving speed of an object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-68184 Summary of the Invention [Problem to be solved by the invention]

[0004] "Detecting an object" means identifying the label of an object captured in a frame input from a camera and identifying the position of the bounding box of that object. A label indicates the type of object captured in the frame. For example, if the object to be detected is a car or a person, the object label "car" or "person" is identified.

[0005] The inventors have considered the following object detection device as an object detection device that detects objects from frames continuously input from a camera. Figure 8 is a block diagram showing the object detection device considered by the inventors. Object detection device 100 includes first detection unit 111, second detection unit 112, and pre-processing unit 121.

[0006] The first detection unit 111 can detect an object from a frame with high accuracy, but the detection takes time.

[0007] The second detection unit 112 has low detection accuracy but can detect an object in a short time.

[0008] Even if the first detection unit 111 takes time for detection, it is assumed that the desired throughput (F [fps]) can be achieved. In other words, it is assumed that the first detection unit 111 and the second detection unit 112 can each process one frame in 1 / F seconds. Note that [fps] is an abbreviation for "frames per second." In this case, one cycle is 1 / F seconds.

[0009] The first detection unit 111 and the second detection unit 112 are realized by a processor that does not allow interrupts to its processing. Examples of processors that do not allow interrupts to its processing include a GPU (Graphics Processing Unit) and an AI (Artificial Intelligence) chip. The following describes an example in which the first detection unit 111 and the second detection unit 112 are realized by a GPU 110.

[0010] The preprocessing unit 121 is realized by a CPU (Central Processing Unit) 120 provided separately from the GPU 110 .

[0011] Preprocessing is required for detection. Examples of preprocessing include image resizing, color conversion (e.g., conversion from RGB to GBR), and data type conversion (e.g., conversion from int to float). The preprocessing for detection by the second detection unit 112 is equivalent to the detection by the second detection unit 112 in terms of delay. Therefore, the preprocessing for detection by the second detection unit 112 is included in the processing of the second detection unit 112 and is executed by the GPU 110.

[0012] In terms of delay, the pre-processing for detection by the first detection unit 111 has a low priority, so the pre-processing for detection by the first detection unit 111 is executed by the CPU 120 (pre-processing unit 121).

[0013] When the first detection unit 111 identifies the label of an object shown in a frame and the position of the bounding box through detection, it predicts the area of ​​the bounding box, for example, several frames ahead, based on the history of the bounding box position, and outputs the predicted bounding box area to the second detection unit 112. The second detection unit 112 targets the predicted bounding box area, identifies the label of the object shown, and identifies the position of the bounding box of that object.

[0014] The second detector 112 performs detection within the bounding box area predicted by the first detector 111, and therefore can achieve both low latency and high object detection accuracy.

[0015] FIG. 9 is an ideal timing chart for the object detection device shown in FIG. 8. The numbers in FIG. 9 represent frame numbers. DL seconds is the time from when a frame is input to the GPU 110 until the second detection unit 112 starts processing that frame. DH seconds is the time it takes the pre-processing unit 121 to perform pre-processing for detection by the first detection unit 111. Note that the value of DH may vary. The processing time for the processing performed by the second processing unit 112 within one cycle is assumed to be constant at PL seconds. Similarly, the processing time for the processing performed by the first detection unit 111 within one cycle is assumed to be constant at PH seconds. The values ​​of PL and PH are constants obtained by approximately accurate estimation.

[0016] 9, the first detection unit 111 predicts the area of ​​the bounding box in the next frame and transmits this area to the second detection unit 112. The second detection unit 112 detects an object within the predicted area.

[0017] In the ideal timing chart shown in FIG. 9, the second detector 112 obtains the object detection result (DL+PL) seconds after a frame is input.

[0018] 10 is a timing chart showing an example in which it takes a long time for preprocessing by the preprocessing unit 121. Since the time for preprocessing varies, there are cases in which preprocessing takes a long time.

[0019] As shown in Fig. 10, in the second cycle, preprocessing takes time, and the preprocessing time is assumed to be DH' seconds. This delays the timing at which the processing by the first detection unit 111, which started in the second cycle, ends. Here, since the processing by the second detection unit 112 cannot be interrupted, in the third cycle, the processing by the second detection unit 112 starts when the processing by the first detection unit 111 ends. As a result, in the third cycle, an object detection result is obtained (α+PL) seconds after the frame input.

[0020] The time from the frame input to the end of processing by the second detection unit 112 is preferably (DL+PL) seconds, as shown in the example of FIG.

[0021] Therefore, an object of the present disclosure is to provide an object detection device, an object detection method, and an object detection program that can avoid delays in the timing at which the processing of the second detection unit ends. [Means for solving the problem]

[0022] The object detection device according to the present disclosure is an object detection device that receives successive input frames, and includes a first detection means that identifies the label of an object appearing in the input frame and the position of the object's bounding box, and predicts the area of ​​the bounding box in a frame input after that frame based on a history of the position of the bounding box of the same object, and a second detection means that targets the predicted bounding box area in the subsequently input frame to identify the label of the object appearing in the input frame and the position of the object's bounding box, wherein the first detection means and the second detection means share hardware resources, and wherein, when the frame input period is 1 / F seconds, the processing time of the first detection means corresponding to one period is PH seconds, and the processing time of the second detection means corresponding to one period is PL seconds, the device includes an arbitration means that controls the on / off of a start prohibition flag for prohibiting the first detection means from starting processing based on 1 / F, PH, and PL, and the first detection means starts processing when the start prohibition flag is off.

[0023] The object detection method according to the present disclosure is an object detection method applied to an object detection device that receives successive input frames, wherein a first detection means of the object detection device identifies the label of an object appearing in the input frame and the position of the object's bounding box, and predicts the area of ​​the bounding box in a frame input after that frame based on a history of the position of the bounding box of the same object; a second detection means of the object detection device targets the predicted bounding box area in the frame input after that and identifies the label of the object appearing in the frame and the position of the object's bounding box; an arbitration means of the object detection device controls the on / off of a start prohibition flag for prohibiting the first detection means from starting processing based on 1 / F, PH, and PL, where the frame input period is 1 / F seconds, the processing time of the first detection means corresponding to one period is PH seconds, and the processing time of the second detection means corresponding to one period is PL seconds; and the first detection means starts processing when the start prohibition flag is off.

[0024] The object detection program of the present disclosure is an object detection program installed in a computer to which frames are input continuously, and causes the computer to perform a first detection process that identifies the label of an object appearing in the input frame and the position of the object's bounding box, and predicts the area of ​​the bounding box in a frame input after that frame based on a history of the position of the bounding box of the same object; a second detection process that targets the predicted bounding box area in a frame input after that frame and identifies the label of the object appearing in the frame and the position of the object's bounding box; and an arbitration process that controls the on / off of a start prohibition flag for prohibiting the start of the first detection process based on 1 / F, PH, and PL, where the frame input period is 1 / F seconds, the processing time of the first detection process corresponding to one period is PH seconds, and the processing time of the second detection process corresponding to one period is PL seconds, and causes the computer to start the first detection process when the start prohibition flag is off. [Effects of the Invention]

[0025] According to the present disclosure, it is possible to avoid delays in the timing at which the processing of the second detection unit ends. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a block diagram illustrating an example configuration of an object detection device according to the present disclosure. [Figure 2] FIG. 10 is a schematic diagram showing an example of history information. [Figure 3] 10 is a timing chart illustrating an example of the operation of the object detection device according to the present disclosure. [Figure 4] 10 is a timing chart showing a case where the arbitration unit 6 turns off the start prohibition flag immediately after the start of the processing time of the second detection unit 3. [Figure 5] This is an example of a timing chart in which processing for a plurality of frames is combined into one process, and then the process is divided into a first half process and a second half process. [Figure 6]FIG. 2 is a schematic block diagram illustrating an example of the configuration of a computer related to the object detection device. [Figure 7] 1 is a block diagram illustrating an overview of an object detection device according to the present disclosure. [Figure 8] 1 is a block diagram showing an object detection device studied by the inventors. [Figure 9] 9 is an ideal timing chart of the object detection device shown in FIG. 8. [Figure 10] 10 is a timing chart showing an example in which preprocessing by the preprocessing unit 121 takes a long time. DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings.

[0028] As mentioned above, "detecting an object" means identifying the label of an object captured in a frame input from a camera and identifying the position of the bounding box of that object.

[0029] 1 is a block diagram showing an example configuration of an object detection device according to the present disclosure. The object detection device 1 includes a first detection unit 2, a second detection unit 3, a preprocessing unit 4, a delay measurement unit 5, and an arbitration unit 6.

[0030] The first detection unit 2 and the second detection unit 3 are implemented by a processor that does not allow interrupts to its processing. Examples of processors that do not allow interrupts to its processing include GPUs and AI chips. Below, an example will be described in which the first detection unit 2 and the second detection unit 3 are implemented by a GPU 10. The first detection unit 2 and the second detection unit 3 share the hardware resource of the GPU 10.

[0031] The first detection unit 2 can detect an object from the frame with high accuracy, but the detection takes time.

[0032] The second detection unit 3 has low detection accuracy but can detect an object in a short time.

[0033] Even if it takes time for detection in the first detector 2, it is assumed that the desired throughput (F [fps]) can be achieved. In other words, it is assumed that the first detector 2 and the second detector 3 can each process one frame in 1 / F seconds.

[0034] The preprocessing unit 4 is realized by the CPU 20 .

[0035] The delay measurement unit 5 is realized by the CPU 30.

[0036] The arbitration unit 6 is realized by the CPU 40 .

[0037] The preprocessing unit 4, the delay measurement unit 5, and the arbitration unit 6 may be realized by the same CPU.

[0038] Frames captured by a camera are continuously input to the GPU 10. The continuously input frames have consecutive frame numbers. Each frame is input to the GPU 10 at a constant frame rate. This frame rate is assumed to be F [fps].

[0039] The first detection unit 2 holds a model generated in advance by machine learning such as deep learning. The first detection unit 2 then applies the model to an entire input frame, thereby identifying the label of an object shown in the frame and determining the position of the bounding box of the object. The number of objects detected from a single frame is not limited to one, and multiple objects may be detected. In this case, the first detection unit 2 identifies the label of each object and the position of the bounding box of the object for each object.

[0040] When the first detection unit 2 detects an object from a frame, it assigns an ID to the identified bounding box. If multiple bounding box positions are obtained, the first detection unit 2 assigns an ID to each bounding box. At this time, the first detection unit 2 assigns the same ID to bounding boxes that share a common object (in other words, bounding boxes that are presumed to share a common object).

[0041] The first detection unit 2 then generates history information. The history information is information that indicates, for each ID, the history of combinations of frame numbers and bounding box positions. FIG. 2 is a schematic diagram showing an example of history information. The example shown in FIG. 2 indicates, for each bounding box ID, the history of combinations of frame numbers, labels, and bounding box positions. In this example, the position of a bounding box is specified by the coordinates of the upper left vertex and the lower right vertex of the bounding box. For example, (x1, y1) shown in FIG. 2 are the coordinates of the upper left vertex of the bounding box, and (x2, y2) are the coordinates of the lower right vertex of the bounding box.

[0042] Based on the history information, the first detection unit 2 predicts the area of ​​the bounding box in a frame input after the frame in which the object was detected. The delay measurement unit 5 determines how far after the frame the bounding box area is predicted.

[0043] The delay measurement unit 5 monitors the frame number of the frame currently being processed by the first detection unit 2 and the frame number of the next frame to be input to the second detection unit 3, and subtracts the former from the latter. The first detection unit 2 predicts the area of ​​the bounding box in the frame that follows by the subtraction result. For example, suppose the frame currently being processed by the first detection unit 2 has frame number 11, and the frame number to be next input to the second detection unit 3 is 13. In this case, the first detection unit 2 predicts the area of ​​the bounding box in the frame that is 2 (=13-11) frames later than the frame currently being processed.

[0044] The first detection unit 2 may predict the area of ​​the bounding box in a frame input after the frame in which the object is detected, by linear prediction based on the history information.

[0045] Alternatively, the first detection unit 2 may predict the area of ​​the bounding box in a frame input after the frame in which the object is detected by using a Kalman filter based on the history information.Furthermore, the first detection unit 2 may predict the area of ​​the bounding box in a frame input after the frame in which the object is detected by using a movement prediction model generated by machine learning such as deep learning based on the history information.

[0046] The first detector 2 outputs to the second detector 3 the predicted bounding box area in a frame to be input later.

[0047] The second detection unit 3 identifies the label of the object shown and the position of the bounding box for the bounding box area predicted by the first detection unit 2 in the most recent input frame. If there are multiple bounding box areas predicted by the first detection unit 2, the second detection unit 3 identifies the label of the object shown and the position of the bounding box for each predicted bounding box area.

[0048] The second detection unit 3 holds a model that has been generated in advance by machine learning such as deep learning. The second detection unit 3 then applies the predicted bounding box area in the latest frame to the model, thereby identifying the label of the object in the image and the position of the bounding box of the object. The size of the model held by the second detection unit 3 may be smaller than the size of the model held by the first detection unit 2.

[0049] When the processing is completed, the second detection unit 3 transmits a notification that the processing has been completed to the arbitration unit 6. The arbitration unit 6 recognizes from this notification that the processing by the second detection unit 3 has been completed.

[0050] The preprocessing unit 4 performs preprocessing for the detection performed by the first detection unit 2. As mentioned above, examples of preprocessing include image resizing, color conversion, data type conversion, etc. The time required for this preprocessing may vary.

[0051] Since the amount of pre-processing for detection by the second detection unit 3 is small, the pre-processing for detection by the second detection unit 3 is included in the processing by the second detection unit 3 and is executed by the GPU 10.

[0052] When frames are input at a frame rate of F [fps], the input period of the frames is 1 / F seconds.

[0053] The processing time performed by the first detection unit 2 within one cycle is assumed to be constant at PH seconds. Similarly, the processing time performed by the second detection unit 3 within one cycle is assumed to be constant at PL seconds. The values ​​of PH and PL are constants obtained by fairly accurate estimation. Also, it is assumed that PH + PL < 1 / F.

[0054] Based on 1 / F, PH, and PL, arbitration unit 6 controls the on / off of a start prohibition flag for prohibiting the start of processing by first detection unit 2. Specifically, arbitration unit 6 turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of second detection unit 3 corresponding to one cycle, and turns off the start prohibition flag at any timing between immediately after the start of the processing time of second detection unit 3 corresponding to the next cycle and the end of that processing time.

[0055] The first detection unit 2 does not start processing when the start prohibition flag is on, and starts processing when the start prohibition flag is off. Furthermore, once the first detection unit 2 starts processing, it may continue the processing even if the start prohibition flag is turned on during the processing.

[0056] If the first detection unit 2 starts processing after (1 / F-PL-PH) seconds or more have passed since the end of processing by the second detection unit 3, the processing by the first detection unit 2 will inevitably overlap with the ideal processing timing of the second detection unit 3 for the next cycle, delaying the start of processing by the second detection unit 3 for the next cycle. Therefore, the arbitration unit 6 turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of the second detection unit 3 corresponding to one cycle, preventing the first detection unit 2 from starting processing until the start prohibition flag is turned off.

[0057] 3 is a timing chart showing an example of the operation of the object detection device according to the present disclosure, in which the numbers shown in FIG. 3 represent frame numbers.

[0058] 3, DL seconds is the time from when a frame is input to the GPU 10 until the second detection unit 3 starts processing that frame. DH seconds and DH' seconds shown in Fig. 3 are the time it takes the pre-processing unit 4 to perform pre-processing for detection by the first detection unit 2. DL seconds is shorter than the time it takes the pre-processing unit 4 to perform pre-processing.

[0059] 3, the case where the arbitration unit 6 turns off the start prohibition flag when the processing time of the second detection unit 3 ends will be described as an example.

[0060] The start timing of a cycle is the time when a frame is input. In the first cycle, the second detection unit 3 starts processing when DL seconds have elapsed after the start of the cycle. When this processing is completed, the second detection unit 3 sends a notification to the arbitration unit 6 that the processing has been completed. The arbitration unit 6 turns off the start prohibition flag when it receives this notification, and turns on the start prohibition flag (1 / F-PL-PH) seconds after that point.

[0061] In the first cycle, preprocessing by the preprocessing unit 4 ends while the start prohibition flag is off (see FIG. 3), and so when preprocessing ends, the first detection unit 2 starts processing for frame 1. Even if the start prohibition flag turns on after this processing has started, the first detection unit 2 may continue processing.

[0062] Here, the frame number of the next frame input to the second detection unit 3 is 2, so after detecting an object, the first detection unit 2 predicts the area of ​​the bounding box in the frame one frame after frame 1 (i.e., frame 2). The first detection unit 2 outputs the predicted area of ​​the bounding box to the second detection unit 3.

[0063] In the second cycle, the second detector 3 starts processing DL seconds after the cycle starts. At this time, the second detector 3 performs detection in frame 2 within the bounding box area predicted by the first detector 2, and outputs the detection results (the object label and the position of the object's bounding box).

[0064] The arbitration unit 6 turns off the start prohibition flag when the processing by the second detection unit 3 ends, and turns on the start prohibition flag (1 / F-PL-PH) seconds after that point.

[0065] In the second cycle, the preprocessing by the preprocessor 4 ends while the start prohibition flag is on (see FIG. 3). Therefore, the first detector 2 cannot start processing for frame 2 when the preprocessing ends.

[0066] In the third cycle, the second detector 3 starts processing when DL seconds have elapsed since the start of the cycle. Then, when PL seconds have elapsed since the start of this processing (i.e., (DL+PL) seconds after the start of the cycle), the arbitrator 6 turns off the start prohibition flag. At this point, the first detector 2 starts processing for frame 2 (see FIG. 3). Even if the start prohibition flag is turned on after this processing starts, the first detector 2 may continue processing.

[0067] Here, the frame number of the next frame input to the second detection unit 3 is 4, so after the first detection unit 2 detects an object, it predicts the area of ​​the bounding box in the frame two frames after frame 2 (i.e., frame 4). The first detection unit 2 outputs the predicted area of ​​the bounding box to the second detection unit 3.

[0068] In the fourth cycle, DL seconds after the cycle starts, the second detector 3 starts processing. At this time, the second detector 3 performs detection in frame 4 within the bounding box area predicted by the first detector 2, and outputs the detection results (the object label and the position of the object's bounding box).

[0069] The arbitration unit 6 turns off the start prohibition flag when the processing by the second detection unit 3 ends, and turns on the start prohibition flag (1 / F-PL-PH) seconds after that point.

[0070] In this way, the object detection device 1 proceeds with the processing.

[0071] According to this embodiment, in each cycle, the second detection unit 3 can end the process (DL+PL) seconds after the start of the cycle, and delays in the timing at which the second detection unit 3 ends the process can be avoided.

[0072] 3 has been described as an example in which the arbitration unit 6 turns off the start prohibition flag at the end of the processing time of the second detection unit 3. Here, while the second detection unit 3 is processing, the first detection unit 2 cannot start processing even if the start prohibition flag is off. This is because the GPU 10 cannot interrupt processing.

[0073] Therefore, the timing at which the arbitration unit 6 changes the start prohibition flag from on to off may be any timing from immediately after the start of the processing time of the second detection unit 3 to the end of that processing time. FIG. 4 is a timing chart for when the arbitration unit 6 changes the start prohibition flag off immediately after the start of the processing time of the second detection unit 3. The example shown in FIG. 4 is the same as the example shown in FIG. 3 except for the timing at which the arbitration unit 6 changes the start prohibition flag from on to off. Therefore, in the example shown in FIG. 4 as well, the second detection unit 3 can end processing in each cycle (DL+PL) seconds after the start of the cycle, thereby avoiding a delay in the timing at which the second detection unit 3 ends its processing.

[0074] In the above embodiment, the first detection unit 2 and the second detection unit 3 are provided with the GPU 10 as a hardware resource. The first detection unit 2 and the second detection unit 3 may be implemented by separate processors, with the two processors sharing memory and communication paths as hardware resources. Even in this case, delays in the timing at which the processing of the second detection unit 3 ends can be avoided by performing operations similar to those in the above embodiment.

[0075] In the above embodiment, it is assumed that PH+PL<1 / F. The processing time PH of the first detection unit 2 for one frame may be too long, resulting in (PL+PH) exceeding 1 / F seconds. In such a case, even if the first detection unit 2 processes multiple frames (e.g., three frames) at once using parallel processing or the like, the processing time is only slightly longer than the processing time for one frame. If the processing time for these multiple frames (e.g., three frames) is divided into a first half process and a second half process, and the time for the first half process and the time for the second half process are respectively designated as PH, then PH+PL<1 / F can be satisfied. The above embodiment can also be applied in this case. FIG. 5 is an example of a timing chart when the processing for multiple frames is combined into one process and then divided into a first half process and a second half process.

[0076] 6 is a schematic block diagram showing an example of the configuration of a computer related to the object detection device. The computer 2000 includes, for example, a GPU 2001, CPUs 2002 to 2004, a main memory device 2005, an auxiliary memory device 2006, and an interface 2007.

[0077] The object detection device according to the present disclosure is realized by, for example, a computer 2000. The operation of the object detection device is stored in the form of a program (object detection program) in an auxiliary storage device 2006. A GPU 2001 and CPUs 2002 to 2004 load the program into a main storage device 2005 and execute the processing described in the above embodiment in accordance with the program. The GPU 2001 operates as a first detection unit 2 and a second detection unit 3. The CPU 2002 operates as a preprocessing unit 4. The CPU 2003 operates as a delay measurement unit 5. The CPU 2004 operates as an arbitration unit 6.

[0078] The auxiliary storage device 2006 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a semiconductor memory connected via the interface 2007.

[0079] Next, an overview of the object detection device according to the present disclosure will be described. FIG. 7 is a block diagram showing an overview of the object detection device according to the present disclosure. Frames are input consecutively to the object detection device according to the present disclosure. The object detection device according to the present disclosure includes a first detection means 72, a second detection means 73, and an arbitration means 76.

[0080] The first detection means 72 (e.g., the first detection unit 2) identifies the label of an object shown in an input frame and the position of the object's bounding box, and predicts the area of ​​the bounding box in a frame input after that frame based on the history of the position of the bounding box of the same object.

[0081] The second detection means 73 (e.g., the second detection unit 3) targets the predicted bounding box area in a frame that is input later, identifies the label of the object that is captured, and identifies the position of the object's bounding box.

[0082] The first detection means 72 and the second detection means 73 share hardware resources.

[0083] The arbitration means 76 (e.g., arbitration unit 6) controls the on / off of a start prohibition flag for prohibiting the start of processing by the first detection means 72 based on 1 / F, PH, and PL, where the input period of a frame is 1 / F seconds, the processing time of the first detection means 72 corresponding to one period is PH seconds, and the processing time of the second detection means 73 corresponding to one period is PL seconds.

[0084] The first detection means 72 starts processing when the start prohibition flag is off.

[0085] With such a configuration, delay in the timing at which the processing of the second detection means 73 ends can be avoided.

[0086] The above embodiment can also be described as follows, but is not limited to the following:

[0087] (Appendix 1) An object detection device in which frames are input continuously, a first detection means for identifying a label of an object shown in an input frame and a position of a bounding box of the object, and predicting an area of ​​the bounding box in a frame input after the input frame based on a history of the position of the bounding box of the same object; a second detection means for identifying a label of an object appearing in the subsequently input frame and a position of the bounding box of the object, the second detection means being configured to target an area of ​​the predicted bounding box in the subsequently input frame; the first detection means and the second detection means share hardware resources; an arbitration means for controlling on / off of a start prohibition flag for prohibiting start of processing by the first detection means, based on 1 / F, PH, and PL, where the input period of a frame is 1 / F seconds, the processing time of the first detection means corresponding to one period is PH seconds, and the processing time of the second detection means corresponding to one period is PL seconds; The first detection means starts processing when the start prohibition flag is off. An object detection device characterized by:

[0088] (Appendix 2) The arbitration means turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of the second detection means corresponding to one period, and turns off the start prohibition flag at any timing between immediately after the start of the processing time of the second detection means corresponding to the next period and the end of the processing time. 10. The object detection device of claim 1.

[0089] (Appendix 3) An object detection method applied to an object detection device to which frames are continuously input, comprising: a first detection means of the object detection device identifies a label of an object shown in an input frame and a position of a bounding box of the object, and predicts an area of ​​the bounding box in a frame input after the input frame based on a history of positions of the bounding box of the same object; a second detection means of the object detection device, which targets the region of the predicted bounding box in the subsequently input frame, identifies a label of the object shown in the frame and identifies a position of the bounding box of the object; an arbitration means of the object detection device controls on / off of a start prohibition flag for prohibiting start of processing by the first detection means, based on 1 / F, PH, and PL, when an input period of a frame is 1 / F seconds, a processing time of the first detection means corresponding to one period is PH seconds, and a processing time of the second detection means corresponding to one period is PL seconds; The first detection means starts processing when the start prohibition flag is off. 1. An object detection method comprising:

[0090] (Appendix 4) The arbitration means turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of the second detection means corresponding to one period, and turns off the start prohibition flag at any timing between immediately after the start of the processing time of the second detection means corresponding to the next period and the end of the processing time. 4. The object detection method according to claim 3.

[0091] (Appendix 5) An object detection program installed in a computer to which frames are successively input, The computer, a first detection process for identifying a label of an object shown in an input frame and a position of a bounding box of the object, and predicting the area of ​​the bounding box in a frame input after the input frame based on a history of the positions of the bounding boxes of the same object; a second detection process for identifying the label of an object and the position of the bounding box of the object within the subsequently input frame, the second detection process being performed within the predicted bounding box; and an arbitration process is executed to control the on / off of a start prohibition flag for prohibiting the start of the first detection process, based on 1 / F, PH, and PL, where the input period of a frame is 1 / F seconds, the processing time of the first detection process corresponding to one period is PH seconds, and the processing time of the second detection process corresponding to one period is PL seconds; The computer, When the start prohibition flag is off, the first detection process is started. Object detection program for.

[0092] (Appendix 6) The computer, In the arbitration process, the start prohibition flag is turned on (1 / F-PL-PH) seconds after the end of the processing time of the second detection process corresponding to one cycle, and the start prohibition flag is turned off at any timing between immediately after the start of the processing time of the second detection process corresponding to the next cycle and the end of the processing time. Attachment 5. An object detection program according to claim 5.

[0093] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Industrial Applicability]

[0094] The present disclosure is suitably applied to an object detection system that detects an object captured in a frame. [Explanation of symbols]

[0095] 1. Object detection device 2 First detection unit 3 Second detection unit 4 Pretreatment section 5 Delay measurement section 6 Mediation Department 10 GPU 20, 30, 40 CPU

Claims

1. An object detection device in which frames are input continuously, a first detection means for identifying a label of an object shown in an input frame and a position of a bounding box of the object, and predicting an area of ​​the bounding box in a frame input after the input frame based on a history of positions of the bounding box of the same object; a second detection means for identifying a label of an object shown in the subsequently input frame and a position of the bounding box of the object, with the region of the predicted bounding box as the target; the first detection means and the second detection means share hardware resources; an arbitration means for controlling on / off of a start prohibition flag for prohibiting start of processing by the first detection means, based on 1 / F, PH, and PL, where the input period of a frame is 1 / F seconds, the processing time of the first detection means corresponding to one period is PH seconds, and the processing time of the second detection means corresponding to one period is PL seconds; The first detection means starts processing when the start prohibition flag is off. An object detection device characterized by:

2. The arbitration means turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of the second detection means corresponding to one period, and turns off the start prohibition flag at any timing between immediately after the start of the processing time of the second detection means corresponding to the next period and the end of the processing time. The object detection device according to claim 1 .

3. An object detection method applied to an object detection device to which frames are continuously input, comprising: a first detection means of the object detection device identifies a label of an object shown in an input frame and a position of a bounding box of the object, and predicts an area of ​​the bounding box in a frame input after the input frame based on a history of positions of the bounding box of the same object; a second detection means of the object detection device, which targets the region of the predicted bounding box in the subsequently input frame, identifies a label of the object shown in the frame and identifies a position of the bounding box of the object; an arbitration means of the object detection device controls on / off of a start prohibition flag for prohibiting start of processing by the first detection means, based on 1 / F, PH, and PL, when an input period of a frame is 1 / F seconds, a processing time of the first detection means corresponding to one period is PH seconds, and a processing time of the second detection means corresponding to one period is PL seconds; The first detection means starts processing when the start prohibition flag is off.

1. An object detection method comprising:

4. The arbitration means turns on the start prohibition flag (1 / F-PL-PH) seconds after the end of the processing time of the second detection means corresponding to one period, and turns off the start prohibition flag at any timing between immediately after the start of the processing time of the second detection means corresponding to the next period and the end of the processing time. The object detection method according to claim 3 .

5. An object detection program installed in a computer to which frames are successively input, The computer, a first detection process for identifying a label of an object shown in an input frame and a position of a bounding box of the object, and predicting an area of ​​the bounding box in a frame input after the input frame based on a history of positions of the bounding box of the same object; a second detection process for identifying the label of an object and the position of the bounding box of the object within the subsequently input frame, the second detection process being performed within the predicted bounding box; and an arbitration process is executed to control the on / off of a start prohibition flag for prohibiting the start of the first detection process, based on 1 / F, PH, and PL, where the input period of a frame is 1 / F seconds, the processing time of the first detection process corresponding to one period is PH seconds, and the processing time of the second detection process corresponding to one period is PL seconds; The computer, When the start prohibition flag is off, the first detection process is started. Object detection program for.

6. The computer, In the arbitration process, the start prohibition flag is turned on (1 / F-PL-PH) seconds after the end of the processing time of the second detection process corresponding to one cycle, and the start prohibition flag is turned off at any timing between immediately after the start of the processing time of the second detection process corresponding to the next cycle and the end of the processing time. The object detection program according to claim 5 .

Images