Object detection device and object detection method

Abstract

This object detecting device detects an object from an image included in a moving image, and comprises: an acquisition unit that acquires an image from a moving image; a number-of-aspects setting unit that sets a number of aspects to be obtained by dividing the image into a plurality of partial aspects using differences between consecutive images; an allocation control unit that allocates a frequency for detecting an object to each partial aspect obtained by the division; a division processing unit that divides the image into a plurality of partial aspects in accordance with the set number of aspects, and detects the object from the partial aspects in accordance with the allocated frequencies; an entirety processing unit that shrinks the image to an entirety aspect showing the entirety of the image, and detects the object from the entirety aspect; and a synthesis processing unit that synthesizes each of the detection results detected from the partial aspects and the entirety aspect, thereby detecting the object from the image.

Description

[Technical Field]

[0001] The disclosed technology relates to an object detection device and an object detection method. [Background technology]

[0002] Technologies have been disclosed that detect objects by detecting metadata, including the position, attributes, and reliability of objects contained in an input image. For example, video processing technologies such as YOLO (You Only Look Once) and SSD (Single Shot Multibox), which use deep learning to detect object metadata, have been disclosed, and their application to surveillance cameras, drone flight control, etc. is being considered. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Joseph Redmon et.al, “YOLOv3: An Incremental Improvement”.<URL:https: / / arxiv.org / abs / 1804.02767> [Non-patent document 2] Wei Liu et.al, “SSD:Single Shot MultiBox Detector”.<URL:https: / / arxiv.org / pdf / 1512.02325.pdf> [Non-patent document 3] H.Uzawa et. al, “High-definition object detection technology based on AI inference scheme and its implementation”, IEICE Electronics Express, 2021, Volume 18, Issue 22, Pages 20210323. [Non-patent document 4] Takayuki Ujiie et al., “Reducing the Load of CNN Object Detection Using Motion Vectors in Video Codecs”, Vol.2018-CVIM-210 No.4 IPSJ Research Report

[0004] Non-Patent Document 1 discloses a method for detecting an object using YOLO from input images of 320×320 pixels, 416×416 pixels, and 608×608 pixels.

[0005] Non-Patent Document 2 discloses a method for detecting objects using SSD.

[0006] Non-Patent Document 3 discloses a method of dividing an input image into multiple images, detecting an object by YOLO using partial planes that show a part of the input image and a whole plane that shows the entire image by reducing the input image, and synthesizing the results of detection from the partial planes and the whole plane to obtain a final object detection result.

[0007] Non-Patent Document 4 discloses a method that makes it possible to thin out frames for performing object detection by predicting the moving position of each object based on a motion vector and correcting the object position. Summary of the Invention [Problem to be solved by the invention]

[0008] Incidentally, when detecting an object from an image using a trained model that has undergone deep learning, the size of the image to be detected is limited. For example, when performing object detection processing on ultra-high definition video such as 4K (3840 × 2160 pixels), detection may be performed using partial surfaces obtained by dividing the input image into multiple parts, or a reduced overall image of the input image. Here, if an input image of ultra-high definition video is divided into partial surfaces of 608 × 608 pixels, object detection processing will be performed on each of 28 partial surfaces, resulting in an enormous amount of processing. Therefore, as a method for reducing the amount of processing, as described above, partial surfaces for detecting an object are thinned out, and the detection results for the thinned partial surfaces are corrected for position by predicting the movement of the object, thereby achieving reduction in the amount of processing.

[0009] However, if the number of partial surfaces on which detection can be performed per frame is small and the total number of partial surfaces is large, the number of times each partial surface is thinned out (detection processing is not performed) also increases. Therefore, if a sudden change occurs in the video, such as a sharp turn of the drone, the movement of the object cannot be accurately predicted, and object tracking performance may deteriorate.

[0010] By reducing the total number of partial surfaces, the number of partial surfaces on which object detection can be performed increases, thereby expanding the range in which objects can be detected in the same frame and improving object tracking performance. On the other hand, reducing the total number of partial surfaces involves image reduction in each partial surface, which leads to a deterioration in object detection performance. In other words, when detecting objects from ultra-high definition video, there is a risk that object tracking performance and object detection performance cannot be achieved at the same time.

[0011] The present disclosure has been made in consideration of these circumstances, and aims to propose an object detection device and an object detection method that can achieve both object tracking performance and object detection performance when detecting objects from ultra-high definition video, etc. [Means for solving the problem]

[0012] A first aspect of the present disclosure is an object detection device that detects an object from an image included in a video, the device including: an acquisition unit that acquires images from the video; a plane number setting unit that sets the number of planes to divide the image into a plurality of partial planes using a difference between consecutive images; an allocation control unit that assigns a frequency of object detection to each divided partial plane; a division processing unit that divides the image into a plurality of partial planes according to the set number of planes and detects objects from the partial planes according to the assigned frequency; an overall processing unit that reduces the image to an overall plane that represents the entire image and detects objects from the overall plane; and a synthesis processing unit that synthesizes the detection results detected from the partial planes and the overall plane to detect objects from the image. an estimation unit that estimates a vector indicating the movement of an object from an image; and a generation unit that generates a predicted image that predicts the current position of the object using the vector; Equipped with The plane number setting unit sets the plane number using the predicted image and the acquired image. It is an object detection device. A second aspect of the present disclosure is an object detection device that detects an object from an image included in a moving image, comprising: an acquisition unit that acquires images from the moving image; a plane number setting unit that sets the number of planes to divide the image into multiple partial planes using the difference between consecutive images; an allocation control unit that assigns a frequency of object detection to each divided partial plane; a segmentation processing unit that divides the image into multiple partial planes according to the set number of planes and detects objects from the partial planes according to the assigned frequency; an overall processing unit that reduces the image to an overall plane that represents the entire image and detects objects from the overall plane; and a synthesis processing unit that synthesizes the partial planes and each detection result detected from the overall plane to detect objects from the image, wherein the allocation control unit sets a period including multiple images, and assigns a frequency in the next period to each partial plane based on the difference between the number of object detections in the current period and the average number of object detections up to the current period.

[0013] The present disclosure 3 The aspect is an object detection method for detecting an object from an image included in a video, which acquires an image from the video, sets the number of planes for dividing the image into a plurality of partial planes using differences between consecutive images, assigns a frequency of object detection to each divided partial plane, divides the image into a plurality of partial planes according to the set number of planes, detects an object from the partial planes according to the assigned frequency, reduces the image to a full plane that represents the entire image, detects an object from the full plane, and combines the detection results from the partial planes and the full plane to detect the object from the image. estimating a vector indicating the movement of the object from the image and generating a predicted image that predicts a current position of the object using the vector, and setting the number of planes includes setting the number of planes using the predicted image and the acquired image. An object detection method. [Effects of the Invention]

[0014] According to the disclosed technology, when detecting an object from an ultra-high definition video or the like, it is possible to achieve both object tracking performance and object detection performance. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of the object detection device according to the present embodiment. [Figure 2] 1 is a block diagram showing an example of a functional configuration of an object detection device 10 according to the present embodiment. [Figure 3]FIG. 10 is a data flow diagram showing an example of a data flow of a detection process to explain object detection according to the present embodiment. [Figure 4] FIG. 10 is a data flow diagram showing an example of the data flow of a page count setting process, which is provided for explaining page count setting according to the embodiment. [Figure 5] 10 is a graph showing an example of time-series data of an average difference value and the number of faces, which is provided for explaining a guard time according to the present embodiment. [Figure 6] FIG. 10 is a data flow diagram showing an example of the data flow of allocation control processing provided for explaining allocation of detection frequencies for each partial surface according to the present embodiment. [Figure 7] 10 is a flowchart illustrating an example of an object detection process according to the present embodiment. [Figure 8] 10 is a flowchart illustrating an example of a page count setting process according to the present embodiment. [Figure 9] 10 is a flowchart illustrating an example of allocation control processing according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0017] First, the hardware configuration of an object detection device 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the object detection device 10 according to this embodiment.

[0018] As shown in Fig. 1, object detection device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other so as to be able to communicate with each other via a bus 18. Note that the above-described configuration using the CPU and memory is merely an example, and the device may be implemented as a device specialized in object detection that includes a dedicated arithmetic circuit, for example.

[0019] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores an object detection processing program for detecting an object from an image.

[0020] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0021] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0022] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0023] The communication interface 17 is an interface for communicating with other devices such as a display device. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used. The communication interface 17 obtains input data from an external memory and transmits output data to the external memory.

[0024] Next, the functional configuration of the object detection device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the object detection device 10 according to this embodiment.

[0025] 2, object detection device 10 has, as its functional configuration, an acquisition unit 21, a division processing unit 22, an overall processing unit 23, a synthesis processing unit 24, a memory unit 25, an estimation unit 26, a generation unit 27, a face count setting unit 28, an allocating unit 29, and an allocation control unit 30. When CPU 11 executes an object detection processing program, it functions as acquisition unit 21, division processing unit 22, overall processing unit 23, synthesis processing unit 24, memory unit 25, estimation unit 26, generation unit 27, face count setting unit 28, allocating unit 29, and allocation control unit 30.

[0026] The acquisition unit 21 acquires an image 32 for each frame from a moving image 31, as shown in FIG.

[0027] The division processing unit 22 divides the acquired image 32 into partial images (hereinafter referred to as "partial planes") 33 according to the set number of planes, and detects an object for each divided partial plane 33 according to the assigned detection frequency. Here, the number of planes is set by a plane number setting unit 28, which will be described later, and the detection frequency is set by an allocation control unit 30. Note that the division processing unit 22 according to this embodiment is a learning model that divides the image 32 into partial planes according to the number of planes, and performs machine learning to detect an object from each partial plane. The division processing unit 22 detects metadata including the position of the object included in each partial plane 33 (the center of the object, the height and width of the area including the object), the attributes of the object, and the reliability of the object.

[0028] As shown in Figure 3, the segmentation processing unit 22 detects metadata of objects contained in each partial surface 33 and outputs metadata with a reliability level equal to or greater than a predetermined level as the detection result (hereinafter referred to as the "segmentation processing result") 34.

[0029] The overall processing unit 23 reduces the acquired image 32, detects object metadata from an image showing the entire image (hereinafter referred to as the "overall surface") 35, and outputs metadata associated with a reliability of a predetermined size or higher as a detection result (hereinafter referred to as the "overall processing result") 36. The overall processing unit 23 according to this embodiment is a learning model that reduces the image 32 to the entire surface and performs machine learning to detect objects from the reduced overall surface.

[0030] The synthesis processing unit 24 synthesizes the division processing result 34 and the overall processing result 36 to detect and output objects from the image 32. Specifically, as shown in FIG. 3, the synthesis processing unit 24 uses the division processing result 34 and the overall processing result 36 to detect corresponding metadata and outputs the metadata as a detection result (hereinafter referred to as a "synthesis processing result") 37. The synthesis processing unit 24 also detects objects (metadata) that are included in the division processing result 34 but not included in the overall processing result 36 and outputs the detection result 37.

[0031] The storage unit 25 stores the acquired image 32 and the segmentation processing result 34. Here, the segmentation processing result 34 is metadata of the object in each partial plane.

[0032] 4, the estimation unit 26 performs a motion search between an image 32 in the current frame acquired by the acquisition unit 21 and a previous image 39 indicating the image of the previous frame stored in the storage unit 25, and estimates a motion vector 38 indicating the movement of an object. The motion search method will be described as using a conventional technique known to those skilled in the art, such as a method of comparing the image 32 in the current frame with the previous image 39. However, the motion search method according to this embodiment is not limited to this.

[0033] As shown in Figure 4, the generation unit 27 generates a predicted image 40 that predicts the position of an object in the current frame using a past image 39 that indicates the image of the previous frame stored in the memory unit 25 and a motion vector 38 estimated by the estimation unit 26.

[0034] As shown in Fig. 4, the plane number setting unit 28 sets the number of planes 41 of the partial planes 33 using the image 32 indicating the current frame and the predicted image 40 generated by the generation unit 27. Specifically, the plane number setting unit 28 derives the absolute difference between the pixel values ​​of each pixel in the image 32 indicating the current frame and the predicted image 40, and derives the sum of the absolute difference values ​​for all pixels (hereinafter referred to as the "sum of absolute differences"). Here, the sum of absolute differences according to this embodiment is expressed by the following formula:

[0035]

number

[0036] Here, diff is the sum of absolute differences for all pixels, N is a frame number that identifies the frame, c is the number of channels in the image, x is the x coordinate within the image, y is the y coordinate within the image, mvx is the x component of the motion vector 38, and mvy is the y component of the motion vector 38.

[0037] That is, the first term of the above-described equation (1) indicates the pixel value of the image related to the current frame, and the second term indicates the pixel value of the predicted image 40 obtained by correcting the image related to the frame immediately preceding the current frame using the motion vector 38. The plane number setting unit 28 derives, for each pixel and channel of the image, the absolute difference between the pixel value of the image related to the current frame and the pixel value of the predicted image 40 corrected by the motion vector 38. The plane number setting unit 28 sums up the absolute difference sums for all pixels and channels to derive the absolute difference sum diff for the current frame.

[0038] The surface number setting unit 28 derives a moving average of the absolute difference sums (hereinafter referred to as the "average difference sum") 42 using the derived absolute difference sum diff for the current frame and the absolute difference sum diff for past frames derived in the past.

[0039] As shown in FIG. 4, the face number setting unit 28 sets the number of faces 41 according to the derived average difference sum 42. Specifically, as shown in FIG. 5 as an example, when the average difference sum 42 exceeds a predetermined threshold, the face number setting unit 28 sets the number of faces 41 corresponding to the threshold. For example, as shown in FIG. 5, when the average difference sum 42 exceeds a predetermined threshold, the face number setting unit 28 changes the number of faces 41 to M2, which is less than M1. Here, the face number setting unit 28 sets a guard time in advance to prevent excessive changes in the number of faces 41, and when a change in the number of faces occurs, the face number setting unit 28 does not change the number of faces 41 until the guard time has elapsed, regardless of whether the average difference sum 42 has exceeded the predetermined threshold. Furthermore, when the guard time has elapsed and the average difference sum 42 is equal to or less than the predetermined threshold, the face number setting unit 28 changes the number of faces 41 to M1 (the initial value).

[0040] That is, when the average difference sum 42 is large (image fluctuation is large), the object detection range is expanded and object tracking performance is improved by reducing the number of faces 41 (increasing each partial face). Also, when the average difference sum 42 is small (image fluctuation is small), the object detection performance is improved by increasing the number of faces 41 (increasing each partial face).

[0041] In the present embodiment, a configuration in which there is one threshold has been described. However, this is not limiting. There may be multiple thresholds. For example, the face number setting unit 28 sets multiple predetermined thresholds, and when the average difference sum 42 exceeds a threshold, it determines the largest of the exceeded thresholds and changes and sets the face number 41 corresponding to the determined threshold. When multiple thresholds are set, a larger threshold is associated with a smaller number of faces.

[0042] In accordance with the changed number of faces 41, the allocating unit 29 allocates the detected objects included in the segmentation processing result 34 stored in the storage unit 25 to the partial faces 33 corresponding to the changed number of faces 41. For example, if the number of faces 41 is changed, the partial faces 33 on which an object is detected in the current frame may not correspond to the partial faces 33 on which an object is detected in the past frame. Therefore, as shown in FIG. 4, when the number of faces 41 is changed, the allocating unit 29 changes the partial faces 33 of the segmentation processing result 34 related to the past frame to the partial faces 33 corresponding to the changed number of faces 41, and allocates the position of the detected object to the changed partial face 33. This makes it possible to compare the partial faces 33 related to the current frame with the partial faces 33 related to the past frame, even when the number of faces 41 is changed.

[0043] 6, the allocation control unit 30 allocates a detection frequency 43 for detecting an object to each partial surface 33 using the division processing results 34 up to the current frame stored in the storage unit 25 and the number of surfaces 41 set by the surface number setting unit 28. The allocation control unit 30 sets a period spanning multiple frames in advance, and allocates a detection frequency 43 for the next period to each partial surface 33 using the division processing results 34 and the number of surfaces 41 in the current period.

[0044] Specifically, the allocation control unit 30 derives a detection number fluctuation value for each period and for each partial surface, and allocates the allocable amount proportionally to each partial surface 33 according to the derived detection number fluctuation value, thereby allocating the detection frequency 43 in the next period for each partial surface.

[0045] Here, the allocable amount is determined by multiplying a predetermined number of partial surfaces on which detection can be performed by the number of frames included in the cycle. For example, if the number of partial surfaces on which detection can be performed per frame is T and the number of frames included in the cycle is R, the allocable amount for that cycle is T × R. Furthermore, if the number of partial surfaces is greater than the number T of partial surfaces on which detection can be performed per frame, the segmentation processing unit 22 narrows down (thins out) the partial surfaces on which object detection is performed. Therefore, in object detection, the segmentation processing unit 22 corrects the segmentation processing result 34 related to the past frame by applying a motion vector 38, and detects the object by determining whether the thinned out partial surfaces in the current frame include an object. This reduces the amount of processing required for detection.

[0046] The detection number fluctuation value according to this embodiment is expressed by the following formula.

[0047]

number

[0048]

number

[0049] Here, f(n) is the detection number fluctuation value, n is a number for identifying a sub-plane, u is a number for identifying a frame included in the cycle, U is the number of frames in the cycle, D is the detection fluctuation value for each sub-plane in each frame, and k is a number for identifying the cycle. Also, d is the number of detected objects, and davg is the average number of detected objects (hereinafter referred to as the "detection average value"). For example, in the above formula (3), d(n, k, u) indicates the number of objects detected in sub-plane n in frame u of the current cycle k. Also, detection average value avg(n, k-1) indicates the detection average value detected in the previous cycle k-1. The detection average value davg detected up to the current cycle is updated every cycle and is obtained by averaging the detection average value davg up to the previous cycle and the average number of objects d detected in the current cycle, and is used in the next cycle.

[0050] In this embodiment, the detection average value davg(n,k) in the current cycle k and in the partial plane n is derived by averaging the detection average value in the current cycle k and the detection average value davg(n,k-1) up to the previous cycle k-1. However, this is not limiting. The davg in the next cycle may be derived by integrating weight values. Specifically, it may be derived as davg=davg(n,k-1)+(1-i)davg(n,k), where i is a forgetting factor.

[0051] The allocation control unit 30 proportionally allocates the allocable amount to each partial surface 33 so that the larger the detection number fluctuation value f(n) of a partial surface, the higher the detection frequency 43 in the next period.

[0052] Next, the operation of the object detection device 10 according to this embodiment will be described with reference to Fig. 7 to Fig. 9. Fig. 7 is a flowchart showing an example of an object detection process according to this embodiment. The object detection program shown in Fig. 7 is executed by the CPU 11 reading and executing an object detection program from the ROM 12 or the storage 14. The object detection program shown in Fig. 7 is executed, for example, when a video 31 is input as input data and an instruction to execute the object detection process is input.

[0053] In step S101, the CPU 11 sets initial values ​​for the number of faces 41 and the detection frequency 43. For example, the number of faces 41 is set to the largest number of faces that can be set as the number of faces, and the detection frequency 43 is set to 1 for each partial face.

[0054] In step S102, the CPU 11 sets the elapsed frames to 1 and the elapsed time to 1 as initial values.

[0055] In step S103, the CPU 11 acquires the image 32 for each frame as input data.

[0056] In step S104, the CPU 11 divides the image 32 into a plurality of partial planes 33 in accordance with the set number of planes 41.

[0057] In step S105, the CPU 11 detects an object from each partial surface 33 in accordance with the set detection frequency 43. Here, the detection result for a partial surface not to be thinned out is metadata obtained by executing object detection, and the detection result for a partial surface 33 to be thinned out is metadata obtained by reading out the division processing result 34 for a past frame stored in the storage unit 25 and correcting the position of the object using the motion vector 38 estimated by the estimation unit 26.

[0058] In step S106, the CPU 11 reduces the image 32 to the entire screen 35.

[0059] In step S107, the CPU 11 detects an object from the reduced overall surface 35.

[0060] In step S108, the CPU 11 uses the division processing result 34 and the overall processing result 36 to synthesize the detection result.

[0061] In step S109, the CPU 11 detects an object from the synthesis processing result 37.

[0062] In step S110, the CPU 11 stores the image 32 and the division processing result 34.

[0063] In step S111, the CPU 11 outputs the synthesis processing result 37.

[0064] In step S112, the CPU 11 adds 1 to the number of elapsed frames and the elapsed time.

[0065] In step S113, the CPU 11 executes a number-of-pages setting process, which will be described in detail later with reference to FIG.

[0066] In step S114, the CPU 11 determines whether the number of elapsed frames has reached a predetermined cycle and the next cycle has begun. If the next cycle has begun (step S114: YES), the CPU 11 proceeds to step S115. On the other hand, if the next cycle has not begun (if the number of elapsed frames has not reached the predetermined cycle) (step S114: NO), the CPU 11 proceeds to step S116.

[0067] In step S115, the CPU 11 executes allocation control processing, which will be described in detail later with reference to FIG.

[0068] In step S116, the CPU 11 determines whether or not there is a next image 32. If there is no next image 32 (step S116: YES), the CPU 11 ends the object detection process. On the other hand, if there is a next image 32 (step S116: NO), the CPU 11 proceeds to step S103.

[0069] Next, the number of pages setting process according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the number of pages setting process according to this embodiment. The number of pages setting program shown in Fig. 8 is executed by the CPU 11 reading and executing a number of pages setting program from the ROM 12 or storage 14. The number of pages setting program shown in Fig. 8 is executed, for example, when an image 32 is input as input data and an instruction to execute the number of pages setting process is input.

[0070] In step S201, the CPU 11 estimates a motion vector 38 using the acquired image 32 and a previous image 39.

[0071] In step S202, the CPU 11 applies the estimated motion vector to a previous image 39 one frame before to generate a predicted image .

[0072] In step S203, the CPU 11 compares the image 32 of the current frame with the predicted image 40 to derive the sum of absolute differences diff.

[0073] In step S204, the CPU 11 derives the average sum of differences 42 using the derived sum of absolute differences diff and the past sum of absolute differences diff.

[0074] In step S205, the CPU 11 determines whether the elapsed time has exceeded the guard time. If the elapsed time has exceeded the guard time (step S205: YES), the CPU 11 proceeds to step S206. On the other hand, if the elapsed time has not exceeded the guard time (step S205: NO), the CPU 11 proceeds to step S210.

[0075] In step S206, the CPU 11 determines whether the average difference sum 42 exceeds a predetermined threshold. If the average difference sum 42 exceeds the predetermined threshold (step S206: YES), the CPU 11 proceeds to step S207. On the other hand, if the average difference sum 42 does not exceed the predetermined threshold (step S206: NO), the CPU 11 proceeds to step S208.

[0076] In step S207, the CPU 11 sets the number of faces corresponding to the threshold value. If multiple threshold values ​​are set, the CPU 11 sets the number of faces corresponding to the largest threshold value among the threshold values ​​exceeded by the average difference sum 42.

[0077] In step S208, the CPU 11 sets an initial value for the number of faces, which is the largest number of faces that can be set as the initial value.

[0078] In step S209, the CPU 11 changes the partial faces 33 of the stored division processing result 34 according to the set number of faces 41, and allocates the detected objects included in the division processing result 34 to the changed partial faces 33, respectively.

[0079] In step S210, the CPU 11 sets the elapsed time to 1.

[0080] Next, the allocation control process according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the allocation control process according to this embodiment. The CPU 11 reads out and executes an allocation control program from the ROM 12 or the storage 14, thereby executing the allocation control program shown in Fig. 9. The allocation control program shown in Fig. 9 is executed, for example, when a predetermined period has elapsed and an instruction to execute the allocation control process has been input.

[0081] In step S301, the CPU 11 derives a detection number fluctuation value f(n) using the number of objects detected for each partial surface and the average value of the number of objects detected in the past period.

[0082] In step S302, the CPU 11 allocates the detection frequency 43 to each partial surface 33 in accordance with the detection number fluctuation value. Here, the detection frequency 43 allocated to each partial surface 33 is allocated by proportionally allocating the allocable amount in the period in accordance with the detection number fluctuation value.

[0083] In step S303, the CPU 11 updates the detection average value davg to be used in the next cycle. Here, the CPU 11 derives the detection average value davg to be used in the next cycle using the detection average value davg up to the past cycles and the average value of the number of objects d detected in the current cycle.

[0084] In step S305, the CPU 11 sets the number of elapsed frames to 1.

[0085] As described above, according to this embodiment, when detecting an object from an ultra-high definition video or the like, it is possible to achieve both object tracking performance and object detection performance.

[0086] In the above embodiment, the detection frequency 43 is allocated by proportionally allocating the allocable amount according to the detection number fluctuation value. However, this is not limiting. For example, the detection frequency 43 of each partial surface 33 may be assigned 1, and the remaining allocable amount (allocable amount - number of surfaces) may be proportionally allocated according to the detection number fluctuation value.

[0087] [Variations] In the above embodiment, the number of faces 41 is changed and set when a predetermined guard time has elapsed. In this modified example, a mode in which the guard time varies will be described.

[0088] For example, the object detection device 10 may count the number of changes in the number of faces 41 that occur within a predetermined period of time, and if the number of changes exceeds a predetermined number, extend the guard time by adding a predetermined time to the guard time. This prevents the number of faces 41 from changing too frequently.

[0089] In each of the above embodiments, the inference processing executed by the CPU after reading the software (program) may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after manufacture, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors having a circuit configuration specifically designed to execute specific processing. The object detection processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0090] In addition, in each of the above embodiments, the object detection processing program is described as being pre-stored (installed) in the storage 14, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0091] The following additional notes are provided regarding the above-described embodiments.

[0092] (Additional note 1) Memory and at least one processor coupled to said memory; Including, The processor: An object detection device that detects an object from an image included in a moving image, acquiring the image from the video; Using the differences between successive images, determine the number of planes by which the image is divided into a plurality of sub-planes; assigning a frequency of detecting the object to each of the divided partial surfaces; Dividing the image into a plurality of partial surfaces according to the set number of surfaces, and detecting an object from the partial surfaces according to the assigned frequency; reducing the image to a full screen showing the entire image, and detecting objects from the full screen; detecting an object from the image by combining the detection results from the partial surface and the entire surface; The object detection device is configured as follows.

[0093] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform an object detection process, The object detection process for detecting an object from an image included in a moving image includes: acquiring the image from the video; Using the differences between successive images, determine the number of planes by which the image is divided into a plurality of sub-planes; assigning a frequency of detecting the object to each of the divided partial surfaces; Dividing the image into a plurality of partial surfaces according to the set number of surfaces, and detecting an object from the partial surfaces according to the assigned frequency; reducing the image to a full screen showing the entire image, and detecting objects from the full screen; detecting an object from the image by combining the detection results from the partial surface and the entire surface; Non-transitory storage medium. [Explanation of symbols]

[0094] 10. Object detection device 21 Acquisition Department 22 Division processing unit 23 Overall processing section 24 Composition processing section 25 Memory section 26 Estimation part 27 Generation part 28 Number of pages setting section 29 Sorting section 30 Allocation control unit

Claims

1. An object detection device that detects an object from an image included in a moving image, an acquisition unit that acquires the image from the video; a surface number setting unit that sets the number of partial surfaces to divide the image using the difference between successive images; an allocation control unit that allocates a frequency of detecting the object to each of the divided partial surfaces; a division processing unit that divides the image into a plurality of partial surfaces in accordance with a set number of surfaces, and detects objects from the partial surfaces in accordance with the assigned frequencies; an overall processing unit that reduces the image to an overall surface that shows the entire image and detects an object from the overall surface; a synthesis processing unit that synthesizes the detection results from the partial surface and the entire surface to detect an object from the image; an estimation unit that estimates a vector indicating a movement of an object from the image; a generation unit that generates a predicted image by predicting a current position of the object using the vector; Equipped with the plane number setting unit sets the plane number using the predicted image and the acquired image. Object detection device.

2. When the sum of absolute differences calculated using the predicted image and the acquired image satisfies a predetermined condition, the face number setting unit changes and sets the face number to a number corresponding to the predetermined condition. The object detection device according to claim 1 .

3. The page count setting unit changes and sets the page count when a predetermined condition is satisfied after a predetermined period has elapsed since the page count was set. The object detection device according to claim 2 .

4. The page count setting unit adds a predetermined time to the predetermined period when the number of times the page count has been changed exceeds a predetermined number of times. The object detection device according to claim 3 .

5. An object detection device that detects an object from an image included in a moving image, an acquisition unit that acquires the image from the video; a surface number setting unit that sets the number of partial surfaces to divide the image using the difference between successive images; an allocation control unit that allocates a frequency of detecting the object to each of the divided partial surfaces; a division processing unit that divides the image into a plurality of partial surfaces in accordance with a set number of surfaces, and detects objects from the partial surfaces in accordance with the assigned frequencies; an overall processing unit that reduces the image to an overall surface that shows the entire image and detects an object from the overall surface; a synthesis processing unit that synthesizes the detection results from the partial surface and the entire surface to detect an object from the image; Equipped with The allocation control unit sets a period including a plurality of images, and allocates the frequency in the next period to each of the partial surfaces based on the difference between the number of detections of an object in the current period and the average number of detections up to the current period. Object detection device.

6. The allocation control unit derives an average value of the number of detections detected up to the current period using an average value of the number of detections in the current period and an average value of the number of detections in past periods. The object detection device according to claim 5 .

7. The allocation control unit derives an average value of the number of detections detected up to the current period by using a value obtained by multiplying the average value of the number of detections in the current period by a weight value. The object detection device according to claim 6 .

8. An object detection method for detecting an object from an image included in a moving image, comprising: acquiring the image from the video; Using the differences between successive images, determine the number of planes by which the image is divided into a plurality of sub-planes; assigning a frequency of detecting the object to each of the divided partial surfaces; Dividing the image into a plurality of partial surfaces according to the set number of surfaces, and detecting an object from the partial surfaces according to the assigned frequency; reducing the image to a full screen showing the entire image, and detecting an object from the full screen; Detecting an object from the image by combining the detection results from the partial surface and the entire surface; Estimating a vector indicating the movement of an object from the image; generating a predicted image by predicting a current position of the object using the vector; setting the number of planes includes setting the number of planes using the predicted image and the acquired image. Object detection methods.

Images