Information processing apparatus, information processing method, and program product

Abstract

The information processing apparatus includes a detection unit that detects a moving body from each frame image of a dynamic image, a calculation unit that calculates a reliability that the detected moving body is a prescribed photographed body, and a decision unit that decides a detection frame of a first moving body detected in a first frame based on a reliability of the first moving body based on a frame that encloses the first moving body and a reliability of the first moving body in the first frame based on a detection frame of a second moving body detected in a second frame before the first frame, and records to a recording unit.

Description

Technical Field

[0001] This invention relates to information processing apparatus, information processing method, and program products. Background Technology

[0002] As a technique for detecting moving objects from dynamic images, there are known methods that extract pixels that are moving within the image as moving object regions by processing the dynamic image using motion difference (inter-frame difference, background difference). Patent Document 1 discloses a technique for distinguishing and identifying the detection object and other moving objects based on physical quantity information such as the detection position in a moving object.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2000-105835 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, the difference region extracted using the moving body difference method can shift due to differences in movement speed or mode. While the difference region obtained using the moving body difference method can keep up with the latest time-to-time changes and output a detection rectangle (detection box), the extracted moving object region can be unstable due to the influence of the accuracy of inter-frame difference or background difference. For example, a human body performing a non-moving task changes its moving parts along with the time-to-time changes, making it difficult to stably output the rectangular size of the moving object.

[0008] In one aspect, the present invention aims to provide a technique for improving the detection accuracy of moving objects in dynamic images and for stably outputting detection boxes.

[0009] Methods for solving problems

[0010] The present invention employs the following structure to achieve the above objectives.

[0011] The information processing apparatus of the first aspect of the present invention includes: a detection unit that detects a moving body based on each frame of a moving image; a calculation unit that calculates the reliability that the detected moving body is a predetermined subject; and a determination unit that determines the detection frame of the first moving body based on the reliability of the first moving body in a frame connected to the first moving body detected in the first frame, and the reliability of the first moving body in the first frame based on the detection frame of the second moving body detected in the second frame preceding the first frame, and records it to a recording unit.

[0012] For a moving object (first moving object) detected in the current frame (first frame), the information processing device determines the detection frame for the first moving object based on the reliability of the detection frame for the moving object (second moving object) detected in the previous frame (second frame). By using a detection frame with higher reliability, the information processing device can improve the detection accuracy of the moving object and output the detection frame stably. The "defined subject" refers to a moving object such as a human body.

[0013] The information processing device may also include a determination unit that determines the subject that is the same as the first moving body among the multiple moving bodies detected in the second frame, i.e., the second moving body. By more accurately determining the moving body that is the same as the first moving body among the moving bodies detected in the second frame, the information processing device can stably output detection frames for the same subject.

[0014] Alternatively, the determination unit can determine the subject being filmed, i.e., the second moving object, based on the center-to-center distance between the frame circumscribed with the first moving object and the detection frames of each moving object detected in the second frame. By determining the subject being filmed, i.e., the second moving object, using a simple method, the information processing device can reduce its processing burden.

[0015] Alternatively, the determination unit can determine the subject being filmed, i.e., the second moving object, based on the proportion of the overlapping area relative to the area occupied by the frame inscribed around the first moving object and the detection frame of each moving object detected in the second frame. By determining the subject being filmed, i.e., the second moving object, which is the same as the first moving object, using a simple method, the information processing device can reduce its processing burden.

[0016] Alternatively, the determination unit uses a machine learning-based matching algorithm to match the first moving object with each moving object detected in the second frame, thereby determining that the subject being filmed is the same as the first moving object, i.e., the second moving object. The information processing device is able to determine the subject being filmed, i.e., the second moving object, with high accuracy.

[0017] Alternatively, the determination unit determines, within each frame, which of the moving bodies detected in the previous frames is the same as the first moving body. If the highest reliability among the detection frames of the first moving body determined to be the same as the first moving body in each frame is greater than the reliability of the first moving body based on frames external to the first moving body, the determination unit determines the detection frame with the highest calculated reliability as the detection frame of the first moving body. By tracing back several frames and using a detection frame with higher reliability, the information processing device improves the reliability of the output detection frame and can output a stable detection frame.

[0018] Alternatively, if the reliability of the first moving body based on the frame circumscribed with the first moving body is greater than a first threshold, the determination unit determines the frame circumscribed with the first moving body as the detection frame of the first moving body. When the reliability based on the circumscribed frame is greater than the first threshold, the information processing device can reduce the processing burden by determining the detection frame without comparing it with the reliability of the detection frame based on the previous frame.

[0019] Alternatively, if the reliability of the first moving body based on the detection frame of the second moving body is greater than the reliability of the first moving body based on the frame externally connected to the first moving body, the decision unit may determine the detection frame of the second moving body as the detection frame of the first moving body. By employing a detection frame with higher reliability, the information processing device can improve the detection accuracy of the moving body.

[0020] Alternatively, if the reliability of the detection frame of the first moving body determined is greater than a second threshold, the determination unit records the detection frame of the first moving body to the recording unit. Since frames with a reliability below the second threshold are not recorded to the recording unit, the information processing device is able to output stable detection frames.

[0021] Alternatively, if the reliability of the first moving body based on the detection frame of the second moving body is greater than the reliability of the first moving body based on the frame external to the first moving body, and if the number of consecutive frames where the difference between the frame external to the first moving body and the detection frame of the second moving body is greater than a third threshold is less than a predetermined number, the determination unit determines the detection frame of the second moving body as the detection frame of the first moving body and records the detection frame of the first moving body to the recording unit. The difference can be, for example, the area change from the detection frame of the second moving body to the frame external to the first moving body, or the proportion of this area change relative to the area of ​​the detection frame of the second moving body. When there are consecutive frames where the difference between the external frame in the current frame and the detection frame in the previous frame is greater than the third threshold, the information processing device can reduce the output of detection frames due to false detections by not recording the detection frame for the first moving body.

[0022] The information processing device may also include an output unit that overlaps the detection frame of the first moving object recorded on the recording unit with the first frame and outputs it. This improves the detection accuracy of moving objects in the dynamic image, enabling the information processing device to output a stable detection frame.

[0023] Alternatively, if the reliability of the detection frame of the first moving body recorded on the recording unit is greater than a second threshold, the output unit outputs the detection frame of the first moving body. The information processing device can reliably output detection frames with a reliability greater than the second threshold.

[0024] Alternatively, if the reliability of the first moving body based on the detection frame of the second moving body is greater than the reliability of the first moving body based on the frame external to the first moving body, and if the number of consecutive frames where the difference between the frame external to the first moving body and the detection frame of the second moving body is greater than a third threshold is less than a predetermined number, the output unit outputs the detection frame of the first moving body recorded in the recording unit. When there are consecutive frames where the difference between the external frame in the current frame and the detection frame in the previous frame is greater than the third threshold, the information processing device can reduce the output of detection frames due to false detections by controlling the output of the detection frame of the first moving body to not be output.

[0025] Alternatively, if the number of consecutive frames where the reliability of the detection frame of the first moving object is greater than a first threshold exceeds a predetermined number, the output unit outputs the detection frame of the first moving object. When the number of consecutive frames where the reliability of the detection frame of the first moving object is greater than the first threshold is known, the information processing device controls the output of the detection frame of the first moving object to continue outputting highly reliable detection frames.

[0026] The information processing device may also include a correction unit that corrects the detection frame of the second moving body based on the position and size changes of the detection frame of the second moving body and the detection frame of the moving body determined to be the same as the first moving body in the frame preceding the second frame. By correcting the detection frame of the moving body detected in the previous frame, the correction unit 125 can improve the reliability of the moving body when the correction frame is applied to the current frame.

[0027] Alternatively, the detection unit may detect moving objects using at least one of the inter-frame difference method and the background difference method. Alternatively, the calculation unit may calculate the reliability that the detected moving object is the specified subject using a recognizer based on at least one of the neural network, boosting algorithm, and support vector machine.

[0028] The information processing method of the second aspect of the present invention enables a computer to include: a detection step of detecting a first moving object from a first frame contained in a moving image; a calculation step of calculating the reliability that the first moving object is a predetermined subject using a frame externally connected to the first moving object and a detection frame recorded in a recording unit, i.e., a detection frame of the second moving object detected in a second frame prior to the first frame; and a determination step of determining the detection frame of the first moving object based on the reliability of the first moving object based on the frame externally connected to the first moving object and the reliability of the first moving object in the first frame based on the detection frame of the second moving object, and recording it in the recording unit.

[0029] This invention can also be understood as a program for implementing the method via a computer, and a recording medium for non-transitory recording of the program. Furthermore, the aforementioned units and processes can be combined with each other as much as possible to constitute this invention.

[0030] Invention Effects

[0031] According to the present invention, the detection accuracy of moving objects in dynamic images can be improved and the detection box can be output stably. Attached Figure Description

[0032] Figure 1 This is a diagram illustrating an application example of the information processing apparatus involved in the implementation method.

[0033] Figure 2 This is a diagram illustrating the hardware structure of an information processing device.

[0034] Figure 3 This is a diagram illustrating the functional structure of an information processing device.

[0035] Figure 4 This is a flowchart illustrating an example of processing the output of a detection rectangle.

[0036] Figure 5 A~ Figure 5 C is a diagram illustrating the method for determining the same subject.

[0037] Figure 6 This is a flowchart illustrating an example of the detection rectangle output processing involved in Implementation Method 2.

[0038] Figure 7 This is a flowchart illustrating an example of the detection rectangle output processing involved in Implementation Method 3.

[0039] Figure 8 This is a flowchart illustrating another example of the detection rectangle output processing involved in Implementation Method 3.

[0040] Figure 9 A and Figure 9 B is a diagram illustrating the application of implementation method 4.

[0041] Figure 10 This is a flowchart illustrating an example of the detection rectangle output processing involved in Implementation Method 4.

[0042] Figure 11 This is a flowchart illustrating another example of the detection rectangle output processing involved in Implementation 4.

[0043] Figure 12 This is a flowchart illustrating an example of the detection rectangle output processing involved in Implementation 5.

[0044] Figure 13This is a flowchart illustrating an example of the detection rectangle output processing involved in Implementation 6.

[0045] Figure 14 This is a diagram illustrating the functional structure of the information processing apparatus according to Embodiment 7.

[0046] Figure 15 This is a diagram illustrating the correction of the detection rectangle involved in Implementation Method 7.

[0047] Figure 16 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 7. Detailed Implementation

[0048] Hereinafter, an embodiment of one aspect of the present invention will be described with reference to the accompanying drawings.

[0049] <Application Example>

[0050] Figure 1 This diagram illustrates an application example of the information processing apparatus according to the embodiment. The information processing apparatus acquires a moving image input from a camera and detects moving objects (hereinafter also referred to as moving bodies) based on each image frame of the acquired moving image. The camera is envisioned, for example, as a fixed camera such as a surveillance camera.

[0051] Information processing devices can extract moving body regions, for example, by background subtraction, which extracts regions that vary between frame images and pre-prepared background images, inter-frame subtraction, which extracts regions that vary between frames, or both. Figure 1 The example illustrates an instance where moving body A1 is extracted at time T. The information processing device generates a rectangle A2 that is circumscribed to the extracted moving body A1. Furthermore, in the description of this application example and the following embodiments, the shape of the frame representing the moving body region is described as a rectangle, but it is not limited to a rectangle; any shape that surrounds the moving body region using an ellipse, polygon, or a curve circumscribed to the moving body region is acceptable.

[0052] Information processing devices obtain the reliability of moving objects, for example, by inputting the detected moving objects into a machine learning recognizer. Figure 1 The reliability in the example is the reliability of human body similarity. The circumscribed rectangle A2 includes the region from which the parts of the human body, excluding the head, are extracted as moving body regions. Therefore, the reliability is 500 when an image of the region enclosed by the circumscribed rectangle A2 is input to the recognizer.

[0053] If the reliability of the object detected in the current frame is below a predetermined threshold, the information processing device uses the detection rectangle of the same subject detected in the previous frame to calculate the reliability of the image cropped from the current frame. The information processing device compares the calculated reliability with the reliability based on the bounding rectangle of the moving object detected in the current frame.

[0054] exist Figure 1 In the example, if the specified threshold is set to 700, then the reliability of the moving body A1 based on the circumscribed rectangle A2 at time T (the current frame) is 500, which is lower than the specified threshold of 700. Therefore, the information processing device uses the detection rectangle A3 for the same subject as the moving body A1 at time T-1 (the previous frame) to calculate the reliability of the image captured in the current frame at time T. The calculated reliability of 1000 is higher than the reliability based on the circumscribed rectangle A2 of the moving body A1 detected at time T.

[0055] If the reliability of the detection rectangle in the previous frame is higher than the reliability of the bounding rectangle in the current frame, the information processing device will determine the detection rectangle of the moving object detected in the previous frame as the detection rectangle of the moving object detected in the current frame. Figure 1 In the example, since the reliability of detection rectangle A3 at time T-1 is 1000, which is higher than the reliability at time T is 500, the information processing device determines detection rectangle A3 as the detection rectangle for the moving body A1 detected in the current frame at time T. By using detection rectangle A3, which has higher reliability, instead of the circumscribed rectangle A2 that surrounds the region that does not contain the human head, the detection accuracy at time T is improved.

[0056] As described above, the information processing device determines the detection rectangle for the moving object based on the reliability of the bounding rectangle of the moving object detected in the current frame and the reliability of the detection rectangle of the same moving object detected in the previous frame. By adopting the rectangle with higher reliability as the detection rectangle, the information processing device can improve the accuracy of moving object detection. Furthermore, even when the moving object stops moving in a dynamic image, or when there is almost no movement of the moving object, the information processing device can output a stable detection rectangle by adopting the detection rectangle of the previous frame. Therefore, even when moving objects are detected using the inter-frame difference method, the detection accuracy of static objects is improved.

[0057] <Implementation Method 1>

[0058] (Hardware structure)

[0059] Reference Figure 2 An example of the hardware structure of the information processing device 1 will be described. Figure 2This is a diagram illustrating the hardware structure of an information processing device 1. The information processing device 1 includes a processor 101, a main storage device 102, an auxiliary storage device 103, a communication interface (I / F) 104, and an output device 105. The processor 101 performs its function as an output device by reading a program stored in the auxiliary storage device 103 into the main storage device 102 and executing it. Figure 3 The functions of each functional structure are described. Communication interface 104 is an interface for wired or wireless communication. Output device 105 is, for example, a display or other device for output.

[0060] The information processing device 1 can be a general-purpose computer such as a personal computer, server computer, tablet terminal, or smartphone, or an embedded computer such as an in-vehicle computer. For example, the information processing device 1 can be implemented through distributed computing using multiple computer devices, or by implementing parts of each functional unit through a cloud server. Furthermore, parts of each functional unit of the information processing device 1 can also be implemented using dedicated hardware devices such as FPGAs or ASICs.

[0061] Information processing device 1 is connected to camera 2 via wired (USB cable, LAN cable, etc.) or wireless (WiFi, etc.) connection to receive image data captured by camera 2. Camera 2 is a camera device having an optical system including lenses and an image sensor (CCD, CMOS, etc.).

[0062] Alternatively, the information processing device 1 can be integrated with the camera 2. Furthermore, part of the processing performed by the information processing device 1, such as motion detection and human identification processing of the captured image, can be executed on the camera 2. Moreover, the results of human detection based on the information processing device 1 can be sent to an external device and displayed to the user.

[0063] (Functional Structure)

[0064] Figure 3 This is a diagram illustrating the functional structure of the information processing apparatus 1. The information processing apparatus 1 includes an image acquisition unit 11, a processing unit 12, a detection rectangle database (DB) 13, and an output unit 14. The processing unit 12 includes a detection unit 121, a calculation unit 122, a determination unit 123, and a decision unit 124.

[0065] The image acquisition unit 11 sends the motion image data acquired from the camera 2 to the processing unit 12. The detection unit 121 of the processing unit 12 detects moving objects for each frame of the motion image received from the image acquisition unit 11. The detection unit 121 can detect moving objects, for example, by background subtraction or inter-frame subtraction.

[0066] The computing unit 122 calculates the reliability of the detected moving object being a specified subject (e.g., a human body). The computing unit 122 can calculate the reliability using neural network algorithms such as CNN (Convolutional Neural Network). Alternatively, the computing unit 122 can also use machine learning-based recognizers such as boosting algorithms or Support Vector Machines (SVM) to calculate the reliability.

[0067] The determination unit 123 determines which of the moving objects detected in the previous frame is the same as the moving object detected in the current frame. Information about the moving objects detected in the previous frame and their corresponding detection rectangles is stored in the detection rectangle database 13. For example, the determination unit 123 determines whether the moving object detected in the current frame is the same as the moving object detected in the previous frame based on the center-to-center distance between the rectangle circumscribed by the moving object detected in the current frame and the detection rectangle of the moving object detected in the previous frame.

[0068] The decision unit 124 determines the detection rectangle for the moving object detected in the current frame based on the reliability calculated by the calculation unit 122 and registers it in the detection rectangle database 13. For example, if the reliability based on the circumscribed rectangle of the moving object detected in the current frame is greater than a predetermined threshold, the decision unit 124 determines that the circumscribed rectangle is the detection rectangle for the moving object and registers it in the detection rectangle database 13.

[0069] Furthermore, if the reliability of the bounding rectangle in the current frame is below a predetermined threshold, the determination unit 124 applies the detection rectangle of the same subject detected in the previous frame to the current frame and calculates the reliability. The determination unit 124 determines the rectangle with higher reliability among the bounding rectangle in the current frame and the detection rectangle of the same subject in the previous frame as the detection rectangle of the moving object detected in the current frame, and registers it in the detection rectangle database 13.

[0070] The detection rectangle database 13 stores the moving objects detected in each frame of the dynamic image and the corresponding detection rectangles determined by the determination unit 124. For example, the detection rectangle database 13 stores information such as the position and size of the detection rectangles within a frame, as information about the detection rectangles. Furthermore, the detection rectangle database 13 may also store the reliability of the moving object based on the detection rectangles, calculated by the calculation unit 122, as information about the detection rectangles. The detection rectangle database 13 is an example of a recording unit.

[0071] The output unit 14 uses the information of the moving body and the corresponding detection rectangle stored in the detection rectangle database 13 to overlay the detection rectangle of the detected moving body onto the image of each frame, and outputs it to the output device 105 such as a display.

[0072] (Detection rectangle output processing)

[0073] Reference Figure 4 The overall process of processing the output of the detection rectangle is explained. Figure 4 This is a flowchart illustrating an example of rectangle detection output processing. Rectangle detection output processing begins, for example, when frames of a moving image acquired by the image acquisition unit 11 are sent to the processing unit. Figure 4 The detection rectangle output processing shown is a process performed for each frame of the dynamic image.

[0074] In S101, the detection unit 121 detects moving objects based on the image of the frame of the processing object (hereinafter referred to as the current frame) received from the image acquisition unit 11. The detection unit 121 can detect moving objects by background subtraction method, which extracts the region that changes between the frame image and the prepared background image, and inter-frame subtraction method, which extracts the region that changes between frames.

[0075] In S102, the detection unit 121 generates a rectangle that is inscribed in each moving body detected in the current frame. For each moving body i (i=1,…,N) detected in the current frame, the processing from S103 to S109 is repeated.

[0076] In S103, the calculation unit 122 calculates the reliability of the image captured from the current frame based on the bounding rectangle generated in S102. The reliability is the reliability that the moving object i within the captured image is a defined subject, such as a person. The calculation unit 122 can calculate the reliability, for example, using algorithms of neural networks such as CNNs, boosting algorithms, or machine learning-based recognizers such as SVMs.

[0077] In S104, the decision unit 124 determines whether the reliability of the circumscribed rectangle calculated in S103 is greater than a predetermined threshold TH1 (first threshold). If the reliability of the circumscribed rectangle is greater than TH1 (S104: Yes), the process proceeds to S109. If the reliability of the circumscribed rectangle is less than TH1 (S104: No), the process proceeds from S105 to the loop process L2 of S108.

[0078] In loop processing L2, the calculation unit 122 uses the motion j of the subject that is the same as the motion i in the current frame from the motion j (j=1,…,M) detected in the previous frame (hereinafter also referred to as the previous frame). mThe reliability of moving body i is calculated using the detection rectangle. The decision unit 124 determines the detection rectangle of moving body i in the current frame based on the calculated reliability and the reliability of the bounding rectangle of moving body i. The processing of each step will be explained in detail below.

[0079] In S105, the determination unit 123 determines whether the subject of the moving body j detected in the previous frame is the same as the subject of the moving body i in the current frame. If it is determined that the subject of the moving body j detected in the previous frame is the same as the subject of the moving body i in the current frame (S106: Yes), the process proceeds to S107. If it is determined that they are different (S106: No), the process proceeds to the loop processing L2 for the detection rectangle of the next moving body j+1.

[0080] Among them, reference Figure 5 A~ Figure 5 C illustrates three examples of methods used in S105 and S106 to determine whether the moving body j in the previous frame and the moving body i in the current frame are the same subject. Alternatively, the three methods illustrated below can be combined, within reasonable limits, to determine whether the subjects are the same.

[0081] Figure 5 A represents the first example of the same determination method. The determination unit 123 determines whether the subject of the motion body j detected in the previous frame is the same as the subject of the motion body i in the current frame, based on the center distance d between the rectangle A512 circumscribed with the motion body i in the current frame and the detection rectangle A511 of the motion body j in the previous frame.

[0082] For example, if the center-to-center distance d is less than a predetermined threshold, the determination unit 123 determines that the subject of the moving body j detected in the previous frame is the same as the subject of the moving body i in the current frame. The predetermined threshold for the center-to-center distance d can be set, for example, to 1 / 2 of the width of the rectangle A512 that is circumscribed by the moving body i in the current frame.

[0083] Figure 5 B represents an example of the second identical determination method. The determination unit 123 determines whether the subject detected in the previous frame of motion body j is the same as the subject detected in the current frame of motion body i based on the IoU (Intersection over Union) of the rectangle A522 circumscribed by motion body i in the current frame and the detection rectangle A521 of motion body j in the previous frame. IoU is the proportion of the overlapping area of ​​the two rectangles relative to the area occupied by the rectangle A522 circumscribed by motion body i in the current frame and the detection rectangle A532 of motion body j in the previous frame.

[0084] For example, if the IoU is greater than a predetermined threshold, the determination unit 123 determines that the subject of the moving body j detected in the previous frame is the same as the subject of the moving body i in the current frame. The predetermined threshold for IoU can be set to, for example, 80%.

[0085] Figure 5 C represents an example of the third identical determination method. The determination unit 123 uses a matching algorithm based on machine learning (Re-Id (Re-Identification)) to match the moving body i in the current frame and the moving body j in the previous frame, thereby determining whether the subject of the moving body j detected in the previous frame is the same as the subject of the moving body i in the current frame.

[0086] exist Figure 5 In example C, relative to the moving body A532 detected at time T, moving body A531 detected at time T-1 and moving body A541 are determined to be the same subject being photographed. Furthermore, relative to the moving body A542 detected at time T, moving body A541 is determined to be the same subject being photographed. By using a machine learning-based matching algorithm, the determination unit 123 can determine the same subjects with high accuracy.

[0087] The determination unit 123, for example, obtains the similarity between the moving object detected in the current frame and multiple moving objects detected in the previous frame. The determination unit 123 can determine that the moving object with the highest similarity among the moving objects with a similarity of 1 or higher (for example, 0.5 if 1 is taken as the maximum value) is the same subject being photographed as the moving object detected in the current frame.

[0088] exist Figure 4 In S107, the calculation unit 122 determines the motion j that is the same as the subject being photographed in the current frame as the motion i. m The reliability of the moving body i captured from the current frame is calculated using the detection rectangle.

[0089] In S108, the decision unit 124 compares the motion j calculated in S107 based on the previous frame. m The reliability of the detection rectangle and the reliability calculated in S103 based on the circumscribed rectangle. The reliability of the circumscribed rectangle of motion body i based on the current frame is higher than that of motion body j based on the previous frame. m In terms of the reliability of the detection rectangle, the decision unit 124 determines that the circumscribed rectangle is the detection rectangle for the moving body i in the current frame. On the other hand, based on the moving body j in the previous frame... m If the reliability of the detection rectangle is higher than that based on the reliability of the circumscribed rectangle, the decision unit 124 determines that the moving body j is from the previous frame. mThe detection rectangle is used as the detection rectangle for the moving body i in the current frame.

[0090] Additionally, there are multiple subjects j that are determined to be the same as the subject i in the current frame, and these subjects j are also present. m In this case, the detection rectangle with the highest reliability calculated in S107 is used and compared with the reliability calculated in S103 based on the circumscribed rectangle.

[0091] In S109, the decision unit 124 records the information of the detection rectangle determined in S108 as the detection rectangle of the moving body i in the current frame into the detection rectangle database 13. The information of the detection rectangle includes the image information of the moving body i, the position and size of the determined detection rectangle, and the reliability value of the moving body i based on the determined detection rectangle.

[0092] In S109, the detection rectangle of motion body i in the current frame, which is recorded in the detection rectangle database 13, is used to calculate the reliability of motion bodies detected in the next frame. For each motion body detected in the current frame, if the loop processing L1 from S103 to S109 ends, the processing proceeds to S110.

[0093] In S110, the output unit 14 overlaps the detection rectangle determined in S108 onto the image of the current frame and outputs it. The detection rectangle output process for the current frame ends.

[0094] (Effects)

[0095] In Embodiment 1 described above, the information processing device 1 compares the reliability of the moving object based on a rectangle circumscribed around the moving object in the current frame with the reliability of the moving object in the current frame based on a detection rectangle of the same subject detected in the previous frame. The information processing device 1 decides to use the rectangle with the higher reliability among the rectangles compared as the detection rectangle for the moving object in the current frame. Since the detection rectangle with higher reliability is used, the information processing device 1 can improve the detection accuracy of the moving object and output the detection rectangle stably.

[0096] Furthermore, if the reliability of the moving object based on its bounding rectangle in the current frame is greater than a predetermined threshold (first threshold), the information processing device 1 records the bounding rectangle as the detection rectangle of the moving object. Since no comparison with the reliability of the detection rectangle based on the previous frame is performed when the reliability is greater than the predetermined threshold, the information processing device 1 can reduce its processing burden.

[0097] In addition, Figure 4In the processing of S105 and S106 of the detection rectangle output process shown, the information processing device 1 determines whether the moving object detected in the current frame is the same subject as the moving object in the previous frame. Figure 5 Compared to the same decision-making method based on machine learning described in C, in Figure 5 The method for determining the similarity using the intercenter distance described in section A, and in... Figure 5 The identity determination method using IoU described in section B can determine whether subjects are the same with less overhead. Furthermore, compared to identity determination methods using intercenter distance or IoU, in... Figure 5 The machine learning-based identity determination method described in C can accurately determine whether the subjects are the same.

[0098] <Implementation Method 2>

[0099] In Embodiment 1, if the reliability of the bounding rectangle detected in the current frame is greater than a predetermined threshold, the information processing device 1 determines the detection rectangle of the detected moving object based on the bounding rectangle in the current frame, without comparing it with the reliability based on the detection rectangle in the previous frame. In contrast, in Embodiment 2, regardless of the reliability based on the bounding rectangle of the moving object detected in the current frame, the information processing device 1 compares the reliability with the detection rectangle of the moving object detected in the previous frame, and determines the rectangle with the higher reliability as the detection rectangle of the moving object detected in the current frame.

[0100] Since the hardware structure and functional structure of the information processing device 1 involved in Embodiment 2 are the same as those in Embodiment 1, the description is omitted. Figure 6 This is a flowchart illustrating an example of the detection rectangle output processing according to Embodiment 2. The detection rectangle output processing according to Embodiment 2 is... Figure 4 The detection rectangle output processing of Embodiment 1 shown differs at points where the determination processing of S104 is absent. For... Figure 4 The detection rectangle output processing shown in Implementation 1 is the same, with the same symbols used and descriptions omitted. Additionally, Figure 6 The detection rectangle output processing involved in Implementation Method 2 can also be achieved through... Figure 4 In the detection rectangle output processing, the threshold TH1 of S104 is set to the maximum value of reliability.

[0101] In Implementation 2, regardless of whether the reliability of the bounding rectangle of the moving body i is greater than the threshold TH1, the determination unit 123 compares the reliability based on the bounding rectangle with the reliability of the moving body i based on the detection rectangle of the moving body j detected in the previous frame. Since the reliability is independent of the bounding rectangle of the moving body i, and a rectangle with higher reliability, including the detection rectangle in the previous frame, is used, the accuracy of the output detection rectangle is improved.

[0102] <Implementation Method 3>

[0103] Embodiment 3 is an embodiment in which the detection rectangle is not output when the reliability of the detection rectangle determined by the determination unit 124 is below a predetermined threshold, and the determined detection rectangle is output when it is above the predetermined threshold. By not outputting the detection rectangle when the reliability is below the predetermined threshold, the information processing device 1 can continue to output detection rectangles with stable reliability.

[0104] Since the hardware structure and functional structure of the information processing 1 involved in Embodiment 3 are the same as those in Embodiment 1, the description is omitted. Figure 7 as well as Figure 8 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 3. (Compared to...) Figure 4 The detection rectangle output processing of Embodiment 1 shown is supplemented by the detection rectangle output processing of Embodiment 3, which adds a determination process (S701, S801) based on whether the reliability of the detection rectangle is greater than a predetermined threshold. For comparison with... Figure 4 The detection rectangle output processing shown in Implementation 1 is the same as that processing, with the same symbols and the description omitted.

[0105] exist Figure 7 Detection rectangle output processing and Figure 8 In the processing of the detection rectangle output, the timing for determining whether the reliability of the detection rectangle is greater than the specified threshold TH2 (second threshold) differs. Figure 7 In step S109, whether the reliability of the detected rectangle is greater than the predetermined threshold TH2 is determined before the detected rectangle information is stored in the detected rectangle database 13. That is, if the reliability of the detected rectangle is below the predetermined threshold TH2, the detected rectangle is neither stored in the detected rectangle database 13 nor output. Conversely, in... Figure 8 In step S110, it is determined whether the reliability of the detection rectangle is greater than the specified threshold TH2. That is, if the reliability of the detection rectangle is below the specified threshold TH2, the detection rectangle is stored in the detection rectangle database 13, but is not output.

[0106] exist Figure 7In the example, if the detection rectangle for moving body i is determined in loop processing L2, the process proceeds to S701. In S701, the determination unit 124 determines whether the reliability based on the determined detection rectangle is greater than a predetermined threshold TH2. The predetermined threshold TH2 can be set to a value below the threshold TH1, for example. If the reliability based on the determined detection rectangle is greater than the predetermined threshold TH2 (S701: Yes), the process proceeds to S109. If the reliability based on the determined detection rectangle is less than the predetermined threshold TH2 (S701: No), the process proceeds to loop processing L1 for the next moving body i+1.

[0107] In S109, information about detection rectangles whose reliability is greater than a predetermined threshold TH2 is stored in the detection rectangle database 13. In S110, for a moving object detected in the current frame, the output unit 14 outputs the detection rectangles stored in the detection rectangle database 13. That is, the output unit 14 outputs the circumscribed rectangle of the moving object i whose reliability is greater than the predetermined threshold TH1 in S104, and the detection rectangles determined to have a reliability greater than the predetermined threshold TH2 in S701. By not outputting rectangles with reliability below the predetermined threshold, the information processing device 1 can continue to output detection rectangles with stable reliability.

[0108] exist Figure 8 In the example, if the detection rectangles of each moving object detected in the current frame are recorded in the detection rectangle database 13 during the loop processing L1, then the processing proceeds to S801. In S801, the output unit 14 determines whether the reliability based on the detection rectangles of each moving object recorded in the detection rectangle database 13 is greater than a predetermined threshold TH2.

[0109] For moving objects with a reliability greater than the specified threshold TH2 (S801: Yes), processing proceeds to S110. For moving objects with a reliability below the specified threshold TH2 (S801: No), the detection rectangle is not output. Figure 8 The detection rectangle output processing for the current frame shown has ended.

[0110] In S110, the output unit 14 outputs the detection rectangles stored in the detection rectangle database 13 that were determined in S801 to have a reliability greater than the predetermined threshold TH2. By not outputting rectangles with a reliability below the predetermined threshold, the information processing device 1 can continue to output detection rectangles with stable reliability.

[0111] <Implementation Method 4>

[0112] Implementation method 4 is an implementation method for eliminating the situation where the detection rectangle for static objects is used as the detection rectangle for moving objects because it has higher reliability than the bounding rectangle of the moving objects in the current frame, and this situation remains. Since the hardware structure and functional structure of the information processing 1 involved in implementation method 4 are the same as those in implementation method 1, the description is omitted.

[0113] If the number of consecutive frames in which the difference between the bounding rectangle of a moving object detected in the current frame and the detection rectangle of a moving object identified as the same subject in the previous frame exceeds a predetermined threshold exceeds a predetermined number, the information processing device 1 prevents the output of the detection rectangle. The difference can be, for example, the change in area from the detection rectangle of the previous frame to the bounding rectangle of the current frame, or the proportion of that area change relative to the area of ​​the detection rectangle of the previous frame. That is, if the number of frames in which the difference between the bounding rectangle of a moving object in the current frame and the detection rectangle of the previous frame exceeds the predetermined threshold is less than a predetermined number, the information processing device 1 records the detection rectangle determined by the determination unit 124 as the detection rectangle of the moving object. Therefore, the information processing device 1 can control the detection rectangle of a static object that is incorrectly used as a detection rectangle of a moving object to not be used in subsequent frames.

[0114] Reference Figure 9 A and Figure 9 B will be used to explain the situation of application implementation method 4. Figure 9 A represents an example of detecting a human body as the detection object from a frame image. Object 902 is an object such as an electric fan that is detected as a moving object. Object 903 is an object that is superimposed on object 902 and may be mistakenly detected as a human body. Object 903 is, for example, an object that is imagined as a robot, a poster of a human body, a coat hanger, a wall pattern, etc., that superimposes on object 902 and may be identified as a human body. Figure 9 Example A illustrates an application of this embodiment. Human body 901 passes by overlapping with object 902 as viewed from the position of camera 2.

[0115] Figure 9 Example B is shown in Figure 9 In case A, the detection results of the moving body from time T-1 to time T+1. Time T is the moment after human body 901 has just passed the position where it overlaps with object 902 as seen from the position of camera 2.

[0116] In the frame image at time T-1, human body 901 is detected around object 902, and detection rectangle A91 is recorded in detection rectangle database 13 as the detection rectangle of human body 901. If object 902 is detected at time T, the determination unit 123 can determine that human body 901 at time T-1 is the same subject as object 902 based on the center distance between the outer rectangle A92 of object 902 and the detection rectangle A91 of human body 901. In this case, calculation unit 122 uses the detection rectangle A91 of human body 901 to calculate the reliability of object 902. The reliability of object 902 based on detection rectangle A91 (the reliability of human body) becomes higher than the reliability based on outer rectangle A92 of object 902 due to the presence of object 903. Therefore, determination unit 124 determines the detection rectangle A91 at time T-1 as the detection rectangle of object 902.

[0117] When object 903 is a static object, even at time T+1, as in the case of time T, the decision unit 124 will determine the detection rectangle A91 at time T-1 and time T as the detection rectangle of object 902. Similarly, after time T+1, the falsely detected detection rectangle A91 is recorded as the detection rectangle of object 902 in the detection rectangle database 13.

[0118] To avoid this situation, if the difference between the outer rectangle in the current frame and the detection rectangle in the previous frame becomes greater than the predetermined threshold TH3 for a predetermined number of consecutive frames, the information processing device 1 controls not to record the detection rectangle A91 in the detection rectangle database 13.

[0119] For example, Figure 9 In example B, the difference can be set as the ratio of the change in area from detection rectangle A91 to the circumscribed rectangle A92 relative to the area of ​​detection rectangle A91 in the previous frame. In this case, if there are more than 5 consecutive frames with a difference greater than 50% of the predetermined threshold TH3, the information processing device 1 can control not to record detection rectangle A91 in the detection rectangle database 13. That is, if the number of consecutive frames with a difference greater than 50% of the predetermined threshold TH3 is less than 5, the information processing device 1 records detection rectangle A91. By controlling whether to record detection rectangles based on the difference between the detection rectangle of the previous frame and the circumscribed rectangle of the current frame, the information processing device 1 can avoid outputting more than a predetermined number of falsely detected detection rectangles consecutively.

[0120] Figure 10 as well as Figure 11 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 4. (Compared to...) Figure 4The detection rectangle output processing of Embodiment 1 shown is supplemented by the detection rectangle output processing of Embodiment 4, which adds a determination process (S1001~S1004, S1101~S1104) for the number of consecutive frames whose rectangle difference is greater than a predetermined threshold. Figure 4 The detection rectangle output processing shown in Implementation 1 is the same as that processing, with the same symbols and the description omitted.

[0121] exist Figure 10 Detection rectangle output processing and Figure 11 In the detection rectangle output processing, the timing for determining whether the number of consecutive frames whose difference in the rectangle is greater than a specified threshold TH3 (third threshold) is greater than a specified number TH4 differs. Figure 10 In step S109, whether the consecutive counts are greater than the predetermined number TH4 is determined before the detection rectangle information is stored in the detection rectangle database 13. That is, if the consecutive counts are below the predetermined number, the detection rectangle is neither stored in the detection rectangle database 13 nor output. Conversely, in... Figure 11 In step S110, before the detection rectangle is output, it is determined whether the number of consecutive counts is greater than a predetermined number TH4. That is, if the number of consecutive counts is less than the predetermined number TH4, the detection rectangle is stored in the detection rectangle database 13, but is not output.

[0122] exist Figure 10 In the example, if the detection rectangle for moving body i is determined in loop processing L2, the process proceeds to S1001. In S1001, the difference between the detection rectangle of the previous frame and the bounding rectangle of the current frame is calculated. The difference in rectangles can be calculated, for example, as the change in area between the bounding rectangle of moving body i and the detection rectangle of moving body i determined in S108. Furthermore, the difference in rectangles, along with the information of the detection rectangles, is recorded in the detection rectangle database 13.

[0123] The decision unit 124 determines whether the difference in the rectangle of the moving body i is greater than a predetermined threshold TH3. If the difference in the rectangle of the moving body i is greater than the predetermined threshold TH3 (S1001: Yes), the process proceeds to S1002. If the difference in the rectangle of the moving body i is less than the predetermined threshold TH3 (S1001: No), the process proceeds to S1003. In S1003, the decision unit 124 initializes the number of consecutive frames F1 in which the change in the difference in the rectangle of the moving body i is greater than the predetermined threshold TH3. The process proceeds to S109, and the detection rectangle corresponding to the moving body i determined in S108 is recorded in the detection rectangle database 13.

[0124] In S1002, the decision unit 124 increments the number F1 of consecutive frames where the difference between the rectangles of the moving body i is greater than the predetermined threshold TH3 by 1. The number F1 of consecutive frames where the difference between the rectangles of the moving body i is greater than the predetermined threshold TH3 is recorded in the detection rectangle database 13 and is referenced in the processing of each frame.

[0125] In S1004, the decision unit 124 determines whether the consecutive number F1 exceeds the predetermined number TH4. If the consecutive number F1 exceeds the predetermined number TH4 (S1004: Yes), the detection rectangle corresponding to the moving body i is not recorded in the detection rectangle database 13, and the process proceeds to the next loop process L1. If the consecutive number F1 is less than or equal to the predetermined number TH4 (S1004: No), the process proceeds to S109, and the detection rectangle corresponding to the moving body i is recorded in the detection rectangle database 13.

[0126] If the number of consecutive frames in which the difference between rectangles becomes greater than a predetermined threshold exceeds a predetermined number, the information processing device 1 can reduce the output of detection rectangles due to false detection by preventing the detection rectangles from being output.

[0127] exist Figure 11 In the example, the processing from S1101 to S1103 is respectively compared with... Figure 10 S1001 and S1003 are the same. After incrementing the consecutive number F1 by 1 in S1102, or after initializing F1 to 0 in S1103, the decision unit 124 records the consecutive number F1 into the detection rectangle database 13. In S109, regardless of the value of the consecutive number F1, the decision unit 124 records the information of the moving body i and the corresponding detection rectangle into the detection rectangle database 13.

[0128] If the detection rectangles of each moving object detected in the current frame are recorded in the detection rectangle database 13, the process proceeds to S1104. In S1104, the output unit 14 determines whether the consecutive number F1 exceeds the predetermined number TH4.

[0129] For motion body i whose consecutive F1 values ​​exceed the specified number TH4 (S1104: Yes), no detection rectangle is output. Figure 11 The detection rectangle output processing for the current frame shown ends. In this case, the output unit 14 initializes the consecutive number F1 of the moving body i recorded in the detection rectangle database 13 to 0. For moving body i whose consecutive number F1 is less than or equal to a predetermined number TH4 (S1104: No), the processing proceeds to S110.

[0130] In S110, the output unit 14 outputs detection rectangles that are not stored in the detection rectangle database 13 and that are determined in S1104 to have a consecutive number F1 below a predetermined number TH4. When the number of consecutive frames in which the difference between rectangles becomes greater than a predetermined threshold exceeds a predetermined number, the information processing device 1 can reduce the output of detection rectangles due to false detection by not outputting detection rectangles.

[0131] <Implementation Method 5>

[0132] Implementation method 5 is an implementation method in which a detection rectangle is output when a predetermined number of frames have been consecutively processed and the reliability becomes greater than a predetermined threshold. By preventing the detection rectangle from being output when the reliability is below the predetermined threshold, the information processing device 1 can continue to output a detection rectangle with stable reliability.

[0133] The hardware and functional structures of the information processing device 1 involved in Embodiment 5 are the same as those in Embodiment 1, so the description is omitted. Figure 12 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 5. (Compared to...) Figure 4 The detection rectangle output processing of Embodiment 1 shown is supplemented by the detection rectangle output processing of Embodiment 5, which adds a determination process (S1201~S1204) for the consecutive number of frames with a reliability greater than a predetermined threshold. For comparison with... Figure 4 The detection rectangle output processing shown in Implementation 1 is the same as that processing, with the same symbols and the description omitted.

[0134] exist Figure 12 In the example, in S104, if the reliability based on the circumscribed rectangle is greater than the specified threshold TH1 (S104: Yes), the process proceeds to S1202.

[0135] In S1202, the decision unit 124 increments the number F2 of consecutive frames with a reliability greater than a predetermined threshold by 1. The number F2 of consecutive frames with a reliability greater than the predetermined threshold is recorded in the detection rectangle database 13 and is referenced in the processing of each frame.

[0136] Furthermore, if in Figure 12 If the detection rectangle for moving body i is determined in loop processing L2, the process proceeds to S1201. In S1201, the determination unit 124 determines whether the reliability of moving body i based on the detection rectangle determined in loop processing L2 is greater than a predetermined threshold TH1. If the reliability based on the determined detection rectangle is greater than the predetermined threshold TH1 (S1201: Yes), the process proceeds to S1202. If the reliability based on the determined detection rectangle is less than TH1 (S1201: No), the process proceeds to S109.

[0137] In S1202, the decision unit 124 increments the number F2 of consecutive frames with a reliability greater than a predetermined threshold by 1. In S109, regardless of the value of the number F2, the information of the moving body i and the corresponding detection rectangle is recorded in the detection rectangle database 13.

[0138] In S1203, since frames that were determined to have a reliability of less than TH1 and a reliability greater than a specified threshold in S1201 become discontinuous, the decision unit 124 initializes the number of consecutive frames F2 for object i to 0.

[0139] If the detection rectangles of each moving object detected in the current frame are recorded in the detection rectangle database 13, the process proceeds to S1204. In S1204, the output unit 14 determines whether the consecutive number F2 exceeds the predetermined number TH5.

[0140] For motion body i whose consecutive F2 counts exceed the specified number TH5 (S1204: Yes), proceed to S110. For motion body i whose consecutive F2 counts are less than the specified number TH5 (S1204: No), do not output the detection rectangle. Figure 12 The detection rectangle output processing for the current frame shown has ended.

[0141] In S110, the output unit 14 outputs the detection rectangles that are not stored in the detection rectangle database 13 and that were determined in S1204 to have a consecutive number F2 exceeding a predetermined number TH5. When the consecutive number F2 is less than or equal to the predetermined number TH5, the information processing device 1 can continue to output highly reliable detection rectangles by preventing the detection rectangles from being output.

[0142] <Implementation Method 6>

[0143] In the embodiments described above, the reliability of the bounding rectangle of the moving object in the current frame is compared with the reliability of the detection rectangle of the same subject detected in the previous frame. In contrast, Embodiment 6 compares the reliability of the bounding rectangle of the moving object in the current frame with the reliability of each detection rectangle of the same subject detected multiple frames prior. In Embodiment 6, the information processing device 1 outputs the rectangle with the higher reliability among the bounding rectangle of the current frame and the detection rectangles from multiple frames prior as the detection rectangle of the moving object in the current frame.

[0144] The hardware and functional structures of the information processing device 1 involved in Embodiment 6 are the same as those in Embodiment 1, so the description is omitted. Figure 13 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 6. (Compared to...) Figure 4The detection rectangle output processing of Embodiment 1 shown is modified by Embodiment 6 by adding a loop processing L3 for backtracking frames. For the detection rectangle output processing of Embodiment 1, the loop processing L3 for backtracking frames is also added. Figure 4 The detection rectangle output processing shown in Implementation 1 is the same as that processing, with the same symbols and the description omitted.

[0145] exist Figure 13 In the example, the loop processing L4 in S105, S106, and S1301 is repeated in the frames up to frame k (k=1, ..., L). The number of backtracking frames L is set to, for example, 5 frames, depending on the processing time and workload. In S1301, ... Figure 4 Similarly, in S107, the calculation unit 122 determines the motion j that is the same as the subject being photographed in the current frame's motion i. m The reliability of the moving body i captured from the current frame is calculated using the detection rectangle.

[0146] In S1302, the decision unit 124 compares the reliability calculated in each frame of the backtracking with the reliability calculated in S103 based on the circumscribed rectangle. The decision unit 124 determines the rectangle with the highest reliability among the compared reliability values ​​as the detection rectangle for the moving body i. Furthermore, the reliability comparison processing in S1302 can also be performed after the reliability calculation in S1301.

[0147] In embodiment 6, the information processing device 1 compares the reliability of the detection rectangle obtained by tracing back multiple frames to the present frame with the reliability based on the bounding rectangle in the present frame. By tracing back multiple frames, not limited to the previous frame, the reliability of the output detection rectangle is improved, and the information processing device 1 is able to output a stable detection rectangle.

[0148] <Implementation Method 7>

[0149] Implementation method 7 is an implementation method that corrects the position and size of the detection rectangle in the previous frame and uses the corrected detection rectangle to calculate the reliability of the moving object detected in the current frame. Even if the detection rectangle in the previous frame is applied to the current frame by moving from the previous frame, the moving object detected in the current frame may not achieve the desired reliability. Therefore, the information processing device 1 improves the reliability based on the detection rectangle in the previous frame by correcting the position or size of the detection rectangle in the previous frame.

[0150] The hardware structure of the information processing device 1 involved in Embodiment 7 is the same as that in Embodiment 1, so the description is omitted. Figure 14 This is a diagram illustrating the functional structure of the information processing apparatus according to Embodiment 7. Figure 3 Based on the functional structure shown in Embodiment 1, the information processing device 1 according to Embodiment 7 includes a correction unit 125. For... Figure 3 For the same functional structure, use the same symbols and omit the explanation.

[0151] The correction unit 125 corrects the detection rectangle of the same subject detected in the previous frame as the moving object detected in the current frame. (Referencing...) Figure 15 The correction of the detection rectangle is explained. Time T, time T-1, and time T-2 are the recording times of the current frame, the previous frame, and the frame before that, respectively. Rectangle A151 is the detection rectangle for the moving object detected in the frame before that. Rectangle A152 is the detection rectangle for the moving object detected in the previous frame. The information of rectangles A151 and A152 is stored in the detection rectangle database 13. Rectangle A153 is the circumscribed rectangle of the moving object detected in the current frame. Figure 15 In the example, the human head is not recognized as a moving body, and rectangle A153 becomes the rectangle that encloses the part excluding the head.

[0152] If rectangle A152 from the previous frame is applied to the current frame without correction, it will result in an offset from the position of the moving object in the current frame due to the movement of the moving object. Therefore, the reliability of rectangle A152 may be lower than that of rectangle A153, which is based on the head not being recognized as a moving object.

[0153] The correction unit 125 merges the position and size of the rectangle A152 in the previous frame into the position of the moving object in the current frame for correction. For example, the correction unit 125 can calculate the estimated values ​​of the width, height, and center coordinates of the rectangle in the current frame based on the changes in the width, height, and center coordinates of the rectangle A152 in the previous frame and the rectangle A151 in the frame before that.

[0154] Specifically, the correction unit 125 can estimate the direction and distance of object movement using the center coordinates of the detection rectangle in the previous frame and the previous frame, and calculate the center coordinates in the current frame. Furthermore, the correction unit 125 can use the average width and height of the detection rectangle in the previous frame and the previous frame as the width and height in the current frame for calculation. The correction unit 125 can generate a correction rectangle A154 based on the calculated estimated values.

[0155] By using the correction rectangle A154 to calculate the reliability of the moving object in the current frame, the information processing device 1 can output a detection rectangle with higher reliability. Furthermore, not limited to the previous frame and the frame before that, the correction rectangle can also be generated based on information from the bounding rectangle of the current frame and detection rectangles obtained back several frames.

[0156] Figure 16 This is a flowchart illustrating an example of the detection rectangle output processing according to Implementation Method 7. (Replace) Figure 4The detection rectangle output processing steps S107 and S108 of Embodiment 1 are shown. Embodiment 7 adds a step of correcting the detection rectangle in the previous frame and calculating reliability based on the corrected rectangle (from S1601 to S1603). For... Figure 4 The detection rectangle output processing shown in Implementation 1 is the same as that processing, with the same symbols and the description omitted.

[0157] exist Figure 16 In the example, in S106, for object j that is determined to be the same as the subject being photographed as the moving body i in the current frame... m The process proceeds to S1601. In S1601, the correction unit 125 performs corrections based on the moving body j. m The position and size changes of the detection rectangle of the subject j and the detection rectangle of the subject j identified as the same moving body i in the previous frame are used to correct the moving body j. m The detection rectangle.

[0158] In S1602, the calculation unit 122 calculates the reliability of the moving body i captured from the current frame using the correction rectangle corrected in S1601. In S1603, the decision unit 124 compares the reliability calculated in S1602 with the reliability based on the circumscribed rectangle calculated in S103. If the reliability based on the circumscribed rectangle of the moving body i in the current frame is higher than the reliability based on the correction rectangle, the decision unit 124 determines the circumscribed rectangle as the detection rectangle for the moving body i in the current frame. On the other hand, if the reliability based on the correction rectangle is higher than the reliability based on the circumscribed rectangle, the decision unit 124 determines the correction rectangle as the detection rectangle for the moving body i in the current frame.

[0159] In embodiment 7, the correction unit 125 corrects the detection rectangle of the moving object detected in the previous frame based on the detection rectangle in the previous frame. By correcting the detection rectangle in the previous frame, the information processing device 1 can improve the reliability of the moving object when applying the correction rectangle to the current frame.

[0160] <Other>

[0161] The above embodiments are merely illustrative examples illustrating the structure of the present invention. The configuration of each embodiment is not limited to the specific methods described above, and can be suitably combined and utilized within the scope of the technical concept of the present invention. Furthermore, the present invention can have various modifications without departing from this technical concept.

[0162] Furthermore, in the embodiments described above, reliability at the level of a human body was described as reliability of an unspecified human body, but it is not limited to this. Reliability can also be the reliability of a specific person being tested.

[0163] Furthermore, in the embodiments described above, examples of sequentially backtracking consecutive frames when tracing back to the previous frame or multiple frames have been given, but this is not a limitation. The information processing device 1 may also backtrack to previous frames at intervals of 2 frames, 3 frames, etc., and output a rectangle with higher reliability as the detection rectangle for the current frame.

[0164] Furthermore, in the embodiments described above, the reliability of the moving object in the current frame is calculated using the detection rectangle of the moving object detected in frames preceding the current frame, but this is not a limitation. In a moving image after capture, the information processing device 1 may also use the bounding rectangle of the moving object detected in frames following the current frame to calculate the reliability of the moving object in the current frame. In this case, if the reliability calculated using the bounding rectangle of the moving object detected in a later frame is greater than the reliability based on the bounding rectangle of the moving object in the current frame, the information processing device can determine the bounding rectangle of the later frame as the detection rectangle in the current frame.

[0165] <Postscript>

[0166] (1) An information processing device (1), comprising:

[0167] The detection unit (121) detects moving objects based on each frame of the dynamic image;

[0168] The calculation unit (122) calculates the reliability that the detected moving body is a specified photographic subject; and

[0169] The decision unit (124) determines the detection frame of the first moving body based on the reliability of the first moving body based on the frame of the first moving body detected in the first frame and the reliability of the first moving body in the first frame based on the detection frame of the second moving body detected in the second frame before the first frame, and records it to the recording unit.

[0170] (2) An information processing method, wherein the computer includes:

[0171] Detection step (S101): Detect the first moving object from the first frame contained in the dynamic image;

[0172] Calculation steps (S103, S107) use a frame connected to the first moving body and a detection frame recorded on the recording unit, i.e., the detection frame of the second moving body detected in the second frame before the first frame, to calculate the reliability that the first moving body is a specified subject; and

[0173] In the decision steps (S108, S109), the detection frame of the first moving body is determined based on the reliability of the first moving body based on the frame connected to the first moving body and the reliability of the first moving body in the first frame based on the detection frame of the second moving body, and recorded to the recording unit.

[0174] Explanation of reference numerals in the attached figures

[0175] 1: Information processing device; 2: Camera; 11: Image acquisition unit; 12: Processing unit; 121: Detection unit; 122: Calculation unit; 123: Judgment unit; 124: Decision unit; 125: Correction unit; 13: Detection rectangle database; 14: Output unit.

Claims

1. An information processing device, comprising: The detection unit detects moving objects based on each frame of the dynamic image; The calculation unit calculates the reliability that the detected moving body is a specified photographic subject; and The decision unit determines the detection frame of the first moving body based on the reliability of the first moving body's frame inscribed with the first moving body detected in the first frame, and the reliability of the first moving body in the first frame based on the detection frame of the second moving body detected in the second frame preceding the first frame, and records it to the recording unit. If the reliability of the first moving body based on the detection frame of the second moving body is greater than the reliability of the first moving body based on the frame externally connected to the first moving body, the decision unit determines the detection frame of the second moving body as the detection frame of the first moving body.

2. The information processing apparatus according to claim 1, wherein, It also includes a determination unit that determines the subject that is the same as the first moving body among the multiple moving bodies detected in the second frame, namely the second moving body.

3. The information processing apparatus according to claim 2, wherein, The determination unit determines the subject being photographed, i.e., the second moving object, based on the distance between the center of the frame circumscribed with the first moving object and the detection frame of each moving object detected in the second frame.

4. The information processing apparatus according to claim 2, wherein, The determination unit determines the subject being photographed, i.e., the second moving object, based on the proportion of the overlapping area relative to the area occupied by the frame inscribed with the first moving object and the detection frame of each moving object detected in the second frame.

5. The information processing apparatus according to claim 2, wherein, The determination unit uses a machine learning-based matching algorithm to match the first moving object with each moving object detected in the second frame, thereby determining that the subject being filmed is the same as the first moving object, i.e., the second moving object.

6. The information processing apparatus according to claim 2, wherein, The determination unit determines in each frame which of the moving bodies detected in the multiple frames preceding the first frame is the same as the first moving body being photographed. If the reliability of the first moving body among the detection frames of the moving body that is determined to be the same as the first moving body in each frame is greater than the reliability of the first moving body based on the frame external to the first moving body, the decision unit will determine the detection frame with the calculated maximum reliability as the detection frame of the first moving body.

7. The information processing apparatus according to claim 1, wherein, If the reliability of the first moving body based on the frame inscribed outside the first moving body is greater than a first threshold, the decision unit determines the frame inscribed outside the first moving body as the detection frame of the first moving body.

8. The information processing apparatus according to claim 1, wherein, If the reliability of the detection frame of the first moving body is greater than the second threshold, the decision unit records the detection frame of the first moving body to the recording unit.

9. The information processing apparatus according to claim 1, wherein, If the reliability of the first motion body based on the detection frame of the second motion body is greater than the reliability of the first motion body based on the frame externally connected to the first motion body, and if the number of consecutive frames in which the difference between the frame externally connected to the first motion body and the detection frame of the second motion body is greater than a third threshold is less than a predetermined number, the determination unit determines the detection frame of the second motion body as the detection frame of the first motion body and records the detection frame of the first motion body to the recording unit.

10. The information processing apparatus according to claim 1, wherein, It also includes an output section that causes the detection frame of the first moving body recorded in the recording section to overlap with the first frame and output it.

11. The information processing apparatus according to claim 10, wherein, If the reliability of the detection frame of the first moving body recorded in the recording unit is greater than a second threshold, the output unit outputs the detection frame of the first moving body.

12. The information processing apparatus according to claim 10, wherein, When the reliability of the first motion body based on the detection frame of the second motion body is greater than the reliability of the first motion body based on the frame externally connected to the first motion body, and when the number of consecutive frames in which the difference between the frame externally connected to the first motion body and the detection frame of the second motion body is greater than a third threshold is less than a predetermined number, the output unit outputs the detection frame of the first motion body recorded to the recording unit.

13. The information processing apparatus according to claim 10, wherein, If the number of consecutive frames in which the reliability of the detection frame of the first moving object is greater than a first threshold exceeds a predetermined number, the output unit outputs the detection frame of the first moving object.

14. The information processing apparatus according to claim 1, wherein, It also includes a correction unit that corrects the detection frame of the second moving body based on the position and size change of the detection frame of the second moving body and the detection frame of the moving body that is determined to be the same as the first moving body in the frame before the second frame.

15. The information processing apparatus according to claim 1, wherein, The detection unit detects moving objects using at least one of the inter-frame difference method and the background difference method.

16. The information processing apparatus according to any one of claims 1 to 15, wherein, The computing unit calculates the reliability that the detected moving object is the specified subject by using a recognizer based on at least one of neural networks, boosting algorithms, and support vector machines.

17. An information processing method, comprising: Executed by computer: The detection step involves detecting a first moving object from the first frame contained in the moving image. The calculation step involves using a frame connected to the first moving body and a detection frame recorded in the recording unit, i.e., the detection frame of the second moving body detected in the second frame before the first frame, to calculate the reliability that the first moving body is a specified subject. as well as The decision-making step involves determining the detection frame of the first moving body based on the reliability of the first moving body within the frame inscribed with the first moving body, and the reliability of the first moving body within the first frame based on the detection frame of the second moving body, and recording this information to the recording unit. The decision step further includes: determining the detection frame of the second motion body as the detection frame of the first motion body when the reliability of the first motion body based on the detection frame of the second motion body is greater than the reliability of the first motion body based on the frame externally connected to the first motion body.

18. A program product for causing a computer to perform the steps of the method according to claim 17.

Images