Video information processing apparatus, person tracking system, video information processing method, and program

Abstract

To provide a video information processing apparatus capable of improving the possibility of continuing tracking when a feature amount of a person being tracked is changed.SOLUTION: A video information processing device for tracking a target person in a camera video includes a person detection processing unit that detects a person from the camera video, a feature quantity calculation processing unit that calculates a feature quantity of the person, a feature quantity comparison processing unit that determines, as a target person, a person whose difference between the feature quantity of the person calculated by the feature quantity calculation processing unit and a reference feature quantity set as the feature quantity of the target person is within a reference range among persons detected by the person detection processing unit, and a feature quantity resetting processing unit that sets, as the reference feature quantity, a feature quantity of a person whose target behavior is detected among the persons detected by the person detection processing unit when there is no feature quantity whose difference is within the reference range among the feature quantities calculated by the feature quantity calculation processing unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a video information processing device, a person tracking system, a video information processing method, and a program. [Background technology]

[0002] Conventionally, there are human tracking systems that track suspicious persons and the like in images captured by surveillance cameras, etc. Patent Document 1 discloses a moving object tracking device that makes it difficult for erroneous correspondence to occur even when the appearance of a moving object changes. The moving object tracking device of Patent Document 1 determines the viewpoint from the imaging means to the moving object at each time based on the position and moving direction of the moving object at that time, and tracks the moving object by comparing, from among features extracted in the past, a feature corresponding to the viewpoint at the current time with the feature of the moving object at the current time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-223434 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the moving object tracking device described in Patent Document 1 has a problem in that when tracking a person, if the feature values ​​of the person being tracked change due to, for example, the person taking off or putting on a jacket, tracking may not be possible.

[0005] The present disclosure has been made in consideration of the above circumstances, and provides a video information processing device, a person tracking system, a video information processing method, and a program that can increase the possibility of continuing tracking when the features of a person being tracked change compared to conventional methods. [Means for solving the problem]

[0006] This disclosure has been made to solve the above-mentioned problems, and one aspect of the present disclosure is a video information processing device that tracks a target person in camera footage, comprising: a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates features of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, a person, among the people detected by the person detection processing unit, whose feature calculated by the feature calculation processing unit has a difference between the feature of the person calculated by the feature calculation processing unit and a reference feature that is set as the feature of the target person, within a standard range; and a feature resetting processing unit that, if there is no feature calculated by the feature calculation processing unit whose difference is within the standard range, sets, among the people detected by the person detection processing unit, the feature of a person whose target behavior has been detected as the reference feature.

[0007] Another aspect of the present disclosure is the above-described video information processing device, further comprising: a target behavior detection processing unit that detects the target behavior, wherein the target behavior is looking around.

[0008] Another aspect of the present disclosure is the above-described video information processing device, further comprising a target behavior detection processing unit that detects the target behavior, wherein the target behavior is running.

[0009] Another aspect of the present disclosure is the above-described video information processing device, further comprising a target behavior detection processing unit that detects the target behavior, wherein the target behavior is carrying an object.

[0010] Furthermore, another aspect of the present disclosure is the above-mentioned video information processing device, which includes a target behavior detection processing unit that detects the target behavior, and a trajectory prediction processing unit that predicts trajectories of the person detected by the person detection processing unit, and when the target behavior is detected for multiple people by the target behavior detection processing unit, the feature resetting processing unit selects, from the feature amounts of the multiple people, the feature amount of a person to be set in the feature comparison processing unit as the reference feature amount, based on the trajectories predicted by the trajectory prediction processing unit.

[0011] Another aspect of the present disclosure is a person tracking system that tracks a target person in camera footage, the person tracking system including: a camera that captures an image of a target area and acquires the camera footage; a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates feature amounts of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, any person among the people detected by the person detection processing unit, for which a difference between the feature amount of the person calculated by the feature calculation processing unit and a reference feature amount that is set as the feature amount of the target person, is within a standard range; a feature resetting processing unit that sets, if there is no feature amount calculated by the feature comparison processing unit, the feature amount of a person detected by the person detection processing unit whose target behavior has been detected, as the reference feature amount; an image synthesis processing unit that synthesizes an image showing the target person into the camera footage; and an image display unit that displays the camera footage with the image synthesized by the image synthesis processing unit.

[0012] Another aspect of the present disclosure is a video information processing method for tracking a target person in camera footage, the video information processing method including: a first step of detecting a person from the camera footage; a second step of calculating features of the person detected in the first step; a third step of determining, as the target person, any person whose feature calculated in the second step has a difference within a standard range from a reference feature set as the target person; and a fourth step of setting, as the reference feature, the feature of a person detected in the first step whose target behavior has been detected, if the difference between the feature calculated in the second step is not within the standard range.

[0013] Another aspect of the present disclosure is a program for causing a computer to function as a video information processing device that tracks a target person in camera footage, the video information processing device including: a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates feature quantities of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, a person among the people detected by the person detection processing unit, if a difference between the feature quantity of the person calculated by the feature calculation processing unit and a reference feature quantity that is set as the feature quantity of the target person is within a standard range; and a feature resetting processing unit that, if there is no feature quantity calculated by the feature calculation processing unit whose difference is within the standard range, sets, among the people detected by the person detection processing unit, the feature quantity of a person whose target behavior has been detected as the reference feature quantity. [Effects of the Invention]

[0014] According to this disclosure, the video information processing device, person tracking system, video information processing method, and program can increase the possibility of continuing tracking when the feature amount of the person being tracked changes compared to conventional methods. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a schematic block diagram showing the configuration of a person tracking system 1 according to an embodiment of the present disclosure. [Figure 2] 1 is a schematic block diagram showing the configurations of a camera 10 and a video information processing device 20 in the embodiment. [Figure 3] 2 is a schematic diagram illustrating an example of the operation of the video information processing device 20 according to the embodiment. FIG. [Figure 4] 10 is a schematic block diagram showing a first configuration example of a target behavior detection processing unit 207 in the embodiment. FIG. [Figure 5] FIG. 10 is a schematic block diagram showing a second configuration example of the target behavior detection processing unit 207 in the embodiment. [Figure 6] FIG. 10 is a schematic block diagram showing a third configuration example of the target behavior detection processing unit 207 in the embodiment. [Figure 7] 1 is a diagram illustrating a hardware configuration of an apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a person tracking system 1 according to an embodiment of the present disclosure. The person tracking system 1 includes a camera 10, an image information processing device 20, and an image display device 30 (image display unit). The person tracking system 1 tracks a target person in camera images captured by cameras 10 installed in power plants, factories, commercial buildings, office buildings, amusement facilities, schools, etc., and displays the camera images on the image display device 30 in such a way that the target person can be identified, for example, by encircling the target person with a rectangle or highlighting the target person. The target person may be, for example, a suspicious person, or a person to be monitored, such as a lost child, a wheelchair user, or a white cane user.

[0017] The camera 10 and the video information processing device 20 are communicably connected via an IP (Internet Protocol) network, etc. The video information processing device 20 and the video display device 30 are connected via a means for transmitting video signals, such as an HDMI (registered trademark) (High-Definition Multimedia Interface) cable.

[0018] The camera 10 is, for example, a surveillance camera installed on a ceiling or a wall, and transmits video information V obtained by capturing and encoding the video (camera video) using video compression coding such as HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding) to the video information processing device 20. The camera 10 may be a single camera or multiple cameras. When the camera 10 is multiple cameras, the video information V may be a composite (joined together) of the videos captured by the multiple cameras, or may be an encoded version of each of the videos captured by the multiple cameras.

[0019] The video information processing device 20 receives video information V from the camera 10 and tracks a target person in the video information V based on the feature values ​​of the target person. Multiple feature values ​​may be used for this tracking. The feature values ​​may be, for example, based on color, such as a color distribution (histogram), or based on shape, such as an aspect ratio, or a combination of these. The video information processing device 20 superimposes an image of the target person being tracked onto the video information V and displays it on the video display device 30. The video information processing device 20 determines, as the target person, a person whose feature value is within a reference range and whose reference feature value is set as the target person among the people detected in the video information V. If there is no person whose feature value is within the reference range and whose reference feature value is set as the reference feature value, the video information processing device 20 sets, as the reference feature value, the feature value of the person whose target behavior is detected among the detected people. The video information processing device 20 may be implemented by one or more computers loading and executing a program.

[0020] The video display device 30 includes a display means such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays video in accordance with the video signal input from the video information processing device 20.

[0021] 2 is a schematic block diagram showing the configurations of the camera 10 and the video information processing device 20 in this embodiment. The camera 10 includes a video information acquisition processing unit 101. The video information processing device 20 includes a video information reception processing unit 201, an image data generation processing unit 202, a display screen generation processing unit 203, a person tracking processing unit 204, a person identity detection processing unit 205, a trajectory prediction processing unit 206, a target behavior detection processing unit 207, a feature resetting processing unit 208, and a video synthesis processing unit 209.

[0022] In addition, when the camera 10 is a plurality of cameras and the video information V is obtained by encoding each of the videos captured by the plurality of cameras, the video information processing device 20 may include a video information reception processing unit 201, an image data generation processing unit 202, a display screen generation processing unit 203, a person tracking processing unit 204, a person identity detection processing unit 205, a trajectory prediction processing unit 206, a target behavior detection processing unit 207, a feature resetting processing unit 208, and a video synthesis processing unit 209 for each of the videos captured by the plurality of cameras. In this case, the reference features may be common to the videos captured by the plurality of cameras or may be different. Alternatively, the video information processing device 20 may synthesize (splice) the videos captured by the plurality of cameras into a single video and then process the video.

[0023] The video information acquisition processing unit 101 captures an image of a target area and acquires camera video. The video information acquisition processing unit 101 encodes the camera video using a moving image compression code to generate video information V, and transmits the video information V to the video information processing device 20. The video information V is a stream of encoded camera video.

[0024] The video information reception processing unit 201 receives the video information V transmitted by the video information acquisition processing unit 101 . The image data generation processing unit 202 decodes the video information V received by the video information reception processing unit 201 to generate a camera video, and extracts image data of each frame that constitutes the camera video.

[0025] The display screen generation processing unit 203 uses the image data extracted by the image data generation processing unit 202 to add a display frame, a title, operation buttons, etc. to the image based on the image data, thereby generating a display screen.

[0026] The person tracking processing unit 204 detects a target person from the image data extracted by the image data generation processing unit 202. The person tracking processing unit 204 includes a person detection processing unit 241, a feature amount calculation processing unit 242, and a feature amount comparison processing unit 243.

[0027] The person detection processing unit 241 detects people from the image data extracted by the image data generation processing unit 202. A known image recognition technique can be used to detect people. The feature amount calculation processing unit 242 calculates the feature amount of each person detected by the person detection processing unit 241.

[0028] The feature comparison processing unit 243 determines, as a target person, a person among those detected by the person detection processing unit 241, for which the difference between the feature of the person calculated by the feature calculation processing unit 242 and the reference feature set as the feature of the target person is within a reference range. Here, regarding the difference and the reference range, if the feature is a scalar, the difference may be the absolute value of the difference between the feature and the reference feature, and the reference range may be a difference equal to or less than a threshold. Alternatively, if the feature is composed of multiple values ​​and can be regarded as a vector, the difference may be the Euclidean distance between the feature and the reference feature, and the reference range may be a difference equal to or less than a threshold, or the difference may be the absolute value of the difference between each feature value and the corresponding value of the reference feature, and the reference range may be a difference equal to or less than a corresponding threshold.

[0029] The feature of the target person that is initially set as the reference feature may be the feature of a person specified by an operator who views the camera image, or the feature of a person for whom a specific behavior or a specific behavior and feature has been detected. If the target person is suspicious, the specific behavior may be frequently looking around, suddenly starting to run, brandishing a dangerous object, etc. If the target person is lost, the specific behavior and feature may be running around and having a height below a threshold.

[0030] The trajectory prediction processing unit 206 predicts the trajectory (destination) of each person detected by the person detection processing unit 241. The trajectory prediction processing unit 206 may predict the trajectory using the direction of the person detected by the person detection processing unit 241, or may predict the trajectory from a past trajectory based on the feature amount of each person calculated by the feature amount calculation processing unit 242.

[0031] The person identity detection processing unit 205 detects the identity between the person detected by the person detection processing unit 241 in the immediately previous frame (or multiple frames) and the person detected by the person detection processing unit 241 in the current frame. The person identity detection processing unit 205 may use the feature amount calculated by the feature amount calculation processing unit 242 and the trajectory predicted by the trajectory prediction processing unit 206 to detect this identity.

[0032] The target behavior detection processing unit 207 detects the target behavior of each person detected by the person detection processing unit 241. Note that the target behavior detection processing unit 207 may detect the target behavior when the feature amounts calculated by the feature amount calculation processing unit 242 do not have a difference within a reference range as a result of comparison by the feature amount comparison processing unit 243. For example, if the target person to be tracked is a suspicious person, the target behavior may be looking around, running, carrying a target object (dangerous substance), etc., but it may also be other than these. Furthermore, for example, if the target person to be tracked is a person to be watched, the target behavior may be looking around, running, standing still, etc., but it may also be other than these.

[0033] The target behavior detection processing unit 207 may detect that a person whose identity is detected across multiple frames by the person identity detection processing unit 205 is performing the target behavior using images of the multiple frames of the person. The target behavior detection processing unit 207 may also determine the person whose target behavior is to be detected based on the trajectory predicted by the trajectory prediction processing unit 206. For example, when there is no feature calculated by the feature calculation processing unit 242 whose difference falls within a reference range, the target behavior detection processing unit 207 may predict the position of the target person in the current frame based on the trajectory of the target person predicted by the trajectory prediction processing unit 206 in past frames, and limit the person whose target behavior is to be detected to the periphery of that position. Here, the periphery may be within a predetermined distance from the position or within a distance according to the movement speed of the target person. This limits the people whose target behavior is to be detected, thereby reducing the processing load of the target behavior detection processing unit 207.

[0034] If the feature amounts calculated by the feature amount calculation processing unit 242 do not have a difference within a reference range, the feature amount resetting processing unit 208 sets, as reference feature amounts, the feature amount of a person whose target behavior has been detected by the target behavior detection processing unit 207, among the people detected by the person detection processing unit 241, in the feature amount comparison processing unit 243. Note that the feature amount calculated by the feature amount calculation processing unit 242 for the person whose target behavior has been detected can be used as the reference feature amount. Furthermore, if the target behavior has been detected for multiple people, the feature amount resetting processing unit 208 may select, from the feature amounts of the multiple people, feature amounts of the people to be set as reference feature amounts in the feature amount comparison processing unit 243, based on the trajectories predicted by the trajectory prediction processing unit 206. For example, the feature amount resetting processing unit 208 may predict the position of the target person in the current frame based on the trajectory of the target person predicted by the trajectory prediction processing unit 206 in past frames, and set, as reference feature amounts, the feature amount of a person closest to the position, among the multiple people whose target behavior has been detected, in the feature amount comparison processing unit 243. This makes it possible to select a person whose feature is to be set in the feature comparison processing unit 243 as the reference feature, even if the target behavior is detected for a plurality of people.

[0035] The video synthesis processing unit 209 superimposes an image showing the target person detected by the person tracking processing unit 204 on the display screen generated by the display screen generation processing unit 203, and causes the display screen with the superimposed image to be displayed on the video display device 30. Here, the image showing the target person may be, for example, a rectangle surrounding the target person or a speech bubble attached to the target person.

[0036] FIG. 3 is a schematic diagram illustrating an example of the operation of the video information processing device 20 in this embodiment. In FIG. 3, symbols N to N+6 are images of the target person in frames N to N+6. Reference R1 is the feature of the target person initially set as the reference feature, and reference R2 is the feature of the target person set as the reference feature by the feature resetting processing unit 208. In the example of operation in FIG. 3, the video information processing device 20 stores the feature of the image of the person identified as the target person to be tracked in frame N as the reference feature (reference R1). The video information processing device 20 continues tracking the target person from frames N+1 to N+3 by referring to this reference R1.

[0037] However, after frame N+3, the target person's features change as the target person puts on or takes off a jacket, etc., and the target person can no longer be detected if the difference in features from reference R1 falls within the reference range. In other words, the target person cannot be tracked using features. Then, the video information processing device 20 stores the feature of the person in frame N+4, where target behavior detection D was performed, as reference feature (reference R2). Thereafter, the video information processing device 20 continues tracking the target person in frames N+5, N6, etc., by referring to this reference R2.

[0038] In this way, when it becomes impossible to track the target person using the feature amounts due to a change in the feature amounts of the target person, that is, when there is no person detected by the person detection processing unit 241 whose difference from the reference feature amounts falls within a reference range, the video information processing device 20 sets the feature amounts of the person whose target behavior has been detected as the reference feature amounts. As a result, even when it becomes impossible to track the target person using the feature amounts, the video information processing device 20 can continue tracking the target person as long as the target person is performing the target behavior.

[0039] 4 is a schematic block diagram showing a first configuration example of the target behavior detection processing unit 207 in this embodiment. The first configuration example of the target behavior detection processing unit 207 is a configuration example when the target behavior is looking around. In FIG. 4, the target behavior detection processing unit 207 includes a face part recognition unit 271a, a part direction recognition unit 272a, an angle swing width calculation unit 273a, and a target behavior determination unit 274a.

[0040] The facial part recognition unit 271a recognizes facial parts (eyes, nose, etc.) of the image in each frame of a person whose identity has been detected across multiple frames by the person identity detection processing unit 205. This recognition can be performed using a known image recognition technique. The facial feature direction recognition unit 272a recognizes the direction of the facial features recognized by the facial feature recognition unit 271a.

[0041] The angle variation range calculation unit 273a calculates the angle variation range of the entire face based on the change in the direction of the features recognized by the feature direction recognition unit 272a. The target behavior determination unit 274a determines whether or not the person is looking around based on the swing amplitude within the certain time period calculated by the angle swing amplitude calculation unit 273a. The target behavior determination unit 274a may determine that the person is looking around if, for example, the swing amplitude within the certain time period is within a determination range consisting of a lower limit value and an upper limit value. Note that the target behavior determination unit 274a may make the determination using not only the swing amplitude but also the number of swings.

[0042] 5 is a schematic block diagram showing a second configuration example of the target behavior detection processing unit 207 in this embodiment. The second configuration example of the target behavior detection processing unit 207 is a configuration example when the target behavior is running. In FIG. 5, the target behavior detection processing unit 207 includes a trajectory recording processing unit 271b, a movement speed calculation processing unit 272b, and a target behavior determination unit 273b.

[0043] The trajectory recording processing unit 271b records the trajectory of a person whose identity has been detected across multiple frames by the person identity detection processing unit 205. The trajectory recorded here is coordinates indicating the position of the person in each frame. These coordinates may be coordinates within the image, or may be coordinates obtained by converting the coordinates within the image into real space.

[0044] The moving speed calculation processing unit 272b calculates the moving speed (speed) of the person whose trajectory is recorded at a plurality of times based on the trajectory (coordinates) recorded by the trajectory recording processing unit 271b. The target behavior determination unit 273b determines whether the person whose trajectory is recorded has started running, based on the moving speed calculated by the moving speed calculation processing unit 272b. For example, if the increase in moving speed within the determination time length exceeds a threshold, the target behavior determination unit 273b determines that the person has started running.

[0045] 6 is a schematic block diagram showing a third example configuration of the target behavior detection processing unit 207 in this embodiment. The third example configuration of the target behavior detection processing unit 207 is a configuration example when the target behavior is carrying an object. In FIG. 6, the target behavior detection processing unit 207 includes a body part recognition unit 271c, a foreign object carrying determination unit 272c, and a target behavior determination unit 273c.

[0046] The body part recognition unit 271c recognizes the body parts of a person whose identity has been detected across multiple frames by the person identity detection processing unit 205. A known image recognition technique can be used for this recognition. Note that the body part recognition unit 271c may also recognize the body parts of a person detected by the person detection processing unit 241. In this case, the video information processing device 20 does not need to include the person identity detection processing unit 205.

[0047] Foreign object possession determining section 272c determines that an object that is not a human body part in the image of the person whose body parts have been recognized by body part recognition section 271c is a foreign object possessed by the person. The target behavior determination unit 273c determines whether the foreign object determined by the foreign object possession determination unit 272c is a target object based on the shape of the foreign object. The target object may be set by an operator of the video information processing device 20. For example, if the target person is a suspicious person, the target object may be a bladed object, a tool, or the like, or if the target person is a person to be watched, a wheelchair.

[0048] The target behavior detection processing unit 207 may include more than one of the first to third configuration examples shown in Fig. 4 to Fig. 6. In this case, the target behavior detection processing unit 207 may determine that the target behavior has been detected when any one of the target behavior determination units 274a, 273b, and 273c determines that the target behavior is a target behavior, or may determine that the target behavior has been detected when a predetermined number of the target behavior determination units 274a, 273b, and 273c determine that the target behavior is a target behavior.

[0049] FIG. 7 is a diagram illustrating the hardware configuration of the device according to this embodiment. Each device is a video information processing device 20. Each device includes an input / output module I, a storage module M, and a control module P. The input / output module I is implemented by including some or all of the communication module H11, connection module H12, pointing device H21, keyboard H22, display H23, button H3, microphone H41, speaker H42, camera H51, or sensor H52. The storage module M is implemented by including a drive H7. The storage module M may further be configured by including some or all of memory H8. The control module P is implemented by including memory H8 and a processor H9. These hardware components are connected to each other so as to be able to communicate with each other via a bus, and are supplied with power from a power supply H6.

[0050] The connection module H12 is a digital input / output port such as a USB (Universal Serial Bus). The pointing device H21, keyboard H22, and display H23 may be touch panels. The sensor H52 may be an acceleration sensor, a gyro sensor, a GPS receiver module, a proximity sensor, or the like. The power supply H6 is a power supply unit that supplies the electricity necessary to operate each device. The power supply H6 may be a battery. The drive H7 is an auxiliary storage medium such as a hard disk drive or a solid-state drive. The drive H7 may be a non-volatile memory such as an EEPROM or a flash memory, or a magneto-optical disk drive or a flexible disk drive. The drive H7 is not limited to being built into each device, but may also be an external storage device connected to the connector of the connection module H12. The memory H8 is a main storage medium such as a random access memory. The memory H8 may be a cache memory. The memory H8 stores instructions when executed by one or more processors H9. The processor H9 is a CPU (Central Processing Unit). The processor H9 may be an MPU (microprocessing unit) or a GPU (graphics processing unit). The processor H9 reads programs and various data from the drive H7 via the memory H8 and performs calculations to execute instructions stored in one or more memories H8.

[0051] The input / output module I is used in the video information processing device 20, etc. The control module P is used to implement each part of the video information processing device 20. Note that in this specification, etc., the term "video information processing device 20" may be replaced with the term "control module P."

[0052] The present disclosure may be embodied as follows. (1) One embodiment of the present disclosure is a video information processing device that tracks a target person in camera footage, the video information processing device including: a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates feature quantities of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, a person among the people detected by the person detection processing unit, if the difference between the feature quantity of the person calculated by the feature calculation processing unit and a reference feature quantity set as the feature quantity of the target person is within a standard range; and a feature resetting processing unit that, if the difference between the feature quantities calculated by the feature calculation processing unit is not within the standard range, sets, as the reference feature quantity, the feature quantity of a person detected by the person detection processing unit whose target behavior has been detected.

[0053] (2) One embodiment of the present disclosure is the video information processing device according to (1), further comprising a target behavior detection processing unit that detects the target behavior, wherein the target behavior is looking around.

[0054] (3) One embodiment of the present disclosure is the video information processing device according to (1), further comprising a target behavior detection processing unit that detects the target behavior, wherein the target behavior is running.

[0055] (4) One embodiment of the present disclosure is a video information processing device as described in (1), which includes a target behavior detection processing unit that detects the target behavior, and the target behavior is holding an object.

[0056] (5) One embodiment of the present disclosure is a video information processing device according to any one of (1) to (4), comprising a target behavior detection processing unit that detects the target behavior, and a trajectory prediction processing unit that predicts the trajectory of the person detected by the person detection processing unit, and when the target behavior is detected by the target behavior detection processing unit for multiple people, the feature resetting processing unit selects, from the features of the multiple people, the feature of a person to be set as the reference feature in the feature comparison processing unit based on the trajectories predicted by the trajectory prediction processing unit.

[0057] (6) One embodiment of the present disclosure is a person tracking system that tracks a target person in camera footage, the person tracking system including: a camera that captures an image of a target area and acquires the camera footage; a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates features of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, a person among the people detected by the person detection processing unit, if a difference between the feature of the person calculated by the feature calculation processing unit and a reference feature that is set as the feature of the target person is within a standard range; a feature resetting processing unit that sets, if there is no feature calculated by the feature comparison processing unit, the feature of a person detected by the person detection processing unit whose target behavior has been detected as the reference feature; an image synthesis processing unit that synthesizes an image showing the target person into the camera footage; and an image display unit that displays the camera footage with the image synthesized by the image synthesis processing unit.

[0058] (7) One embodiment of the present disclosure is a video information processing method for tracking a target person in camera footage, the video information processing method including: a first step of detecting a person from the camera footage; a second step of calculating features of the person detected in the first step; a third step of determining, as the target person, any person whose feature calculated in the second step has a difference within a standard range from a reference feature set as the target person; and a fourth step of setting, as the reference feature, the feature of a person detected in the first step whose target behavior has been detected, if the difference between the feature calculated in the second step is not within the standard range.

[0059] (8) One embodiment of the present disclosure is a program for causing a computer to function as a video information processing device that tracks a target person in camera footage, the video information processing device including: a person detection processing unit that detects a person from the camera footage; a feature calculation processing unit that calculates features of the person detected by the person detection processing unit; a feature comparison processing unit that determines, as the target person, a person among the people detected by the person detection processing unit, if the difference between the feature of the person calculated by the feature calculation processing unit and a reference feature that is set as the feature of the target person is within a standard range; and a feature resetting processing unit that, if there is no feature calculated by the feature calculation processing unit whose difference is within the standard range, sets, among the people detected by the person detection processing unit, the feature of a person whose target behavior has been detected as the reference feature.

[0060] 1 and 2 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to realize the video information processing device 20. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0061] "Computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording media" also includes devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over networks like the Internet or over communication lines like telephone lines, and devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients. The programs may also be programs that implement some of the aforementioned functions, or may be programs that can realize the aforementioned functions in combination with programs already stored in the computer system.

[0062] The embodiments of this disclosure have been described in detail above with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of this disclosure. [Explanation of symbols]

[0063] 1. People Tracking System 10 Camera 20. Video information processing device 30 Video display device 101 Video information acquisition processing unit 201 Video information receiving processing unit 202 Image data generation processing unit 203 Display screen generation processing unit 204 Person Tracking Processing Unit 205 Person identity detection processing unit 206 Trajectory prediction processing unit 207 Target behavior detection processing unit 208 Feature resetting processing unit 209 Video composition processing unit 241 Person detection processing unit 242 Feature calculation processing unit 243 Feature comparison processing unit 271a Facial parts recognition unit 272a Part direction recognition unit 273a Angle swing width calculation unit 274a Target behavior determination unit 271b Trajectory recording processing unit 272b Moving speed calculation processing unit 273b Target behavior determination unit 271c Body Part Recognition Unit 272c Foreign object possession determination department 273c Target Behavior Determination Unit

Claims

1. A video information processing device that tracks a target person in a camera video, a person detection processing unit that detects a person from within the camera image; a feature amount calculation processing unit that calculates feature amounts of the person detected by the person detection processing unit; a feature amount comparison processing unit that determines, among the people detected by the person detection processing unit, a person whose difference between the feature amount of the person calculated by the feature amount calculation processing unit and a reference feature amount set as the feature amount of the target person is within a reference range, as the target person; a feature resetting processing unit that sets, when the difference does not fall within the reference range among the feature calculated by the feature calculation processing unit, a feature of a person whose target behavior has been detected among the people detected by the person detection processing unit as the reference feature; A video information processing device comprising:

2. a target behavior detection processing unit that detects the target behavior, The image information processing device according to claim 1 , wherein the target behavior is looking around.

3. a target behavior detection processing unit that detects the target behavior, The image information processing device according to claim 1 , wherein the target behavior is starting to run.

4. a target behavior detection processing unit that detects the target behavior, The image information processing apparatus according to claim 1 , wherein the target behavior is carrying an object.

5. a target behavior detection processing unit that detects the target behavior; a trajectory prediction processing unit that predicts a trajectory of the person detected by the person detection processing unit; Equipped with when the target behavior detection processing unit detects the target behavior of a plurality of persons, the feature amount resetting processing unit selects, from the feature amounts of the plurality of persons, a feature amount of a person to be set in the feature amount comparison processing unit as the reference feature amount, based on the trajectories predicted by the trajectory prediction processing unit. The video information processing device according to claim 1 .

6. A person tracking system that tracks a target person in a camera image, a camera that captures an image of a target area and acquires the camera image; a person detection processing unit that detects a person from within the camera image; a feature amount calculation processing unit that calculates feature amounts of the person detected by the person detection processing unit; a feature amount comparison processing unit that determines, among the people detected by the person detection processing unit, a person whose difference between the feature amount of the person calculated by the feature amount calculation processing unit and a reference feature amount set as the feature amount of the target person is within a reference range, as the target person; a feature resetting processing unit that, when the difference between the feature amounts calculated by the feature comparison processing unit does not fall within the reference range, sets, as the reference feature amount, a feature amount of a person whose target behavior has been detected among the people detected by the person detection processing unit; an image synthesis processing unit that synthesizes an image showing the target person with the camera image; a video display unit that displays the camera video combined with the image by the video composition processing unit; A person tracking system comprising:

7. 1. A video information processing method for tracking a target person in a camera video, comprising: a first step of detecting a person from within the camera image; a second step of calculating feature amounts of the person detected in the first step; a third step of determining, among the persons detected in the first step, a person whose difference between the feature amount of the person calculated in the second step and a reference feature amount set as the feature amount of the target person is within a standard range as the target person; a fourth step of setting, as the reference feature, a feature of a person whose target behavior has been detected among the people detected in the first step, if the difference among the feature calculated in the second step is not within the reference range; A video information processing method comprising:

8. Computer, Image information processing device for tracking a target person in camera images A program for functioning as The video information processing device includes: a person detection processing unit that detects a person from within the camera image; a feature amount calculation processing unit that calculates feature amounts of the person detected by the person detection processing unit; a feature amount comparison processing unit that determines, among the people detected by the person detection processing unit, a person whose difference between the feature amount of the person calculated by the feature amount calculation processing unit and a reference feature amount set as the feature amount of the target person is within a reference range, as the target person; a feature resetting processing unit that sets, when the difference does not fall within the reference range among the feature calculated by the feature calculation processing unit, a feature of a person whose target behavior has been detected among the people detected by the person detection processing unit as the reference feature; A program that includes:

Images