Person tracking method and person tracking device

Abstract

To provide a person tracking method and a person tracking device capable of stably tracking the same person even when a part of the person on a captured image is missing at an image end of the captured image.SOLUTION: A person tracking method for tracking a person in a captured image obtained by imaging a monitoring area from above includes a person detection step of recognizing and detecting a person from the captured image, and a person tracking step of identifying and tracking the detected person from a captured image of the next frame or later. In the person detection step, the person to be tracked is detected by performing whole body recognition of the person through image processing by a vector focus method, and in the person tracking step, the head of the person is tracked in a case where it is predicted that the person to be tracked is not missing in the captured image of the next frame, and the torso or the lower body of the person is tracked in a case where it is predicted that the person to be tracked protrudes out of the captured image at an image end and a part of the person to be tracked is missing in the captured image of the next frame.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a person tracking method and a person tracking device that recognizes and tracks over time a person captured in an image captured from above a surveillance area. [Background technology]

[0002] When recognizing and tracking a person from a captured image of a surveillance area captured by a camera, a method has been developed for recognizing and tracking a person (figure) in the captured image by performing image processing based on data such as the person's image and shape. For example, Patent Document 1 discloses a technology that uses image processing based on a vector focus method to recognize a person in a captured image, detect the person to be tracked, and identify and track the detected person from the next frame of captured images. Image processing based on the vector focus method is disclosed in Patent Documents 2 and 3, for example, and is a technology for recognizing a person by evaluating the concentration of the results of voting processing performed on the captured image in accordance with voting processing data based on the shape of a person in a predetermined human model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7053057 [Patent Document 2] Patent No. 3390426 [Patent Document 3] Patent No. 3406587 Summary of the Invention [Problem to be solved by the invention]

[0004] When processing captured images to recognize and track people, the image range captured by the camera is an important factor. When capturing images of a surveillance area from above with a camera, if the camera is not installed high enough, the person in the captured image will be partially missing from the image at the edges due to the camera's angle of view. For example, as a person approaches the edge of the image from the center, their head will be missing, followed by their upper and lower bodies. Missing a portion of a person at the edge of the captured image can reduce the accuracy of person recognition and lead to errors in tracking the same person. To prevent such tracking errors, one method is to not track people at the edges of the captured image, but this narrows the effective tracking range. On the other hand, using a camera with a variable zoom lens can ensure the required angle of view and enable tracking of people without narrowing the tracking range. However, this method is expensive, increases the physical dimensions of the camera, and complicates image processing settings. Therefore, in practice, it is preferable to use a camera with a small, inexpensive fixed-focus lens. On the other hand, in image processing methods that use stereo images and distance as a basis, if a person's head is missing, the distance cannot be obtained accurately, making it extremely difficult to recognize and track the person. Also, in the case of image processing using AI (artificial intelligence), the learning model is based on recognition when the entire body of the person is displayed, so the accuracy of person recognition decreases when part of the body, such as the head, is missing.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide a person tracking method and person tracking device that make it possible to stably track the same person even when the person in the captured image extends outside the image edge and is missing part of the image. [Means for solving the problem]

[0006] The person tracking method according to the present invention comprises: A person tracking method for recognizing and tracking a person over time in an image captured from above a monitoring area, comprising: a person detection step of recognizing and detecting a person to be tracked from the captured image; a person tracking step of tracking the person detected in the person detecting step by identifying the person from the captured image in a next frame onward, The recognition of the person in the captured image is performed by performing image processing using a vector focus method, which recognizes the person by evaluating a result of voting processing performed on the captured image in accordance with voting processing data based on a vector in a normal direction to an outer contour of a predetermined person model; the person detection step detects a person to be tracked by performing whole-body recognition of the person using image processing based on the vector focus method; The person tracking step is a method of tracking the head of the person to be tracked if it is predicted that there will be no missing part of the person to be tracked in the captured image of the next frame, and tracking the torso or lower body of the person to be tracked if it is predicted that the person to be tracked will extend outside the captured image at the edge of the image in the next frame, causing a missing part of the person to be tracked.

[0007] The person tracking device according to the present invention comprises: A person tracking device that recognizes and tracks over time a person who appears in an image captured by an imaging unit from above a monitoring area, an image input unit to which a captured image captured by the imaging unit is input; an image processing means using a vector focus method to recognize a person by evaluating a result of voting processing of the captured image in accordance with voting processing data based on a vector in a normal direction to an outer contour of a predetermined person model; a person detection unit that processes the captured image input to the image input unit using the image processing means and recognizes and detects a person to be tracked from the captured image; a person tracking unit that performs image processing on the captured image input to the image input unit using the image processing means, and tracks the person detected by the person detection unit by identifying the person from the captured image in a next frame onward, the person detection unit has a person recognition unit that performs whole-body recognition of a person using the image processing means to detect a person to be tracked, The person tracking unit a predicted movement position calculation unit that calculates a predicted movement position to which the person to be tracked in the captured image of the current frame will move and appear in the captured image of the next frame, and predicts whether or not the person to be tracked will protrude outside the captured image at the image edge at the predicted movement position, causing a loss of part of the person to be tracked; a head tracking unit that tracks the head of the person to be tracked when the predicted movement position calculation unit predicts that no missing parts will occur in the person to be tracked; a body tracking unit that tracks a body of the person to be tracked when the predicted movement position calculation unit predicts that a chip will occur in the head of the person to be tracked; The device further includes a lower body tracking unit that tracks the lower body of the person to be tracked when the predicted movement position calculation unit predicts that the head and upper body of the person to be tracked will be missing. [Effects of the Invention]

[0008] According to the present invention, it is possible to stably track the same person even if the person in the captured image is partially missing because the person extends outside the captured image at the edge of the image. Therefore, it is possible to track a person by expanding the tracking area to the edge of the captured image, which was previously difficult to track when a part of the person was missing. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a person tracking device according to an embodiment. [Figure 2] 1 is a schematic diagram for explaining an outline of a method for recognizing a person from a captured image by image processing using a vector focus method; [Figure 3] 10A and 10B are schematic diagrams for explaining a method for predicting a missing part of a person at the edge of a captured image. [Figure 4] 10 is a schematic diagram for explaining positions at which representative points (tracking points) of a person in a captured image are assigned according to portions of the person that are predicted to be missing at the image edge of the captured image. FIG. [Figure 5]10 is a schematic diagram showing positions at which the representative point (tracking point) of a person being tracked is changed when it is predicted that part of the person will be missing at the edge of a captured image. FIG. [Figure 6] 10 is a schematic diagram illustrating an area for performing person identification of a person being tracked when it is predicted that part of the person will be missing at the edge of a captured image. FIG. [Figure 7] 1 is a flowchart illustrating a process flow of a person tracking method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The person tracking device 1 shown in FIG. 1 is an example of a device that recognizes and tracks over time a person captured in an image of a monitoring area captured from above by a camera 10. The person tracking method of this embodiment is executed by this person tracking device 1. The monitoring area can be various places, facilities, areas, etc. where people are present, such as a retail store, a restaurant, a food court, and an office. The camera 10 constitutes an imaging unit that captures images of the monitoring area, and is, for example, a CCD camera or a CMOS camera. The camera 10 is installed on the ceiling or wall of the monitoring area, and one camera 10 can capture images of the monitoring area from above with a field angle of approximately 120 degrees. The camera 10 and the person tracking device 1 are communicatively connected via a wired, wireless, or LAN network. The person tracking device 1 may be configured to read captured image data from a storage device that stores the captured image data of the monitoring area captured by the camera 10, without being connected to the camera 10.

[0011] It is desirable to use a camera with a fixed-focus lens as the camera 10. A camera with a fixed-focus lens has smaller physical dimensions and is less expensive than a camera with a lens with a variable zoom mechanism. On the other hand, when a camera with a fixed-focus lens is used, if the angle of view is narrow (for example, about 120 degrees) and the camera installation height cannot be sufficiently high (for example, about 2500 mm), part of the subject person (for example, the head) may be missing from the edge of the captured image, which may result in an error in tracking the same person. However, the person tracking method of this embodiment, which will be described later, can prevent such tracking errors.

[0012] The person tracking method of this embodiment is a person tracking method that recognizes and tracks over time people who appear in captured images of a monitoring area captured from above by a camera 10, and includes a person detection step (X1) that recognizes and detects a person to be tracked from the captured images of the camera 10, and a person tracking step (X2) that tracks the person detected in the person detection step by identifying the person from the captured images of the next frame onwards (see FIG. 7). In this person tracking method, people in the captured images are recognized by image processing using a vector focus method (Patent Nos. 3390426, 3406587, etc.).

[0013] To explain the image processing of the vector focusing method, referring to FIG. 2(a), a previously created human-shaped standard model (person model) is placed on a captured image (for setting) of the monitoring area. A circular Fourier transform (circular Fourier transform) is performed on each pixel point of the captured image, centered on the processing point, to calculate, for example, the brightness gradient and gradient direction of each pixel. Pixels with a brightness gradient equal to or greater than a certain level are defined as valid pixels. These valid pixels form an area that reflects the standard model. The brightness gradient direction is a vector (normal vector) normal to the outline of the standard model. The normal vector of this gradient direction can be selected, for example, from each of 16 directions obtained by dividing 360 degrees. A collection of selected normal vectors is defined as a vector collection. Next, for the position of the brightness gradient of the valid pixel (the position corresponding to the outline of the standard model), the angle (angle information of the normal vector) and length (position coordinate information of the normal vector) of the gradient direction in the vector collection are calculated between the position of the brightness gradient of the valid pixel (the position corresponding to the outline of the standard model) and a feature point of the standard model (e.g., the center point of the person), and these values ​​are registered as voting processing data based on the standard model. The voting processing data is registered in advance for each standard model that is placed in the entire range of the captured image according to the installation environment of the camera (such as how people appear depending on their positions on the captured image).

[0014] When recognizing a person from a captured image, referring to Figure 2(b), a circular Fourier transform is performed on pixel points in a predetermined area (for example, the area within which the person can move) in the captured image during surveillance to calculate the brightness gradient and gradient direction, and a voting process is performed on feature points for the normal vector representing this brightness gradient and gradient direction in accordance with the voting process data for the standard model.As a result of this voting process, if it is evaluated that the concentration of evaluation points gathered within the predetermined area is above a certain level, the presence of a person matching the standard model can be recognized in the captured image.This is an outline of the person recognition method using image processing using the vector focus method.

[0015] In the person detection step (X1), the whole body of a person is recognized from the captured image by image processing using the vector focus method, and a person to be tracked is detected. When the captured image during surveillance captured by the camera 10 is processed using the vector focus method, the whole body of a person is recognized by performing voting processing according to voting processing data based on the whole body of a previously created standard human model (see Figure 2(a)).

[0016] The person tracking step (X2) recognizes people in the captured images captured by the camera 10 in chronological order during surveillance, using image processing based on the vector focus method, and identifies the person to be tracked (determines the person's identity). A representative point serving as a tracking point is assigned to the person to be tracked, and the person is tracked by linking the representative point. In this embodiment, the person tracking step (X2) tracks the head of the person to be tracked if the entire body of the person to be tracked is captured in the captured image of the next frame and it is predicted that there will be no missing parts of the person. As a result, in captured images captured from above, there is little competition between people's heads, such as overlapping, even in a large group of people. Therefore, if at least the head of a person is captured in the captured image, it is possible to stably track the same person by tracking only the head of the person. On the other hand, the person tracking step (X2) tracks the torso or lower body of the person to be tracked if it is predicted that a part of the person to be tracked (e.g., the head, upper body, etc.) will extend beyond the edge of the captured image in the next frame, resulting in a missing part of the person. This allows the tracking area to be secured up to the edge of the captured image, making it possible to stably track the same person.

[0017] Here, prediction and tracking of missing parts of a person in a captured image can be performed, for example, as follows. With reference to FIG. 3, (1) a predicted movement position of the person (assumed to be the center position of the person in FIG. 3) is calculated based on the person's current position in the captured image and the movement trajectory from the person's past position. (2) A standard human-shaped model (a person model created in advance when setting up the image) is placed at the calculated predicted movement position. (3) Whether the standard model placed at the predicted movement position extends beyond the edge of the image, causing a missing part, and if a missing part occurs, depending on the extent of the missing part, it is determined which part of the person's head, torso, or lower body to track. (4) A movable range is also determined with the predicted movement position as the center. (5) Then, image processing using the vector focus method is performed on the area of ​​the person's part determined in (3) within the movable range, and the person is tracked.

[0018] When assigning a representative point to a person to be tracked, with reference to the captured image shown in FIG. 4, if the entire body is captured in the captured image and there are no missing parts, as in person "a," the representative point is assigned to head center A and the head is tracked. If, as in person "b," head center A approaches the edge of the captured image and part of the head extends outside the captured image, and it is predicted that the head will be missing at the edge of the image, the representative point is changed from head center A to body center B and the body is tracked. Also, if, as in person "c," head center A moves outside the captured image and the entire head extends outside the captured image, and it is predicted that the entire head will be missing at the edge of the image, the representative point is also changed from head center A to body center B and the body is tracked. If, as in person "d," body center B moves outside the captured image and the entire head and part of the upper body extend outside the captured image, and it is predicted that the head and part of the upper body will be missing at the edge of the image, the representative point is changed from body center B to lower body center C and the lower body is tracked. When the representative point of the center C of the lower body of person "e" moves outside the captured image, it is assumed that person e has moved outside the monitoring area, and tracking is terminated.

[0019] When tracking a person in this way, as shown in Figure 5, the position of the representative point is set to the center A of the person's head when the person's entire body is captured in the captured image, but if it is predicted that part or all of the head will be missing at the edge of the image, the representative point is changed to the center B of the person's torso, and if it is predicted that part or all of the torso (part or all of the upper body) will also be missing at the edge of the image, the representative point is changed to the center C of the person's lower body.

[0020] When performing image processing using the vector focus method for person identification during person tracking, with reference to Fig. 6, when tracking a person's head, voting processing is performed on the head region AA of the person to be tracked in accordance with voting processing data based on the head of the person model, as shown in Fig. 6(a), to identify the person. When tracking a person's torso, voting processing is performed on the torso region BB excluding the head of the person to be tracked in accordance with voting processing data based on the torso of the person model, as shown in Fig. 6(b), to identify the person. When tracking a person's lower body, voting processing is performed on the lower body region CC of the person to be tracked in accordance with voting processing data based on the lower body of the person model, as shown in Fig. 6(c), to identify the person.

[0021] For tracking of a person, voting processing of the captured image of the current frame is always performed according to voting processing data obtained using the actual image of the person to be tracked in the captured image of the previous frame as a person model. This makes it possible to identify whether the person to be tracked is the same person even if their shape changes over time, thereby improving tracking accuracy.

[0022] Next, the configuration of the person tracking device 1 of this embodiment will be described. 1 again, person tracking device 1 mainly comprises an image processing unit 2, an image input unit 3, a person detection unit 4, and a person tracking unit 5. Image input unit 3 receives a captured image from camera 10 and outputs the captured image to person detection unit 4 and person tracking unit 5. Image processing unit 2 processes the captured image input to image input unit 3 using a vector focus method.

[0023] The person detection unit 4 performs processing to recognize and detect people that appear in the captured image. The person detection unit 4 has a person recognition unit 41 that detects people from the captured image through image processing using the vector focus method in the image processing unit 2. The person recognition unit 41 detects people by performing whole-body recognition of the person using a standard human-shaped model (a person model created when setting up the image) for the entire captured image. The person recognition unit 41 calculates the recognized coordinate position of the detected person on the captured image (for example, the coordinate position of the head center). The first person detected in the captured image by the person detection unit 4 becomes the person to be tracked.

[0024] The person tracking unit 5 performs a process of tracking a person detected by the person detection unit 4 by assigning a representative point to the person. In the person tracking unit 5, the image processing unit 2 performs image processing using the vector focus method in captured images from the frame following the captured image in which the person detection unit 4 detected the person to be tracked, and tracks the person while identifying the person (determining the person's identity). Here, the person tracking unit 5 always generates voting processing data using an actual image of the person to be tracked from the captured image of the previous frame as a person model, and performs voting processing according to the voting processing data generated based on the actual image of the person in the captured image of the current frame, thereby identifying the person and tracking the person. This person tracking unit 5 has a movement prediction position calculation unit 51, a head tracking unit 52, a torso tracking unit 53, and a lower body tracking unit 54.

[0025] The predicted movement position calculation unit 51 performs a process of calculating a predicted movement position of a tracked person in a captured image of a current frame to which the person will move and appear in a captured image of a next frame. The predicted movement position of the person is calculated based on the person's movement speed and movement direction in consecutive captured images from several frames before the current frame (e.g., eight frames before). The person's movement speed and movement direction are calculated, for example, based on a representative point assigned to the tracked person, and the person's coordinate position, which is the predicted movement position, can be, for example, the coordinate position of the representative point assigned to the tracked person or the center position of the person. After calculating the predicted movement position of the person, the predicted movement position calculation unit 51 virtually places a human-shaped standard model (a human-shaped model created in advance when setting up the image) corresponding to the calculated predicted movement position on the captured image and determines whether a portion of the placed standard model extends outside the captured image, causing a loss in the standard model. Whether or not a portion of the standard model is lost can be predicted based on whether or not a loss in the standard model is present in the actual captured image. The determination of whether or not a part of the standard model is missing can be made from the relationship between the outer periphery of the captured image and that of the placed standard model, and if a part of the standard model is missing, the missing part (head, upper body, lower body, etc.) is recognized. Note that for a person detected for the first time in a captured image by the person detection unit 4, the predicted movement position calculation unit 51 calculates multiple predicted movement positions for the person moving at a predetermined movement speed in multiple assumed movement directions, since there are many predicted movement directions.

[0026] The head tracking unit 52 performs a process of tracking the person's head in a head region of the person within the predicted movement range at the predicted movement position calculated by the predicted movement position calculation unit 51 by performing image processing using the vector focus method in the image processing unit 2. In this image processing, voting processing of the head region is performed in accordance with voting processing data for the head of the person model (the actual image of the person in the previous frame). Note that, when identifying a person, if part or all of the actual image of the person to be tracked cannot be recognized in the captured image one frame before the current frame, such as when a person is detected for the first time in the captured image, the head tracking unit 52 performs voting processing using a standard model as the person model to perform person identification. Below, the torso tracking unit 53 and the lower body tracking unit 54 also perform person identification processing in the same manner as the head tracking unit 52. The torso tracking unit 53 performs a process of tracking the person's torso in a torso region of the person within the predicted movement range at the predicted movement position calculated by the predicted movement position calculation unit 51 by performing image processing using the vector focus method in the image processing unit 2. In this image processing, voting processing is performed in the torso region in accordance with the voting processing data for the torso of the person model (actual image of the person in the previous frame). Lower body tracking unit 54 performs processing to track the lower body of the person by identifying the person in the lower body region of the person within the predicted movement range at the predicted movement position calculated by predicted movement position calculation unit 51 using image processing by the vector focus method in image processing unit 2. In this image processing, voting processing is performed in accordance with the voting processing data for the lower body of the person model (actual image of the person in the previous frame).

[0027] When tracking a person in the person tracking unit 5, if the entire body of the person to be tracked is captured in the captured image and it is predicted that there will be no gaps in the person, the head is tracked by the head tracking unit 52. When it is predicted that part or all of the head of the person to be tracked will extend outside the captured image and there will be gaps in the head, the torso is tracked by the torso tracking unit 53. When it is predicted that part or all of the head and upper body of the person to be tracked will extend outside the captured image and there will be gaps in the head and upper body, the lower body is tracked by the lower body tracking unit 54. Furthermore, when tracking the head with the head tracking unit 52, if the voting process result for the head region is low and the person cannot be identified, and head tracking fails, the person tracking unit 5 switches to tracking with the torso tracking unit 53. When tracking the torso with the torso tracking unit 53, if the voting process result for the torso region is low and the person cannot be identified, and there will be gaps in the torso tracking, the person tracking unit 55 switches to tracking with the lower body tracking unit 54. If it is predicted that the lower half of the person to be tracked will extend outside the captured image and will be missing up to the lower half, or if the voting process result for the lower half area is low when tracking the lower half with the lower body tracking section 54, and the person cannot be identified and tracking of the lower half fails, the person tracking section 5 terminates tracking, assuming that the person to be tracked has moved outside the monitoring area.

[0028] Next, an example of the processing flow of the person tracking method will be described. 7, when the operation of the person tracking device 1 is started, in S1, a captured image captured by the camera 10 is input to the image input unit 3. In S2, the person recognition unit 41 of the person detection unit 4 performs whole-body recognition of the person by image processing using the vector focus method in the image processing unit 2 for the entire captured image input to the image input unit 3, and when the person is recognized, in S3, the recognized coordinate position of the person in the captured image (for example, the center position of the head) is calculated. As a result, the person detection unit 4 detects the person who appears for the first time in the captured image, and this person becomes the person to be tracked.

[0029] When the person detection unit 4 detects a person to be tracked in the captured image, the person tracking unit 5 assigns a representative point to the person to be tracked and tracks them. Here, the representative point is assigned to the center of the person's head. Then, in S4, the predicted movement position calculation unit 51 of the person tracking unit 5 calculates the predicted movement position of the person to be tracked in the captured image of the next frame. The predicted movement position is the coordinate position of the representative point assigned to the person. After calculating the predicted movement position of the person, the predicted movement position calculation unit 51 virtually places a human-shaped standard model corresponding to the predicted movement position on the captured image and determines whether any part of the placed standard model extends beyond the edge of the captured image, causing a loss. In S5, if the predicted movement position calculation unit 51 determines that there is no loss in the head of the standard model, it predicts that there will be no loss in the person to be tracked in the captured image of the next frame, and the process proceeds to S6.

[0030] In S6, the head tracking unit 52 of the person tracking unit 5 assigns a representative point to the head center of the person to be tracked and tracks the person by head tracking. During head tracking, the head tracking unit 52 performs image processing using the vector focus method to identify the person to be tracked based on the head region within a movable range that includes the head region of the person (e.g., a standard human-shaped model placed at the predicted movement position) at the predicted movement position calculated by the predicted movement position calculation unit 51. In S7, if the head tracking unit 52 is able to identify the person to be tracked and head tracking is successful, it assigns a representative point to the head center of the person to be tracked and tracks the person by head tracking. In S14, it outputs the tracking coordinate position of the representative point (head center) of the person to be tracked, and proceeds to S4. If the head tracking unit 52 is unable to identify the person to be tracked and head tracking fails in S7, it proceeds to S9 and tracks the person by the center of the torso. That is, as long as the head of the person to be tracked is not missing from the predicted movement position and head tracking does not fail, the person is tracked by head tracking (S4 → S5 → S6 → S7 → S14).

[0031] On the other hand, if the predicted movement position calculation unit 51 predicts in S5 that the head of the standard model will extend outside the captured image at the predicted movement position, causing a gap in the person's head, then in S8 it determines whether the upper body of the standard model will also extend outside the captured image at the predicted movement position, causing a gap in the person's torso. If the predicted movement position calculation unit 51 predicts in S8 that there will be no gap in the upper body of the standard model, it proceeds to S9.

[0032] In S9, the torso tracking unit 53 of the person tracking unit 5 assigns a representative point to the center of the torso of the person to be tracked and tracks the person by torso tracking. During torso tracking, the torso tracking unit 53 performs image processing using the vector focus method to identify the person to be tracked based on the torso region within a movable range that includes the torso region of the person (e.g., a standard human-shaped model placed at the predicted movement position) at the predicted movement position calculated by the predicted movement position calculation unit 51. In S10, if the torso tracking unit 53 is able to identify the person to be tracked and torso tracking is successful, it continues tracking the person by assigning a representative point to the center of the torso of the person to be tracked and tracks the torso. In S14, it outputs the tracking coordinate position of the representative point (center of the torso) of the person to be tracked, and proceeds to S4. If the torso tracking unit 53 is unable to identify the person to be tracked and torso tracking fails in S10, it proceeds to S12 and tracks the person based on the center of the lower body.

[0033] On the other hand, if the predicted movement position calculation unit 51 predicts in S8 that not only the head but also the upper body of the standard model will protrude outside the captured image at the predicted movement position, causing a cut-off in the torso (upper body) of the person, then in S11 it determines whether or not the lower body of the standard model will also protrude outside the captured image at the predicted movement position, causing a cut-off in the lower body of the person. If the predicted movement position calculation unit 51 predicts in S11 that no cut-off will occur in the lower body of the standard model, it proceeds to S12.

[0034] In S12, lower body tracking unit 54 of person tracking unit 5 assigns a representative point to the center of the lower body of the person to be tracked and tracks the person by lower body tracking. When tracking the lower body, lower body tracking unit 54 performs person identification of the person to be tracked based on the lower body region by image processing using the vector focus method within a movable range including the lower body region of the person (e.g., a standard human-shaped model placed at the predicted movement position) at the predicted movement position calculated by predicted movement position calculation unit 51. In S13, if lower body tracking unit 54 is able to identify the person to be tracked and successfully track the lower body, it continues tracking the person by assigning a representative point to the center of the person to be tracked and tracking the lower body, and in S14, it outputs the tracking coordinate position of the representative point (center of the lower body) of the person to be tracked, and proceeds to S4. In S11, if the predicted movement position calculation unit 51 predicts that a gap will occur in the lower half of the person being tracked, or in S13, if the lower body tracking unit 54 is unable to identify the person being tracked and fails to track the lower half, the person tracking unit 5 determines that the person has moved outside the monitoring area, and proceeds to S15, ending tracking.

[0035] As described above, according to this embodiment, it is possible to stably track the same person even if part of the person in the captured image is missing at the edge of the image. Therefore, it is possible to track a person by expanding the tracking area to the edge of the captured image, which was difficult to do with conventional methods when part of the person was missing.

[0036] Furthermore, even in an environment where the installation height of the camera 10 cannot be sufficiently high, a camera with a fixed-focus lens that has a relatively narrow angle of view can be used, ensuring a tracking area right up to the edge of the captured image and enabling stable tracking of the same person. Therefore, by using a camera with a fixed-focus lens, the person tracking device 1 can be configured inexpensively with small physical dimensions, and image processing settings are not complicated.

[0037] For example, when a camera captures a store's interior from above as a surveillance area, the store's entrance may be positioned near the edge of the captured image. In this case, with conventional people tracking methods, if a person's head is missing at the edge of the captured image, the accuracy of person recognition and person identification decreases. Therefore, if a person exits a store at the store's entrance, which is at the edge of the image, and another person enters from outside the store immediately after that, the person attempting to exit may be mistracked as having turned back inside the store. Such tracking errors make it difficult for people counting devices equipped with people tracking devices to accurately count the number of people entering and exiting the store and record the movement paths of people.

[0038] In this embodiment, even if a part of a person is missing at the edge of the captured image, it is possible to stably track the same person. Therefore, even if a person exiting and a person entering a store entrance located at the edge of the image intersect, it is possible to continue tracking that person until they exit the store entrance. It is also possible to accurately track a person who newly enters a store through an entrance from the moment they start entering. Therefore, the people counting device can accurately count the number of people entering and exiting, and record the movement routes of people.

[0039] (Variation) In the person tracking method of the above embodiment, a person is detected and tracked from the point when the person's entire body enters the captured image. As a variation of the above embodiment, when a tracking target person is detected in the captured image by whole-body recognition in the person detection step (X1), a person tracking step (X2) is performed while tracing back to captured images in previous frames, thereby obtaining the movement path of the tracking target person from the point when the person entered the captured image. That is, the person tracking step (X2) is performed on captured images rewound from the captured image in the current frame in which the person's entire body was detected in the person detection step (X1) to the point in time when the person entered the captured image. In this rewound captured image, the person whose entire body appears to be captured in the captured image approaches the edge of the image, with part of the person missing, and appears to move out of the captured image and disappear. Therefore, by performing the person tracking step (X2) on this rewound captured image, the time when the person began to enter the captured image, their position, and their movement path can be obtained.

[0040] The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims. In the person tracking step, when the head of a person is tracked in the captured image using the center of the head as a representative point, even if the lower half of the person's body goes out of the captured image first at the edge of the captured image and a part of the person's image is missing, the head is tracked by tracking the person without changing the representative point of the head center. For example, when the field of view of the camera is directed diagonally downward to capture an image of a monitoring area, the lower half of the person's body will be missing first at the edge of the image directly below the camera, but when the head is being tracked, the head is tracked by tracking the person without changing the representative point of the head center. [Explanation of symbols]

[0041] 1. Person tracking device 2 Image processing section (vector focus method) 3 Image input section 4. Person detection section 5. Person Tracking Department 10 Camera (imaging unit) 40 Person recognition section 51 Movement prediction position calculation unit 52 Head Tracking Unit 53 Fuselage Tracking Unit 54 Lower body tracking unit X1 Person Detection Step X2 Person Tracking Step

Claims

1. A person tracking method for recognizing and tracking a person over time in an image captured from above a monitoring area, comprising: a person detection step of recognizing and detecting a person to be tracked from the captured image; a person tracking step of tracking the person detected in the person detecting step by identifying the person from the captured image in a next frame onward, The recognition of the person in the captured image is performed by performing image processing using a vector focus method, which recognizes the person by evaluating a result of voting processing performed on the captured image in accordance with voting processing data based on a vector in a normal direction to an outer contour of a predetermined person model; the person detection step detects a person to be tracked by performing whole-body recognition of the person using image processing based on the vector focus method; The person tracking method includes tracking the head of the person to be tracked when it is predicted that the person to be tracked will not be missing in the captured image of the next frame, and tracking the torso or lower body of the person to be tracked when it is predicted that the person to be tracked will extend outside the captured image at the edge of the image in the next frame and that a part of the person to be tracked will be missing.

2. The person tracking method according to claim 1, The person tracking step includes: Identifying the person in the head region of the person to be tracked and tracking the person by assigning a representative point for tracking to the head of the person; When it is predicted that the head of the person to be tracked will be missing at the edge of the captured image, person identification is performed in the body region of the person, and a representative point for tracking is assigned to the body of the person and the person is tracked; A person tracking method, in which, when it is predicted that the head and upper body of the person to be tracked will be missing at the image edge of the captured image, person identification is performed in the lower body region of the person, and a representative point for tracking is assigned to the lower body of the person and the person is tracked.

3. 3. The person tracking method according to claim 1, When the person to be tracked is detected by whole-body recognition in the person detection step, the person tracking step is executed by going back to captured images of previous frames to obtain the movement path of the person to be tracked from the time the person entered the captured image.

4. 3. The person tracking method according to claim 1, This person tracking method places a human-shaped standard model at a predicted position where a person to be tracked moves in a captured image of a current frame and appears on the captured image of a next frame, and predicts that a part of the person to be tracked will be missing in the captured image of the next frame when a part of the placed standard model extends outside the captured image.

5. 3. The person tracking method according to claim 1, The person tracking step is a person tracking method in which, when performing image processing using the vector focus method, an actual image of the person to be tracked in the captured image of the previous frame is used as the person model, and voting processing is performed in accordance with voting processing data based on the person model of the actual image to identify the person.

6. A person tracking device that recognizes and tracks over time a person who appears in an image captured by an imaging unit from above a monitoring area, an image input unit to which a captured image captured by the imaging unit is input; an image processing means using a vector focus method to recognize a person by evaluating a result of voting processing of the captured image in accordance with voting processing data based on a vector in a normal direction to an outer contour of a predetermined person model; a person detection unit that processes the captured image input to the image input unit using the image processing means and recognizes and detects a person to be tracked from the captured image; a person tracking unit that performs image processing on the captured image input to the image input unit using the image processing means, and tracks the person detected by the person detection unit by identifying the person from the captured image in a next frame onward, the person detection unit has a person recognition unit that performs whole-body recognition of a person using the image processing means to detect a person to be tracked, The person tracking unit a predicted movement position calculation unit that calculates a predicted movement position to which the person to be tracked in the captured image of the current frame will move and appear in the captured image of the next frame, and predicts whether or not the person to be tracked will protrude outside the captured image at the image edge at the predicted movement position, causing a loss of part of the person to be tracked; a head tracking unit that tracks the head of the person to be tracked when the predicted movement position calculation unit predicts that no missing parts will occur in the person to be tracked; a body tracking unit that tracks a body of the person to be tracked when the predicted movement position calculation unit predicts that a chip will occur in the head of the person to be tracked; a lower body tracking unit that tracks the lower body of the person to be tracked when the predicted movement position calculation unit predicts that the head and upper body of the person to be tracked will be missing.

Images