Personal location detection system

Abstract

It detects the location of individuals. [Solution] The imaging units 11a to 11c capture images. The personal identification unit 25 analyzes the images captured by the imaging units 11a to 11c to identify the individuals depicted in the images. The position estimation unit 21 analyzes the images captured by the imaging units 11a to 11c to estimate the positions of the people depicted in the images. The position tracking unit 22 tracks the movement of the people based on the position estimated by the position estimation unit 21. The personal tracking unit 26 tracks the movement of the individuals by linking the individuals identified by the personal identification unit 25 with the individuals tracked by the position tracking unit 22.

Description

Technical Field

[0001] The present invention relates to a system for detecting the position of an individual.

Background Art

[0002] Patent Document 1 discloses a person detection system for determining whether a person is included in a captured image. Patent Document 2 discloses a living body determination device that detects a face from a captured image and performs face authentication when the detected face is a living body.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique described in Patent Document 1 can detect the number of people within the detection range and their positions. However, it cannot detect who they are. The technique described in Patent Document 2 can identify an individual by face authentication. However, the face of an individual may not be clearly shown in the captured image to the extent that it can be identified. Also, face authentication has a large processing load, so a computer with high processing power is required to frequently perform face authentication for a large number of people. An object of the present invention is to solve such problems, for example.

Means for Solving the Problems

[0005] The personal location detection system comprises an imaging unit that captures images, a personal identification unit that analyzes the images captured by the imaging unit to identify individuals depicted in the images, a location estimation unit that analyzes the images captured by the imaging unit to estimate the location of individuals depicted in the images, a location tracking unit that tracks the movement of individuals based on the location estimated by the location estimation unit, and a personal tracking unit that tracks the movement of individuals by linking the individuals identified by the personal identification unit with the individuals tracked by the location tracking unit. The personal location detection system may further include a feature extraction unit that analyzes images captured by the imaging unit and extracts features of a person captured in the images, and a tracking and combining unit that combines the movements of the person tracked by the location tracking unit based on the features extracted by the feature extraction unit. [Effects of the Invention]

[0006] By analyzing images to estimate a person's position and tracking that person across multiple images based on the estimated position, it's possible to track a person's movement without identifying them individually. Furthermore, if an individual can be identified through image analysis, that individual is linked to the tracked person's movement. Therefore, if an individual can be identified in one of the images in which their movement is being tracked, their movement can be tracked even if they cannot be identified in the other images. This reduces the processing load required for individual identification. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing an example of a personal location detection system. [Figure 2] A diagram illustrating an example of how a personal location detection system works. [Modes for carrying out the invention]

[0008] Refer to Figure 1. The personal location detection system 10 captures images of the target area using a surveillance camera or the like, analyzes the captured images to identify individuals in the images, and tracks their movements to detect their location. The personal location detection system 10 includes, for example, imaging units 11a to 11c, a location estimation unit 21, a location tracking unit 22, a feature extraction unit 23, a tracking and merging unit 24, a personal identification unit 25, and a personal tracking unit 26.

[0009] The imaging units 11a to 11c are each surveillance cameras installed, for example, on the ceiling of a room, and capture images of a predetermined target area at predetermined intervals (for example, 0.1 seconds) and output the captured images. The imaging ranges of the imaging units 11a to 11c overlap in at least part, and as a whole, they cover the entire predetermined detection range.

[0010] The position estimation unit 21 receives images captured by the imaging units 11a to 11c, analyzes the input images, and estimates the position of the person in the analyzed image. The position estimation unit 21 is, for example, a computer, and the processing unit executes a computer program stored in the memory device. The same applies to the other functional blocks described below. Note that multiple functional blocks may be implemented by a single computer, or conversely, a single functional block may be implemented by multiple computers. The position estimation unit 21 uses, for example, the technology described in Patent Document 1 to estimate where a person in the input image is located within the detection range, based on their position and size within the image. The position estimation unit 21 outputs the estimated result (i.e., the position of each person in the image).

[0011] The position tracking unit 22 receives the results estimated by the position estimation unit 21 and tracks the person's movement based on the input information. The position tracking unit 22 determines whether a person appearing in different images is the same person, based on the position estimation unit 21's estimation of the person's position. For example, based on the results of the position estimation unit 21 for two images taken by different imaging units 11a to 11c at approximately the same time, it determines that a person in approximately the same position is the same person. Alternatively, based on the results of the position estimation unit 21 for two images taken consecutively by the same imaging unit 11a to 11c, it compares the position of the person in the previous image with the position of the person in the later image and determines that a person in a similar position is the same person. Furthermore, the location tracking unit 22 will not determine that the individuals are the same person if it is not certain that they are the same person (for example, if there are multiple people in close proximity and it is not possible to determine which is the same person based on location alone). The position tracking unit 22 generates a movement trajectory of a person (for example, time-series data consisting of pairs of time and the location where the person was at that time) based on the location estimated by the position estimation unit 21 for the person it has determined to be the same person. The position tracking unit 22 outputs the generated movement trajectory.

[0012] The feature extraction unit 23 receives images captured by the imaging units 11a to 11c, analyzes the input images, and extracts the features of people depicted in the analyzed images. The feature extraction unit 23 extracts information from a person in the input image that, for example, does not identify the individual but can be used to identify the person, such as the person's physique (e.g., height and build) and clothing (e.g., whether or not they are wearing a hat and the color of their jacket). The feature extraction unit 23 may extract features from all images captured by the imaging units 11a to 11c, or it may extract features from a selection of images. For example, to reduce the processing load, one image may be extracted at predetermined intervals (e.g., 1 second). The feature extraction unit 23 associates the person from whom features have been extracted with the person whose position has been estimated by the position estimation unit 21, based on their location within the image. The feature extraction unit 23 outputs the extracted features (i.e., information about the characteristics of the people in the image).

[0013] The tracking and combining unit 24 receives the features extracted by the feature extraction unit 23, and based on the input features, determines whether the persons shown in different images are the same person. For example, when the ratio of the features extracted by the feature extraction unit 23 for a person shown in a certain image that matches the features extracted by the feature extraction unit 23 for a person shown in another image is equal to or higher than a predetermined threshold value (e.g., 90%), it is determined that the persons are the same person. When the tracking and combining unit 24 determines that the persons separately tracked by the position tracking unit 22 are the same person, the tracking and combining unit 24 combines the movement trajectories generated by the position tracking unit 22. For example, a plurality of movement trajectories generated by the position tracking unit 22 are merged to generate one movement trajectory. The tracking and combining unit 24 combines and updates the movement trajectories generated by the position tracking unit 22 in this way, and outputs the updated movement trajectories (e.g., the newly generated movement trajectory and the movement trajectories that remained uncombined).

[0014] The personal identification unit 25 receives the images captured by the imaging units 11a to 11c, analyzes the input images, and identifies the individuals shown in the analyzed images. That is, the personal identification unit 25 discriminates who the person shown in the input image is. The personal identification unit 25 identifies an individual based on the information obtained from the input image, such as performing face authentication using, for example, the technique described in Patent Document 2. Even when a person is shown in the input image, there may be cases where it is impossible to identify who the person is because the face is not clearly shown, etc. Conversely, when a plurality of persons are shown in the input image, there may also be cases where a plurality of individuals can be identified. When the personal identification unit 25 can discriminate an individual, for each discriminated individual, based on the position in the image, etc., the personal identification unit 25 associates the individual with the person estimated by the position estimation unit 21. Note that the personal identification unit 25 may attempt to identify individuals for all the images captured by the imaging units 11a to 11c, or in order to reduce the processing load, it may extract one image at a predetermined interval (for example, every 2 seconds) and attempt to identify individuals. Alternatively, based on the size of the face shown in the image, etc., it may estimate the possibility of being able to identify individuals, and only extract images with a high possibility of successful personal authentication and attempt personal authentication. The personal identification unit 25 associates the identified individual with the person whose position was estimated by the position estimation unit 21 based on the position in the image, etc. The personal identification unit 25 outputs the identification result (for example, employee number, etc., information about the individual shown in the image).

[0015] The personal tracking unit 26 associates the individual with the movement trajectory output by the tracking combination unit 24 based on the result identified by the personal identification unit 25. For example, it associates the information about the individual identified by the personal identification unit 25 with the movement trajectory tracked by the position tracking unit 22 and the tracking combination unit 24 for that person. Thereby, the personal tracking unit 26 tracks the movement of the person. The personal tracking unit 26 outputs the tracking result (for example, a set consisting of time-series data composed of a pair of the time and the position where the person was at that time, and personal information).

[0016] The result tracked by the personal tracking unit 26 is stored in a recording device such as a server device, for example. Alternatively, it may be displayed on a display device such as a liquid crystal display. The display device may display, for example, a plan view of the detection target range, and may display the position of the tracked individual and information about that individual (for example, name, position, etc.) superimposed on the displayed plan view.

[0017] For example, as shown in FIG. 2, when the same person is shown in six images 90a to 90f, first, the position estimation unit 21 estimates the positions 91a to 91f of that person, and based on the result, the position tracking unit 22 generates the movement trajectories 92a, 92b of the person. Here, we assume that the position tracking unit 22 determines that the person in images 90a to 90c is the same person, and that the person in images 90d to 90f is the same person, but that there is a possibility that the person in images 90d to 90f is not the same person as the person in images 90a to 90c. In that case, the position tracking unit 22 outputs two movement trajectories 92a and 92b. Here, movement trajectory 92a is for the person in images 90a to 90c, and movement trajectory 92b is for the person in images 90d to 90f.

[0018] Separately, the feature extraction unit 23 extracts the features of a person from two of the images 90a to 90f, 90b and 90f, and the tracking and merging unit 24 determines from these features that the person in the two images 90b and 90f is the same person. Then, it is determined that the two movement trajectories 92a and 92b belong to the same person, so the tracking and merging unit 24 merges the two movement trajectories 92a and 92b into a single movement trajectory 94a.

[0019] Furthermore, let's assume that the personal identification unit 25 has identified individual 95d as the person pictured in image 90d. Then, we can see that the movement trajectory 94a belongs to that individual 95d.

[0020] Thus, it is not necessary to identify individuals in every image, and the estimated location of a person in an image that does not identify an individual can still be linked to that individual.

[0021] Furthermore, since the system detects an individual's location based on images captured by the imaging unit, individuals do not need to carry their own location-detection devices. Consequently, there is no need to manage inventory or power supplies for such devices. In addition, even if there are individuals other than the one being managed, their movements can still be tracked.

[0022] As described above, the person being detected does not need to be wearing any device or tag to have their location detected. Therefore, there is no need to manage equipment inventory or power supply, and detection is possible even for people who do not have equipment (e.g., not being provided with equipment or forgetting to bring it). This will enable us to provide a more reliable and convenient location detection service. For example, facial recognition and person detection are performed using camera footage installed on the ceiling to determine the individual's position in two dimensions.

[0023] Conventional electromagnetic wave detection systems require a device on the target side, and cannot detect unregistered individuals; they can only detect registered individuals. For example, in the Wi-Fi system, the location is estimated based on the strength of the communication between an antenna installed on the ceiling or elsewhere and the user's device, using Wi-Fi radio waves. In the geomagnetic method, the Earth's magnetic field is measured using a smartphone or similar device, and the location is estimated by comparing it with a pre-prepared geomagnetic map. In the RFID system, an antenna that emits radio waves is installed on the ceiling or elsewhere, and the reflected waves from the RFID tag held by the person who is not detected are detected to estimate their location. In the BLE method, the location is estimated by using Bluetooth® radio waves to determine the signal strength between an antenna mounted on the ceiling or desk and the user's handheld device. Bluetooth® 5.1, in addition to the BLE mechanism, estimates location with greater accuracy by detecting the direction of detection. In the UWB system, the location is estimated by using UWB radio waves and measuring the signal strength between an antenna mounted on the ceiling or elsewhere and the user's handheld device. In the millimeter-wave method, an antenna that emits radio waves in the millimeter-wave band is used to estimate the position based on the phase difference between the emitted wave and the reflected wave.

[0024] In contrast, the image analysis method analyzes video footage captured by a camera to estimate a person's position, eliminating the need for a device on the detection site and enabling the detection of unregistered individuals. For example, the location of an individual can be determined by analyzing camera footage. In this method, facial recognition and person detection are performed using ceiling cameras. A camera captures images of people, and these images are analyzed. Facial recognition is used to identify individuals from the measured video footage, and their location is estimated through body recognition. By combining these two methods, the individual's location is determined and, for example, plotted on a two-dimensional plane. Once facial recognition and location estimation are linked, subsequent tracking is done using human body recognition, so there is no need to constantly perform individual identification. Even if a person is captured across the camera's frame, the individual can still be identified through the same-person recognition function, which uses the characteristic features of the person obtained through image analysis.

[0025] Since detection is possible even if the person being detected does not have the device, false detections due to forgetting the device can be prevented, and unauthorized persons can be detected. In addition, there is no need for device management. Because it uses images, it is well-suited for services such as pulse detection and suspicious person / behavior detection.

[0026] The embodiments described above are examples intended to facilitate understanding of the present invention. The present invention is not limited thereto and includes various modifications, changes, additions, or deletions without departing from the scope defined by the appended claims. This will be readily apparent to those skilled in the art from the above description. [Explanation of Symbols]

[0027] 10 Personal location detection system, 11a-11c Imaging unit, 21 Location estimation unit, 22 Location tracking unit, 23 Feature extraction unit, 24 Tracking and merging unit, 25 Personal identification unit, 26 Personal tracking unit, 90a-90f Image, 91a-91f Location, 92a, 92b, 94a Movement trajectory, 93b, 93f Features, 95d Personal.

Claims

1. Multiple imaging units that capture images, A position estimation unit analyzes images captured by the multiple imaging units and estimates the position of a person in the image, A position tracking unit generates a movement trajectory of the same person based on the position estimation unit's estimation of the person's position, A feature extraction unit analyzes images captured by the multiple imaging units and extracts features of people depicted in the images, Based on the features extracted by the feature extraction unit, a tracking and combining unit combines the movement trajectories of the same person having the same features from among the movement trajectories generated by the position tracking unit, A personal identification unit analyzes images captured by the aforementioned multiple imaging units to identify individuals depicted in the images, The personal tracking unit tracks the movement of the individual by linking the individual identified by the personal identification unit with the person corresponding to the movement trajectory linked by the tracking and linking unit. Equipped with, When the personal identification unit identifies the individual, the personal tracking unit links the movement trajectory, which is linked to the movement trajectory corresponding to the image in which the individual is pictured, with the individual, thereby tracking movement trajectories for which the individual has not been identified by the personal identification unit as the movement of the individual. After the movement trajectory, which is linked to the movement trajectory corresponding to the image in which the individual is pictured, with the individual, the personal identification unit does not identify the individual again, but instead takes over the identification of the individual by tracking based on features extracted by the feature extraction unit. Personal location detection system.

2. The personal identification unit estimates the possibility of personal identification based on the size of the faces of the people captured in the images taken by the plurality of imaging units, and attempts to identify the individuals only in images where the estimated possibility is above a predetermined threshold. The personal location detection system according to claim 1.

3. The position tracking unit determines that people in the same location are the same person based on the location estimation unit's estimation of the person's location in multiple images taken simultaneously by different imaging units among the multiple imaging units, or determines that people are the same person based on the location estimation unit's estimation of the person's location in multiple images taken consecutively by the same imaging unit, if the difference between the person's location in the preceding image and the person's location in the subsequent image is less than or equal to a predetermined threshold. The personal location detection system according to claim 1.

4. The position tracking unit generates a movement trajectory for the same person if it determines that the person is the same person based on the position estimated by the position estimation unit, and generates a different movement trajectory if it does not determine that the person is the same person. The tracking and combining unit, when it determines that multiple movement trajectories generated by the position tracking unit as different movement trajectories belong to the same person based on the features extracted by the feature extraction unit, combines the multiple movement trajectories into a single movement trajectory. The personal location detection system according to claim 1.

5. Each of the aforementioned plurality of imaging units captures a predetermined target area, The target area of ​​each of the multiple imaging units overlaps in part, and as a whole, it covers the entire predetermined detection target area. A personal location detection system according to any one of claims 1 to 4.

Images