Location management system, location management method, and location management program

Abstract

To provide a location management system, a location management method, and a location management program for enhancing security.SOLUTION: A location management system A1 includes: a control unit 21 connected to a camera C1; and a location information storage unit 24. Then, the control unit 21 identifies a person image area in which a person is recognized in a captured image of the camera, sequentially identifies locations of the person according to positions of the person image area, and records the identified locations in the location information storage unit 24. When a code image is detected in the person image area, the control unit 21 records a code corresponding to a code image in the location information storage unit 24 in association with the location.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a location management system, a location management method, and a location management program for managing the presence of a person in a specific area.

Background Art

[0002] Conventionally, in an area where security needs to be ensured, the movement of people may be managed. For this reason, an entrance / exit management device for improving security has been studied (see, for example, Patent Document 1). The entrance / exit management device described in this patent document detects the number of persons who are about to enter or exit a security section that requires security, by image analysis in a captured image. Further, by reading a color bar code attached to a person to be determined, the number of persons who are about to enter or exit the security section is detected. Then, when the number of persons detected by image analysis and the color bar code match, entry / exit to the security section is permitted.

[0003] In addition, an object tracking device for tracking an object included in a moving image has been studied (see, for example, Patent Document 2). The object tracking device described in this patent document detects feature points of each frame by inputting each frame image of the moving image into a machine learning model capable of outputting the type of a subject. The object is tracked by comparing the positions of the feature points of each frame.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the technology described in Patent Document 1, the movement of people within the security area cannot be managed. Further, with the technology described in Patent Document 2, the tracked object cannot be identified.

Means for Solving the Problems

[0006] The location management system for solving the above problems includes a control unit connected to a camera and a location information storage unit. The control unit identifies a person image area in which a person is recognized in the captured image of the camera, sequentially identifies the location of the person according to the position of the person image area, and records it in the location information storage unit. When a code image is detected in the person image area, the code corresponding to the code image is recorded in the location information storage unit in association with the location.

Advantages of the Invention

[0007] According to the present invention, by managing the location of the user, security can be improved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0009] Hereinafter, an embodiment in which the location management system is embodied will be described with reference to FIGS. 1 to 6. In this embodiment, a location management system, a location management method, and a location management program used when managing the location of a worker (user) in a factory will be described. In this embodiment, the code image possessed by the user is recognized to identify the user, and the location of the user in the management target area is tracked by an image tracking technique. A Chameleon Code (registered trademark) is used as the code image. The Chameleon Code is a code image in which a code is displayed by arranging a plurality of colors (cyan, magenta, yellow, black) in a predetermined layout. This Chameleon Code can be recognized collectively even when a plurality of code images are included in a single captured image. As shown in FIG. 1, the location management system A1 is composed of a camera C1, a management terminal 10, a management server 20, etc. connected via a network.

[0010] (Description of Hardware Configuration) Using FIG. 2, the hardware configuration of the information processing device H10 that constitutes the management terminal 10 and the management server 20 will be described. The information processing device H10 includes a communication device H11, an input device H12, a display device H13, a storage device H14, and a processor H15. Note that this hardware configuration is an example, and it is also possible to be realized by other hardware.

[0011] The communication device H11 is an interface that establishes a communication path with other devices and executes data transmission and reception, and is, for example, a network interface or a wireless interface.

[0012] The input device H12 is a device that accepts input of various information, and is, for example, a mouse or a keyboard. The display device H13 is a display or the like that displays various information. The storage device H14 is a storage device that stores data and various programs for executing various functions of the management terminal 10 and the management server 20. Examples of the storage device H14 include a ROM, a RAM, and a hard disk.

[0013] Processor H15 controls each process in management terminal 10 and management server 20 by using programs and data stored in storage device H14. Examples of processor H15 include, for example, a CPU, GPU, MPU, etc. This processor H15 expands a program stored in a ROM or the like to a RAM and executes various processes for each process.

[0014] Processor H15 is not limited to performing software processing for all processes it executes. For example, processor H15 may include a dedicated hardware circuit (for example, an application-specific integrated circuit: ASIC) that performs hardware processing for at least a part of the processes it executes. The processor includes a CPU, GPU, and memories such as a RAM and a ROM, and the memories store program codes or instructions configured to cause the CPU and GPU to execute processes. The memory, that is, the computer-readable medium, includes any available medium that can be accessed by a general-purpose or dedicated computer.

[0015] (Description of location management system A1) Camera C1 transmits a color video obtained by photographing a visual field range of a predetermined range of a management target area to management server 20. This camera C1 is installed at a position (for example, the ceiling) in the factory where the management target area can be overlooked from above. Then, camera C1 generates a photographed image obtained by photographing the lower management target area (predetermined photographing range) from above.

[0016] Management terminal 10 is a computer terminal used by an administrator who manages the location of a user. Management server 20 is a computer that manages the location of a user. Management server 20 includes a control unit 21, a user information storage unit 22, an area information storage unit 23, and a location information storage unit 24.

[0017] The control unit 21 performs the processes (each process of the image recognition stage, the tracking stage, and the code recognition stage) described later. By executing a management program therefor, the control unit 21 functions as an image recognition unit 211, a tracking unit 212, and a code recognition unit 213.

[0018] The image recognition unit 211 executes a process of recognizing a person in a captured image and generating a person image area (bounding box) in which this person was captured. This image recognition unit 211 can be realized, for example, by YOLO (registered trademark), which is an object detection algorithm.

[0019] The tracking unit 212 executes a process of tracking the movement of a person by sequentially specifying the location using the feature amount of the person image area included in each frame image of the video. The code recognition unit 213 executes a process of acquiring a color code by specifying a code image included in a frame image and decoding the code image.

[0020] In the user information storage unit 22, user management data for managing each user is recorded. This user management data is recorded when each user is registered. In this user management data, data regarding a color code, a user ID, a name, and an affiliation is recorded.

[0021] The color code is an identifier for specifying a code image. The user ID is an identifier for specifying a user who carries a code image. The name and the affiliation are information regarding the name and the affiliation of this user.

[0022] In the area information storage unit 23, area management data for managing the area photographed by the camera C1 is recorded. This area management data is recorded when the installation location of the camera C1 is registered. In this area management data, data regarding a camera ID and a photographed area is recorded.

[0023] The camera ID is an identifier for specifying each camera C1. The photographed area is composed of coordinates representing the area included in the photographed image taken by this camera in the area to be managed. Based on this photographed area, the coordinates where the subject (person) is located can be specified in the area to be managed.

[0024] In the location information storage unit 24, location management data for managing the location of each user is recorded. This location management data is recorded when the location of the user is specified. In this location management data, data regarding the feature amount, person image ID, user ID, detection time, and location is recorded.

[0025] The feature amount is information (such as a numerical value or a vector) representing the shape features of the subject included in the detected person image area. The person image ID is an identifier for specifying the person image area.

[0026] The user ID is an identifier of the user specified by recognizing the code image included in the person image area. The detection time is the year, month, day, and time when the person image area was detected. The location is information for specifying the location where this person image area was detected. This location is represented by, for example, coordinates or an area identifier.

[0027] (Tracking process of a person) Next, the tracking process of a person will be described with reference to FIG. 3.

[0028] Here, the control unit 21 of the management server 20 executes the acquisition process of the frame image (step S11). Specifically, the image recognition unit 211 of the control unit 21 acquires a predetermined frame image (photographed image) in the video taken by the camera C1.

[0029] Next, the control unit 21 of the management server 20 executes the recognition process of a person (step S12). Specifically, the image recognition unit 211 of the control unit 21 searches for a person in the frame image by object recognition processing.

[0030] Next, the control unit 21 of the management server 20 executes a determination process as to whether a person is included (step S13). Specifically, when the image recognition unit 211 of the control unit 21 detects a person in the frame image, it determines that a person is included.

[0031] When it is determined that no person is included (in the case of "NO" in step S13), the control unit 21 of the management server 20 returns to the acquisition process of the frame image (step S11). Here, the image recognition unit 211 of the control unit 21 acquires a subsequent frame image in the video.

[0032] On the other hand, when it is determined that a person is included (in the case of "YES" in step S13), the control unit 21 of the management server 20 executes a process of specifying the person image area (step S14). Specifically, the image recognition unit 211 of the control unit 21 specifies the person image area where a person is recognized in the frame image.

[0033] As shown in FIG. 5, in the frame image 500, a plurality of persons P1 and P2 are included. In this case, corresponding to each person P1 and P2, person image areas IP1 and IP2 are specified.

[0034] Next, the control unit 21 of the management server 20 executes a determination process as to whether the feature amount of the person image area is already registered (step S15). Specifically, the tracking unit 212 of the control unit 21 calculates the feature amount of each person image area. Next, the tracking unit 212 checks whether a feature amount with a high similarity to the calculated feature amount is recorded in the location information storage unit 24. Here, for example, when the feature amounts match by 80% or more, it is determined that the similarity is high.

[0035] When it is determined that there are no recorded feature quantities with high similarity and the feature quantities in the person image area are not registered (when "NO" in step S15), the control unit 21 of the management server 20 executes a process of assigning a new registration code (step S16). Specifically, the tracking unit 212 of the control unit 21 assigns a person image ID to the feature quantities in the person image area. Next, the tracking unit 212 records the location management data including the feature quantities and the person image ID in the location information storage unit 24.

[0036] On the other hand, when it is determined that there are recorded feature quantities with high similarity and the feature quantities in the person image area are registered (when "YES" in step S15), the control unit 21 of the management server 20 executes a process of specifying the registered code (step S17). Specifically, the tracking unit 212 of the control unit 21 specifies the person image ID recorded in the location management data in which the feature quantities with high similarity are recorded in the location information storage unit 24.

[0037] Next, the control unit 21 of the management server 20 executes a process of recording the location of the person (step S18). Specifically, the tracking unit 212 of the control unit 21 uses the area information storage unit 23 to specify the location according to the location of the person image area. Then, the tracking unit 212 records the location in the location management data together with the detection time. Next, the control unit 21 of the management server 20 returns to the process of acquiring the frame image (step S11).

[0038] (Code recognition process) Next, the code recognition process will be described with reference to FIG. 4. This process is performed when a frame image is acquired.

[0039] Here, the control unit 21 of the management server 20 executes a process of searching for the code image (step S21). Specifically, the code recognition unit 213 of the control unit 21 searches for the code image in the frame image by pattern recognition of the layout pattern of the code image.

[0040] Next, the control unit 21 of the management server 20 executes a determination process as to whether a code image is included (step S22). Specifically, if the code recognition unit 213 of the control unit 21 detects the layout pattern of the code image in the frame image, it determines that the code image is included. Note that it is also possible to identify a plurality of code images in the frame image.

[0041] If it is determined that no code image is included (in the case of "NO" in step S22), the control unit 21 of the management server 20 ends the code recognition process. Then, when a subsequent frame image is acquired, the code recognition process is executed again.

[0042] On the other hand, if it is determined that a code image is included (in the case of "YES" in step S22), the control unit 21 of the management server 20 executes a code identification process (step S23). Specifically, the code recognition unit 213 of the control unit 21 decodes the code image and identifies the color code.

[0043] Next, the control unit 21 of the management server 20 executes a determination process as to whether it corresponds to a person (step S24). Specifically, the code recognition unit 213 of the control unit 21 compares the detected position of the code image with the arrangement of the person image area. Then, if the detected position of the code image is included in the person image area, it determines that it corresponds to a person.

[0044] If the detected position of the code image is outside the person image area and it is determined that it does not correspond to a person (in the case of "NO" in step S24), the control unit 21 of the management server 20 ends the code recognition process.

[0045] On the other hand, if the detected position of the code image is within the person image area and it is determined that it corresponds to a person (in the case of "YES" in step S24), the control unit 21 of the management server 20 executes a user ID identification process (step S25). Specifically, the code recognition unit 213 of the control unit 21 uses the user information storage unit 22 to identify the user ID corresponding to the color code.

[0046] As shown in FIG. 6, in the frame image 501, the code images CC1 and CC2 are detected. In this case, the code image CC1 within the person image area IP1 of the person P1 and the code image CC2 within the person image area IP2 of the person P2 are respectively associated. On the other hand, since the code image CC3 outside the person image area is outside the person image area, it is not used for specifying the user ID.

[0047] Next, the control unit 21 of the management server 20 executes a recording process of the user ID for the person image (step S26). Specifically, the code recognition unit 213 of the control unit 21 records the user ID in the location management data in which the person image ID of the person image area is recorded.

[0048] (Location confirmation process) When the administrator confirms the location of the user, the management terminal 10 is used to access the management server 20. In this case, the control unit 21 of the management server 20 arranges the movement trajectory connecting the location positions of each user recorded in the location information storage unit 24 in time series on the map of the management target area. Then, the control unit 21 outputs the map on which the movement trajectory is arranged to the management terminal 10.

[0049] (Function of this embodiment) Since object recognition and code recognition are used in combination, with one captured image, the person specified by the object recognition process is tracked, and this person is specified using the code image.

[0050] (Effect of this embodiment) (1) In this embodiment, the control unit 21 of the management server 20 executes the acquisition process of the frame image (step S11) and the recognition process of the person (step S12). Thereby, the presence of a person can be confirmed in the captured image.

[0051] (2) In this embodiment, when it is determined that a person is included (when "YES" in step S13), the control unit 21 of the management server 20 executes a process of specifying a person image area (step S14). As a result, in the frame image, the area where the person appears can be specified.

[0052] (3) In this embodiment, when it is determined that the feature amount of the person image area is not registered (when "NO" in step S15), the control unit 21 of the management server 20 executes a process of assigning a new registration code (step S16). As a result, a person image ID can be assigned to a newly photographed person.

[0053] (4) In this embodiment, when it is determined that the feature amount of the person image area is registered (when "YES" in step S15), the control unit 21 of the management server 20 executes a process of specifying a registered code (step S17). As a result, person images with common feature amounts can be tracked.

[0054] (5) In this embodiment, the control unit 21 of the management server 20 executes a process of recording the location of the person (step S18). As a result, using the photographed image, the location of the person can be specified.

[0055] (6) In this embodiment, the control unit 21 of the management server 20 executes a process of searching for a code image (step S21). As a result, the code image can be specified in the photographed image.

[0056] (7) In this embodiment, the control unit 21 of the management server 20 executes a process of specifying a code (step S23) and a determination process as to whether it corresponds to a person (step S24). As a result, the person recognized by the object recognition process and the code image can be associated with each other.

[0057] (8) In this embodiment, when it is determined that it corresponds to a person (when "YES" in step S24), the control unit 21 of the management server 20 executes a process of specifying the user ID (step S25). Thereby, the location of the user can be managed. Usually, in a factory, workers wear the same work clothes, making it difficult to identify individuals. Also, it is possible to assign unique codes to each individual and track them, but facilities for that are required. For example, when tracking using color codes, it is necessary to install cameras at a distance where the color codes can be recognized, and many cameras are required. Also, when using wireless technologies such as RFID, it is necessary to install many antennas. In the present invention, since the code image and the person are associated to track this person, location management can be performed with relatively simple facilities.

[0058] This embodiment can be implemented with the following modifications. This embodiment and the following modification examples can be implemented in combination with each other within a technically non - conflicting range. · In the above - described embodiment, the image recognition unit 211 is realized by YOLO, which is an object detection algorithm. The tracking process for a person is not limited to the case of using YOLO. · In the above - described embodiment, a chameleon code is used as the code image. It is not limited to the chameleon code as long as it is a code image that can be recognized by a camera.

[0059] · In the above - described embodiment, cameras are provided for each area. Instead of this, a plurality of cameras may be arranged at different positions in one area. In this case, one person is photographed by a plurality of cameras.

[0060] As shown in FIG. 7, cameras C11 and C12 that photograph the same range are installed at different positions. Then, the control unit 21 of the management server 20 executes the acquisition process of the person feature amount for each camera (step S31). Specifically, the image recognition unit 211 of the control unit 21 captures the persons P3 and P4 with the respective cameras C11 and C12. In this case, the camera C11 identifies the person image areas IP31 and IP41 of the persons P3 and P4. Also, the camera C12 identifies the person image areas IP32 and IP42 of the persons P3 and P4. Then, the respective image feature amounts are calculated from the person image areas IP31, IP41, IP32, and IP42. In this case, the image feature amount and the user ID are associated for each of the persons P3 and P4.

[0061] Next, the control unit 21 of the management server 20 executes the identification process of the person feature amounts with high similarity (step S32). Specifically, the tracking unit 212 of the control unit 21 respectively identifies the location management data 241 and 242 in which the feature amounts of the person image areas captured by the cameras C11 and C12 are similar. Then, according to the similarity of the respective image feature amounts, the tracking of the person image areas is performed using the location management data 241 and 242.

[0062] Thereby, since the person feature amounts captured by a plurality of cameras are used, the tracking can be continuously performed complementarily. For example, even when the feature amount of the person image area becomes dissimilar in the captured image of one camera and the tracking becomes impossible, the tracking can be continued with the captured image of the other camera. In this case, when a person image area with a similar person feature amount is detected by one camera and the state returns to a state where tracking can be performed with the captured image, tracking becomes possible again with a plurality of cameras.

Explanation of Reference Numerals

[0063] A1... Location management system, C1... Camera, 10... Management terminal, 20... Management server, 21... Control unit, 211... Image recognition unit, 212... Tracking unit, 213... Coat recognition unit, 22... User information storage unit, 23... Area information storage unit, 24... Location information storage unit.

Claims

1. A location management system comprising a camera and a control unit connected to a location information storage unit, wherein the control unit identifies a person image area in which a person is recognized in a captured image of the camera, sequentially identifies the location of the person according to the position of the person image area and records it in the location information storage unit, and when a code image is detected in the person image area, records a code corresponding to the code image in the location information storage unit in association with the location. A location management system characterized by this.

2. The location management system according to claim 1, wherein the control unit connects the locations in time series to identify a movement trajectory.

3. The control unit recognizes the same person with a plurality of the cameras and identifies each person image area, and when the code image is detected in the person image area of the captured image of at least one of the cameras, associates a code corresponding to the code image with each person image area. The location management system according to claim 1 or 2, characterized by this.

4. A method for managing a location using a location management system comprising a camera and a control unit connected to a location information storage unit, wherein the control unit identifies a person image area in which a person is recognized in a captured image of the camera, sequentially identifies the location of the person according to the position of the person image area and records it in the location information storage unit, and when a code image is detected in the person image area, records a code corresponding to the code image in the location information storage unit in association with the location. A location management method characterized by this.

5. A program for managing a location using a location management system comprising a camera and a control unit connected to a location information storage unit, wherein the control unit is caused to identify a person image area in which a person is recognized in a captured image of the camera, sequentially identify the location of the person according to the position of the person image area and record it in the location information storage unit, and function as means for recording a code corresponding to the code image in the location information storage unit in association with the location when the code image is detected in the person image area. A location management program characterized by this.

Images