Entry / exit management method and management device

Abstract

To detect the entry and exit of a target to be controlled into / from a predetermined area with a single entrance / exit, using a single camera.SOLUTION: Using a camera image captured by a single camera installed near an entrance / exit of a predetermined area, a face image and a full-body image of a given person passing through the entrance / exit are acquired. An entry of the target to be controlled into the predetermined area is detected by a face recognition process using the face image of any person in the camera image and a registered face image of the target to be controlled. Using the full-body image of the target to be controlled whose entry into the predetermined area is detected, feature values of the target to be controlled are extracted. Using the full-body image of any person included in the camera image after the entry of the target to be controlled into the predetermined area, the feature values of any person are extracted. An exit from the predetermined area by the target to be controlled is detected by a re-identification process using the extracted feature values of the target to be controlled and the extracted feature values of any person.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a method and apparatus for managing the entry and exit of people in a predetermined area having one entrance and exit.

Background Art

[0002] Japanese Patent Application Laid-Open No. 2021-152738 discloses an authentication device for a person entering a predetermined area having a first and a second door. This conventional authentication device performs face authentication of a person using a face image of a first camera provided near the first door outside the predetermined area. When the face authentication is successful, the lock of the first door is released, and further, the whole body information of the person subject to face authentication is acquired by the first camera. When acquiring the whole body information of the target person, an instruction is given to the person who has successfully passed the face authentication to take a registration pose in front of the first camera. That is, the whole body information includes a whole body image of the person who has taken the registration pose.

[0003] The conventional authentication device also compares the whole body information acquired by the second camera provided near the second door inside the predetermined area with the whole body information acquired by the first camera. When a person takes the same pose as the registration pose taken near the first door near the second door, whole body information including a whole body image of the person who has taken the authentication pose is obtained. When the similarity between the registration pose and the authentication pose is equal to or higher than a threshold value, the lock of the second door is released.

[0004] As a document showing the technical level of the technical field related to the present disclosure, in addition to Japanese Patent Application Laid-Open No. 2021-152738, Japanese Patent Application Laid-Open No. 2010-154134 can be exemplified.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Consider the case of managing the entry and exit of the same person to a predetermined area having a certain entrance / exit. As a management method in this case, there is a method of detecting the entry of a certain management target to a predetermined area and then detecting the exit of this management target from the predetermined area. Specifically, a camera is provided near the entrance / exit to acquire a face image of a person, and face authentication using this face image is performed. Thereby, it becomes possible to detect the entry of a management target to a predetermined area and the exit of the management target from the predetermined area.

[0007] However, when there is only one entrance / exit, since the traveling directions of the person are opposite at the time of entry and exit, the orientations of the person's face hardly match. Therefore, in face authentication using a face image acquired from a single camera provided near the entrance / exit, there is a possibility that the entry of a management target to a predetermined area and the exit from the predetermined area of the same person cannot be detected.

[0008] In this regard, if two or more cameras are provided near the entrance / exit, it is also possible to solve this problem. However, when there are restrictions on the installation positions of the cameras, there are times when only one camera can be installed near the entrance / exit. Therefore, development of a technology for realizing detection of the entry and exit of a management target to a predetermined area having only one entrance / exit using a single camera is desired.

[0009] One object of the present disclosure is to provide a technology capable of realizing detection of the entry and exit of a management target to a predetermined area having one entrance / exit using a single camera.

Means for Solving the Problems

[0010] The first aspect of the present disclosure is a method for managing the entry and exit of a management target to a predetermined area having one entrance / exit, and has the following features. The method includes the steps of: using a camera image captured by one camera provided near the entrance / exit to obtain a face image and a full-body image of any person passing through the entrance / exit; performing a face authentication process using the face image of the any person included in the camera image and the registered face image of the management target to detect entry of the management target into the predetermined area; using the full-body image of the management target in which entry into the predetermined area has been detected by the face authentication process among the full-body images of the any person included in the camera image to extract feature amounts of the management target; using the full-body image of the any person included in the camera image after entry of the management target into the predetermined area to extract feature amounts of the any person; and performing a re-identification process using the feature amounts of the management target extracted based on the camera image and the feature amounts of the any person extracted based on the camera image to detect exit of the management target from the predetermined area.

[0011] A second aspect of the present disclosure is an apparatus for managing entry and exit of a management target to / from a predetermined area having one entrance / exit, and has the following features. The apparatus includes a processor that performs various processes. The processor uses a camera image captured by one camera provided near the entrance / exit to obtain a face image and a full-body image of any person passing through the entrance / exit, performs a face authentication process using the face image of the any person included in the camera image and the registered face image of the management target to detect entry of the management target into the predetermined area, uses the full-body image of the management target in which entry into the predetermined area has been detected by the face authentication process among the full-body images of the any person included in the camera image to extract feature amounts of the management target, uses the full-body image of the any person included in the camera image after entry of the management target into the predetermined area to extract feature amounts of the any person, and is configured to perform a re-identification process using the extracted feature amounts of the management target and the extracted feature amounts of the any person to detect exit of the management target from the predetermined area.

Advantages of the Invention

[0012] According to the present disclosure, a face image and a full-body image of any person passing through this entrance are obtained by one camera provided near the entrance of a predetermined area. Then, a face authentication process using the obtained face image of any person is performed, and the entry of the management target into the predetermined area is detected. Further, extraction of feature amounts using the obtained full-body image of any person and re-identification processing using these feature amounts are performed. The obtained full-body image of any person includes that of the management target for which entry into the predetermined area has been detected and that of any person after the management target has entered the predetermined area. Therefore, by performing the re-identification processing, the departure of the management target from the predetermined area is detected. Accordingly, it becomes possible to detect the entry and exit of the management target to and from a predetermined area having only one entrance using one camera.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each figure, the same or corresponding parts are denoted by the same reference numerals to simplify or omit the description thereof.

[0015] 1. First Embodiment 1-1. Configuration Example FIG. 1 is a diagram for explaining a configuration example of an entrance / exit management device according to the first embodiment and a configuration example of a predetermined area to which this is applied. The management device 10 shown in FIG. 1 is a management device according to the first embodiment. The management device 10 includes at least one processor 11 and at least one storage device 12. The processor 11 executes various processes. Examples of the processor 11 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), ASICs (Application Specific Integrated Circuits), an FPGA (Field-Programmable Gate Array), and the like. The storage device 12 stores (stores) various information. Examples of the storage device 12 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like.

[0016] The management device 10 is configured to be communicable with a camera 22 provided near the entrance / exit 21 of the predetermined area 20. In the present disclosure, the predetermined area 20 means a space having a certain area. Examples of the space having a certain area include a room provided in a facility (for example, a childcare facility, an educational facility). A room and a passage connecting to this room are also an example of a space having a certain area. The space having a certain area may include a plurality of rooms. However, in the first embodiment, only one camera 22 is provided at the entrance / exit 21. Examples of the entrance / exit 21 include an entrance of a facility, an entrance / exit of a room provided in this facility, and an entrance / exit of a passage connecting to this room. Note that a wired or wireless network is used for the communication network connecting the management device 10 and the camera 22.

[0017] In the example shown in FIG. 1, the camera 22 is provided inside the predetermined area 20. The camera 22 is attached to, for example, the ceiling surface or the side wall surface of the predetermined area 20. The pointing direction of the camera 22 is a direction from the inside to the outside of the predetermined area 20. The imaging range of the camera 22 includes the entire entrance / exit 21 and the floor surface near this entrance / exit 21. According to such a camera 22, it becomes possible to acquire a front image of any person entering the predetermined area 20 through the entrance / exit 21.

[0018] 1-2. Features of the First Embodiment FIG. 2 is a diagram for explaining the focus of the first embodiment. As described with reference to FIG. 1, according to the camera 22, a front image of any person entering the predetermined area 20 can be acquired. Therefore, in the first embodiment, a face image IMF_PS1 of an arbitrary person PS1 is acquired from the camera images constituting the video VD1 acquired by the camera 22. Then, face authentication processing is performed using this face image IMF_PS1. In the face authentication processing, the face image IMF_PS1 and a pre-registered face image IMF_PT are compared. The face image IMF_PT belongs to a pre-registered person (hereinafter also referred to as "management target PT"). When the face image IMF_PS1 and the face image IMG_PT match (for example, when the similarity between the two is equal to or greater than a threshold value), it can be detected that the management target PT has entered the predetermined area 20.

[0019] However, the traveling direction of the person PS1 when entering the predetermined area 20 through the entrance / exit 21 is exactly the opposite of that when exiting the predetermined area 20 through the entrance / exit 21. Therefore, in order to perform the above-described face authentication processing for the purpose of detecting that the management target PT has exited the predetermined area 20, a predetermined operation (for example, a turning-back operation, an operation of looking at the camera 22, etc.) for providing a face image to be compared with the face image IMF_PT has to be forced on the person passing through the entrance / exit 21.

[0020] Therefore, in the first embodiment, in order to detect that the management target PT has left the predetermined area 20, a person re-identification process is performed. The person re-identification process (ReIDentification processing) is a technology for identifying the same person from a plurality of videos. In the person re-identification process, feature amounts of a person extracted from a video are used. This feature amount is also called a ReID feature amount. The extraction of the ReID feature amount is performed, for example, by applying a set of bounding boxes associated as representing the same person at a plurality of time steps to a ReID model based on machine learning. Note that the extraction of the ReID feature amount itself is a well-known technology, and the extraction method applied to the first embodiment is not particularly limited.

[0021] FIG. 3 is a diagram for explaining the features of the first embodiment. In order to extract the ReID feature amount of the management target PT, in the first embodiment, a full-body image IMB_PT of the management target PT in which entry into the predetermined area 20 has been detected in the face authentication process is specified. The full-body image IMB_PT can be specified based on a camera image that constitutes a video VD1 at the time when the management target PT enters the predetermined area 20 (for example, the passing time of the entrance / exit 21 by the management target TP, the time immediately before or after this passing time). By extracting the ReID feature amount of the management target PT, the management target PT can be identified.

[0022] In the first embodiment, in addition, the ReID feature amount of any person PS2 leaving the predetermined area 20 is also extracted. Then, a re-identification process is performed using the ReID feature amount of the management target PT extracted when the management target PT enters the predetermined area 20 and the ReID feature amount of the person PS2 extracted after the management target PT enters the predetermined area 20 (for example, after the passing time of the entrance / exit 21 by the management target TP). Thereby, it can be detected that the management target PT has left the predetermined area 20.

[0023] Thus, according to the first embodiment, in addition to the face authentication process, a person re-identification process is performed. Therefore, it becomes possible to detect the entry and exit of the management target PT for a predetermined area 20 having only one entrance / exit 21 using one camera 22.

[0024] 1-3. Example of Functional Configuration FIG. 4 is a block diagram showing an example of the functional configuration of the management device 10 shown in FIG. 1. In the example shown in FIG. 4, the management device 10 includes a person detection unit 31, a face detection unit 32, a face authentication unit 33, a face image management unit 34, an entry detection unit 35, a tracking unit 36, a feature amount extraction unit 37, a feature amount management unit 38, and an exit detection unit 39. These functions are implemented in a circuit or a processing circuit including, for example, a general-purpose processor, a specific-purpose processor, an integrated circuit, ASICs, a CPU, a conventional circuit, and / or a combination thereof programmed to realize these functions.

[0025] Here, the processor includes transistors and other circuits and is regarded as a circuit or a processing circuit. The processor may be a program processor that executes a program stored in a memory. In the present disclosure, a circuit, a unit, or a means is hardware programmed to realize the described function or hardware that executes this function. The hardware may be any hardware disclosed in this specification or any hardware known as being programmed to realize or execute the above function. When the hardware is a processor regarded as a circuit type, this circuit, means, or unit is a combination of hardware and software used to configure this hardware and / or the processor.

[0026] The video VD1 acquired by the camera 22 is input to the person detection unit 31. The person detection unit 31 performs a person detection process of detecting a person PS1 in the camera images of a plurality of time steps included in the video VD1. In the person detection process, a bounding box is assigned to the person PS1 in each camera image. This bounding box represents the position of the person PS1 detected in the camera image. In the person detection process, the information of the bounding box assigned to the person PS1 in each camera image is acquired. The bounding box is assigned to the image of the face part of the person PS1 or the full body image of the person PS1. Note that the person detection process is a well-known technique and its method is not particularly limited. For example, YOLOX is applied to the person detection unit 31.

[0027] The face detection unit 32 receives the bounding box information from the person detection unit 31. The face detection unit 32 performs a face detection process of detecting the face image of the person PS1 based on this bounding box information. When a bounding box is assigned to the image of the face part of the person PS1 in the person detection process, the bounding box information is passed in the face detection process. When a bounding box is assigned to the full body image of the person PS1 in the person detection process, in the face detection process, the image of the face part is extracted from the full body image.

[0028] The face recognition unit 33 receives the face image of the person PS1 (i.e., the face image IMF_SP1) from the face detection unit 32. The face recognition unit 33 performs face recognition processing using this face image and the face image of the management target TP (i.e., the face image IMF_TP) stored in the face image management unit 34 (e.g., the storage device 12). In the face recognition processing, the face image IMF_SP1 and the face image IMF_TP are compared. If the two match, it is determined that the person PS1 is the same person as the management target TP. The face recognition unit 33 sends the comparison result to the entry detection unit 35. The comparison result includes determination information for the face image IMF_SP1 and attached information for this face image IMF_SP1. When it is determined that the person PS1 is the management target TP, the determination information includes the identification information of the management target TP. The attached information includes the identification information of the camera 22 that acquired the face image IMF_SP1, and the coordinate information and timestamp information of the face image IMF_SP1.

[0029] The entry detection unit 35 receives the comparison result from the face recognition unit 33. The entry detection unit 35 detects the entry of the management target TP based on the determination information included in the comparison result. When the entry of the management target TP is detected, the entry detection unit 35 also outputs the detection information of the entry of the management target TP to the feature amount management unit 38 together with the attached information included in the comparison result.

[0030] The tracking unit 36 receives the bounding box information from the person detection unit 31. The tracking unit 36 performs tracking processing of the person PS1 based on this bounding box information. The tracking processing is a technique for automatically tracking the same person included in the camera image based on a tracking algorithm. In the tracking processing, specifically, a plurality of bounding boxes representing the same person (i.e., the person PS1) at a plurality of time steps are associated with each other. Thereby, information representing the time series of the plurality of bounding boxes is generated. Note that the tracking processing itself is a well-known technique, and its method is not particularly limited.

[0031] The feature extraction unit 37 receives a set of a plurality of bounding boxes associated with each other as representing the same person from the tracking unit 36. The feature extraction unit 37 performs an extraction process of extracting the ReID feature amount of the same person (that is, the person PS1) based on this set of bounding boxes. The extraction process is performed using, for example, a ReID model. The ReID model is, for example, a model based on a Transformer.

[0032] The feature management unit 38 receives information on the ReID feature amount from the feature extraction unit 37. The feature management unit 38 stores the input information from the feature extraction unit 37 in the storage device 12. The feature management unit 38 also receives the detection information of the entry of the management target TP and the attached information included in the collation result from the entry detection unit 35. The feature management unit 38 specifies, based on the input information from the entry detection unit 35, the ReID feature amounts corresponding to the ReID feature amount of the management target TP among the ReID feature amounts included in the input information from the feature extraction unit 37. The specification of the ReID feature amount of the management target TP can be performed using, for example, the attached information included in the collation result, the coordinate information and the timestamp information of the bounding box that was the target of the extraction of the ReID feature amount. The information on the ReID feature amount of the specified management target TP is stored in the storage device 12.

[0033] The exit management unit 39 receives information on the ReID feature amount of the person SP2 from the feature extraction unit 37. The exit management unit 39 performs a re-identification process of the management target TP using the input information from the feature extraction unit 37 and the ReID feature amount of the management target TP stored in the feature management unit 38 (storage device 12). In this re-identification process, the ReID feature amount of the person SP2 input from the feature extraction unit 37 is compared with the ReID feature amount of the management target TP stored in the feature management unit 38. When the two match (for example, when the similarity between the two is equal to or greater than a threshold value), the exit management unit 39 detects the exit of the management target TP.

[0034] 2. Second Embodiment 2-1. Features of the Second Embodiment FIG. 5 is a diagram for explaining the features of the second embodiment of the present disclosure. In the first embodiment, face authentication processing using the face image IMF_PS1 and the face image IMF_PT was performed, and as a result, entry of the management target TP into the predetermined area 20 was detected. However, if the face image IMF_PS1 is unclear, the face authentication processing may not be performed correctly. Then, even though the management target TP has entered the predetermined area 20, this entry may not be detected. Also, if the face authentication processing is not performed correctly, the ReID feature amount of the management target TP based on the collation result of the face authentication processing is not specified. Therefore, the situation where the exit of the management target TP from the predetermined area 20 is not detected occurs.

[0035] Therefore, in the second embodiment, a camera 23 different from the camera 22 is used as a sub-camera, and the face image IMF_PU of the unauthenticated person PU is acquired from the camera images constituting the video VD2 obtained by the camera 23. Similar to the camera 22, the camera 23 is provided inside the predetermined area 20. The pointing direction of the camera 23 is inside the predetermined area 20. A part of the shooting range of the camera 23 may overlap with the shooting range of the camera 22. The total number of cameras 23 is at least 1.

[0036] The unauthenticated person PU is a person PS1 who was not authenticated in the face authentication processing using the camera images constituting the video VD1. The identification of the unauthenticated person PU is performed by the person re-identification process. In this re-identification process, the ReID feature amount of the person PS1 extracted based on the camera images constituting the video VD1 is compared with that of an arbitrary person PS3 extracted based on the camera images constituting the video VD2. When the two match (for example, when the similarity between the two is equal to or greater than the threshold), the person PS1 is determined to be the same person as the person PS3.

[0037] When the person PS1 corresponds to the unauthenticated person PU and it is determined that the person PS1 is the same person as the person PS3, the person PS3 corresponds to the unauthenticated person PU. Therefore, in the second embodiment, face authentication processing is performed using the face image IMF_PS3 of the person PS3 as the face image IMF_PU. In this face authentication processing, the face image IMF_PU and the face image IMF_PT are compared. In this way, in the second embodiment, additional re-identification processing using the camera images constituting the video VD2 and additional face authentication processing are performed.

[0038] In the second embodiment, also, when it is detected as a result of the additional face authentication processing that the management target TP has entered the predetermined area 20, the full-body image IMB_PT is specified based on the camera images constituting the video VD2 at the time of this detection (for example, the time before or after the entry of the management target TP is detected). Then, the ReID feature amount of the management target PT is extracted from this full-body image IMB_PT, and person re-identification processing is performed. This re-identification processing is the same as the processing performed in the first embodiment.

[0039] In this way, according to the second embodiment, additional re-identification processing using the camera images constituting the video VD2, additional face authentication processing, and specification of the full-body image IMB_PT necessary for re-identification processing for detecting the exit of the management target TP are performed. Therefore, even when the face authentication processing of the management target TP using the camera images constituting the video VD1 has failed, it is possible to detect the entry and exit of the management target PT to and from the predetermined area 20.

[0040] 2-2. Example of functional configuration FIG. 6 is a block diagram showing a functional configuration example of the management device 10 related to the second embodiment. In the example shown in FIG. 6, in addition to the person detection units 31 to 38 and the feature amount management unit 37 shown in FIG. 4, the management device 10 includes an unauthenticated person identification unit 41. The departure detection unit 39 is omitted for convenience of explanation. The difference between FIGS. 4 and 6 is that the unauthenticated person identification unit 41 is added, and the videos VD1 and VD2 are input to the person detection unit 31. However, various processes such as face authentication processing and tracking processing using the video VD2 are basically the same as the various processes using the video VD1 described in FIG. 4. Therefore, hereinafter, functions particularly related to the second embodiment will be described.

[0041] The collation result from the face authentication unit 33 is input to the entry detection unit 35. The entry detection unit 35 detects the entry of the TP to be managed based on the determination information included in the collation result. So far, it is the same as the first embodiment. In the second embodiment, when the face image IMF_TP that matches the face image IMF_SP1 is not present in the face image management unit 34, information indicating that the person SP1 corresponds to the unauthenticated person PU is added to the determination information of the face image IMF_SP1. When the information of this unauthenticated person PU is included in the determination information, the entry detection unit 35 outputs the information of this unauthenticated person PU to the feature amount management unit 38 together with the attached information included in the collation result.

[0042] Based on the input information from the entry detection unit 35, the feature amount management unit 38 specifies the ReID feature amounts corresponding to the ReID feature amounts of the unauthenticated person PU among the ReID feature amounts included in the input information from the feature amount extraction unit 37. The specification of the ReID feature amounts of the unauthenticated person PU can be performed using, for example, the attached information included in the collation result, the coordinate information of the bounding box that is the target of extraction of the ReID feature amounts, and the timestamp information. The information on the specified ReID feature amounts of the unauthenticated person PU is stored in the storage device 12.

[0043] The unauthenticated person identification unit 41 performs re-identification processing of the unauthenticated person PU using the ReID feature amount of the unauthenticated person PU stored in the feature amount management unit 38 (storage device 12) and the ReID feature amount of the person PS3 stored in the feature amount management unit 38. In this re-identification processing, the ReID feature amount of the unauthenticated person PU and the ReID feature amount of the person PS3 are compared. When both match, the unauthenticated person identification unit 41 determines that the person PS3 corresponds to the unauthenticated person PU. Then, the unauthenticated person identification unit 41 outputs a command for face authentication processing using the face image IMF_PS3 to the face authentication unit 33.

[0044] When a command for face authentication processing is input, the face authentication unit 33 performs face authentication processing using the face image IMF_PS3 (that is, the face image IMF_TU) and the face image of the management target TP (that is, the face image IMF_TP) stored in the face image management unit 34 (for example, the storage device 12). In the face authentication processing, the face image IMF_SP3 and the face image IMF_TP are collated. When both match, it is determined that the person PS3 is the same person as the management target TP.

[0045] 3. Third Embodiment 3-1. Features of the Third Embodiment In the first embodiment, the ReID feature amount of the management target TP is extracted from the camera images constituting the video VD1 when entering the predetermined area 20 of the management target TP, and the ReID feature amount of any person PS2 leaving the predetermined area 20 is extracted from the camera images constituting the video VD1 after the management target TP enters the predetermined area 20. In the first embodiment, further, re-identification processing using the ReID feature amount of the management target TP and the ReID feature amount of the person PS2 is performed. Therefore, by comparing these ReID feature amounts, it is possible to detect that the management target PT has left the predetermined area 20.

[0046] However, even if the management target TP is the same person as the person PS2, the similarity of the ReID feature amount may be low. For example, when the management target TP changes clothes within the predetermined area 20, the clothes of the management target TP are different at the time of entry and exit. Then, it becomes a situation where it is impossible to detect that the management target PT has left the predetermined area 20.

[0047] Therefore, in the third embodiment, based on the ReID feature amount of the management target TP extracted from the camera images constituting the video VD1 and the ReID feature amount of the person PS3 extracted from the camera images constituting the video VD2, the re-identification process of the management target TP is performed. In this re-identification process, the ReID feature amount of the management target TP is compared with that of the person PS3. When both match (for example, when the similarity between the two is equal to or greater than the threshold value), the person PS3 is determined to be the same person as the management target TP. Since it is assumed that the similarity of the ReID feature amount becomes low, the threshold value used in the re-identification process of the management target TP may be set to a value lower than the threshold value of the re-identification process performed in the exit management of the first embodiment.

[0048] By performing such a re-identification process of the management target TP, it is possible to continuously identify the management target TP within the predetermined area 20. Therefore, even when the management target TP changes clothes within the predetermined area 20, it is possible to continuously track the management target TP within the predetermined area 20 and detect that the management target PT has exited the predetermined area 20.

[0049] 3-2. Functional Configuration Example FIG. 7 is a block diagram showing a functional configuration example of the management device 10 related to the third embodiment. In the example shown in FIG. 7, in addition to the person detection unit 31 to the exit detection unit 39 shown in FIG. 4, the management target tracking unit 51 is provided in the management device 10. The difference between FIG. 4 and FIG. 7 is that the management target tracking unit 51 is added and the videos VD1 and VD2 are input to the person detection unit 31. However, various processes such as face authentication processing and tracking processing using the video VD2 are basically the same as the various processes using the video VD1 described in FIG. 4. Therefore, hereinafter, the functions particularly related to the third embodiment will be described.

[0050] The information on the ReID feature amount of the person SP3 is input from the feature amount extraction unit 37 to the management target tracking unit 51. The management target tracking unit 51 performs re-identification processing of the management target TP using the input information from the feature amount extraction unit 37 and the ReID feature amount of the management target TP stored in the feature amount management unit 38 (storage device 12). In this re-identification processing, the ReID feature amount of the person SP3 input from the feature amount extraction unit 37 is compared with the ReID feature amount of the management target TP stored in the feature amount management unit 38. When the two match, it is determined that the person PS3 is the same person as the management target TP. When it is determined that the person PS3 is the same person as the management target TP, the management target tracking unit 51 outputs this determination information to the feature amount management unit 38.

[0051] The determination information from the management target tracking unit 51 is input to the feature amount management unit 38. The feature amount management unit 38 stores the input information from the management target tracking unit 51 in the storage device 12. When it is determined that the person PS3 is the same person as the management target TP, the ReID feature amount of the person SP3 is specified as the ReID feature amount of the management target TP and stored in the storage device 12.

[0052] The information on the ReID feature amount of the person SP2 is input from the feature amount extraction unit 37 to the exit management unit 39. The exit management unit 39 performs re-identification processing of the management target TP using the input information from the feature amount extraction unit 37 and the ReID feature amount of the management target TP stored in the feature amount management unit 38 (storage device 12). In this re-identification processing, the ReID feature amount of the person SP2 input from the feature amount extraction unit 37 is compared with the ReID feature amount of the management target TP stored in the feature amount management unit 38. When it is determined that the person PS3 is the same person as the management target TP, the information on the ReID feature amount of the management target TP stored in the feature amount management unit 38 is updated by the ReID feature amount of the person SP3.

[0053] 4. Fourth Embodiment FIG. 8 is a diagram for explaining the features of the fourth embodiment of the present disclosure. In the third embodiment, in order to identify the TP to be managed within the predetermined area 20, the ReID feature amount of the person PS3 extracted based on the camera image constituting the video VD2 was extracted. Then, when the ReID feature amount of this person PS3 matches that of the TP to be managed extracted based on the camera image constituting the video VD1, it is determined that the person PS3 is the same person as the TP to be managed.

[0054] However, when there are many candidates having the possibility of being the same person as the TP to be managed, it becomes difficult to identify the TP to be managed by the re-identification process of the TP to be managed within the predetermined area 20. In particular, in facilities such as nursery facilities and educational facilities, it is expected that the ages of the person P3 and the TP to be managed are low. Then, there is a possibility that the person PS3 may be misjudged as being the same person as the TP to be managed.

[0055] Therefore, in the fourth embodiment, the possession OB (for example, a personal locker) of the TP to be managed installed within the predetermined area 20 is photographed by the camera 23. By including the installation location of the possession OB in the shooting range of the camera 23, the image of the possession OB is included in the camera image IMG_VD2 constituting the video VD2 acquired by the camera 23. In the fourth embodiment, when the image of the person P3 is included in this camera image IMG_VD2, the distance between the representative coordinates of the image of the person P3 and the representative coordinates (known) of the image of the possession OB is calculated. Then, when the distance between the representative coordinates is within a predetermined distance, it is presumed that the person P3 photographed by the camera 23 is the same person as the TP to be managed.

[0056] In this way, in the fourth embodiment, the estimation process of the person P3 is performed based on the distance between the representative coordinates of the image of the possession OB on the camera image IMG_VD2 and the representative coordinates of the image of the person P3. When the estimation of the person P3 is performed, the candidates having the possibility of being the same person as the TP to be managed are narrowed down. Therefore, by adding the result of this estimation process to the information of the ReID feature amount of the person SP3, it is possible to suppress the misjudgment that the person PS3 is the same person as the TP to be managed.

Description of Signs

[0057] 10…Management device, 11…Processor, 12…Memory device, 20…Predetermined area, 21…Entrance / exit, 22, 23…Cameras, PS1, PS2, PS3…Any person, PT…Object to be managed, PU…Unidentified person, VD1, VD2…Videos, IMF_PS1, IMF_PS3…Facial images, IMB_PS1, IMB_PS2, IMB_PS2…Full body images

Claims

1. A method for managing the entry and exit of a person to be managed for a predetermined area having one entrance / exit, comprising: obtaining a face image and a full-body image of any person passing through the entrance / exit by using a camera image captured by one camera provided near the entrance / exit; performing a face authentication process using the face image of the any person included in the camera image and the registered face image of the person to be managed, and detecting the entry of the person to be managed into the predetermined area; extracting a feature amount of the person to be managed by using the full-body image of the person to be managed, in which the entry into the predetermined area has been detected by the face authentication process, among the full-body images of the any person included in the camera image; extracting a feature amount of the any person by using the full-body image of the any person included in the camera image after the entry of the person to be managed into the predetermined area; performing a re-identification process using the feature amount of the person to be managed extracted based on the camera image and the feature amount of the any person extracted based on the camera image, and detecting the exit of the person to be managed from the predetermined area; An entry / exit management method, characterized by including the above steps.

2. The method according to claim 1, further comprising: obtaining a full-body image of any person existing inside the predetermined area by using a sub-camera image captured by at least one sub-camera provided inside the predetermined area; extracting a feature amount of the any person by using the full-body image of the any person included in the sub-camera image; extracting a feature amount of an unauthenticated person, which indicates a person not authenticated in the face authentication process, by using the full-body image of the unauthenticated person among the full-body images of the any person included in the camera image; performing a re-identification process using the feature amount of any person extracted based on the sub-camera image and the feature amount of the unauthenticated person extracted based on the camera image, and identifying the unauthenticated person existing inside the predetermined area; when the unauthenticated person is identified, and when the face image of the identified unauthenticated person is included in the sub-camera image, performing an additional face authentication process using the face image of the unauthenticated person and the registered face image of the person to be managed; and further including the above steps. In the additional face authentication process, when the face image of the identified unauthenticated person matches the registered face image of the management target, entry of the management target into the predetermined area is detected. A method for managing entry and exit, characterized by the above.

3. The method according to claim 1, acquiring a full-body image of any person present inside the predetermined area using a sub-camera image captured by at least one sub-camera provided inside the predetermined area; extracting feature amounts of the arbitrary person using the full-body image of the arbitrary person included in the sub-camera image after the management target enters the predetermined area; performing re-identification processing using the feature amounts of the management target extracted based on the camera image and the feature amounts of the arbitrary person extracted based on the sub-camera image to identify the management target present inside the predetermined area; when the management target is identified, tracking the identified management target; A method for managing entry and exit, further comprising the above.

4. The method according to any one of claims 1 to 3, acquiring a full-body image of any person present inside the predetermined area using a sub-camera image captured by at least one sub-camera provided inside the predetermined area; estimating the arbitrary person based on the coordinates on the sub-camera image of the image of the arbitrary person included in the sub-camera image after the management target enters the predetermined area; further comprising, the shooting range of the sub-camera includes the possessions of the management target installed inside or outside the predetermined area, when the distance from the coordinates of the image of the arbitrary person on the sub-camera image to the coordinates of the installation location of the possession in the sub-camera image is equal to or less than a predetermined distance, the arbitrary person is presumed to be the management target corresponding to the possession. A method for managing entry and exit, characterized by the above.

5. An apparatus for managing the entry and exit of a management target for a predetermined area having one entrance and exit, comprising a processor for performing various processes, wherein the processor, acquires a face image and a full-body image of any person passing through the entrance and exit using a camera image captured by one camera provided near the entrance and exit. Perform face authentication processing using the face image of the arbitrary person included in the camera image and the registered face image of the management target to detect entry of the management target into the predetermined area. Extract the feature amount of the management target using the full body image of the management target in which entry into the predetermined area has been detected by the face authentication processing among the full body images of the arbitrary person included in the camera image. Extract the feature amount of the arbitrary person using the full body image of the arbitrary person included in the camera image after entry of the management target into the predetermined area. Perform re-identification processing using the extracted feature amount of the management target and the extracted feature amount of the arbitrary person to detect exit of the management target from the predetermined area. An entry / exit management device characterized by being configured as described above.

Images