Management System

Abstract

To efficiently achieve face authentication and person re-identification using a camera.SOLUTION: A management system performs person re-identification processing on the basis of an image captured by a camera. When predetermined conditions are satisfied, the management system permits face authentication processing to authenticate a target on the basis of an image captured by the same camera. The predetermined conditions include at least one of a condition that a distance between the camera and the target within a camera view angle is less than a predetermined threshold value and a condition that the target in the image can be zoomed in.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to technology for performing face identification and person re-identification using a camera. [Background technology]

[0002] Patent Documents 1, 2, and 3 disclose face recognition technologies based on images captured by a camera.

[0003] Patent Document 4 discloses a person re-identification technology based on an image captured by a camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-152490 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-024498 [Patent Document 3] Japanese Patent Application Publication No. 2019-219721 [Patent Document 4] Japanese Patent Publication No. 2022-059972 Summary of the Invention [Problem to be solved by the invention]

[0005] Person re-identification is a technology that identifies and tracks the same person from multiple images captured by one or more cameras. A person is detected from the images that make up the video, and overall features of the detected person are extracted. Person re-identification is performed based on the extracted features. The feature extraction is performed based on a person re-identification model based on machine learning.

[0006] On the other hand, facial recognition is performed based on an image of the face rather than the entire person, and in order to perform facial recognition with high accuracy, it is desirable to obtain a high-resolution image of the face.

[0007] As described above, the types of images suitable for person re-identification and face authentication differ. However, it is inefficient to provide separate cameras for person re-identification and face authentication.

[0008] One object of the present disclosure is to provide a technology that can efficiently realize face authentication and person re-identification using a camera. [Means for solving the problem]

[0009] One aspect of the present disclosure relates to a management system that performs person re-identification processing based on an image captured by a camera. The management system comprises one or more processors. If a predetermined condition is met, the one or more processors allow a facial recognition process to be performed to recognize a target based on an image captured by the camera. The predetermined conditions are: The distance between the camera and the target within the camera's angle of view is less than a predetermined threshold; The ability to zoom in on targets in the image It includes at least one of the following. [Effects of the Invention]

[0010] According to the present disclosure, the camera used for the person re-identification process is also used for the face authentication process. Therefore, compared to a case where a camera dedicated to the face authentication process and a camera dedicated to the person re-identification process are separately provided, the face authentication process and the person re-identification process can be realized more efficiently. In addition, the installation cost of the camera is reduced. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram for explaining an overview of a management system according to an embodiment; [Figure 2] 1 is a block diagram illustrating an example of a configuration of a management system according to an embodiment. [Figure 3] FIG. 10 is a conceptual diagram for explaining a comparative example. [Figure 4] FIG. 1 is a conceptual diagram illustrating a management system according to an embodiment. [Figure 5] FIG. 10 is a conceptual diagram for explaining an example of face authentication processing using a common camera according to an embodiment. [Figure 6] FIG. 10 is a conceptual diagram for explaining another example of face authentication processing using the common camera according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0013] 1. Overview of the management system FIG. 1 is a conceptual diagram for explaining an overview of a management system 100 according to this embodiment. The management system 100 manages a predetermined area. Examples of the predetermined area include a city, a building, etc. One or more cameras 10 are installed in the predetermined area. The management system 100 acquires and uses images (video) IMG captured by the one or more cameras 10.

[0014] For example, the management system 100 performs "person re-identification processing." In the following description, the person re-identification processing is referred to as "ReID processing." The ReID processing is a technology for identifying and tracking the same person from multiple images IMG captured by one or multiple cameras 10. A person is detected from the image IMG, ReID features of the entire detected person are extracted, and person re-identification is performed based on the extracted ReID features. The ReID features are extracted based on a person re-identification model based on machine learning. This ReID processing makes it possible to track the same person across multiple cameras 10. The tracking result of the same person is also called a "track." A track can also be said to be a set of bounding boxes representing the same person over multiple time steps. Note that the ReID processing is a well-known technology, and the method used is not particularly limited.

[0015] The management system 100 also performs "face identification processing." Face recognition processing is a technology for authenticating (identifying) an individual based on a facial image. The facial image of the person to be authenticated is registered in advance. The management system 100 performs face recognition processing based on the facial image of the person registered in advance, thereby identifying the individual appearing in the image IMG. Note that such face recognition processing is a well-known technology, and the method used is not particularly limited.

[0016] By combining face recognition processing and ReID processing, it is possible to track a specific target T within a predetermined area. For example, at a certain timing, the target T is individually identified by face recognition processing. The individually identified target T is associated with a track used in the ReID processing. Then, the specific target T is tracked by the ReID processing.

[0017] The tracking results of a specific target T can be used for various services. For example, an image IMG (video) showing the specific target T can be provided to the user. This can be used for a monitoring service that allows parents to keep an eye on their children.

[0018] FIG. 2 is a block diagram showing an example of the configuration of a management system 100 according to this embodiment. The management system 100 includes one or more processors 110 (hereinafter simply referred to as "processors 110"), one or more storage devices 120 (hereinafter simply referred to as "storage devices 120"), and an interface 130. The processor 110 executes various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information required for the processes. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory. The interface 130 includes a network interface and a user interface. Examples of the user interface include a display device, a touch panel, a keyboard, and buttons.

[0019] The management program 200 is a computer program for managing a predetermined area. The management program 200 is stored in the storage device 120. The management program 200 may be recorded on a computer-readable recording medium. The management program 200 is executed by the processor 110. The functions of the management system 100 are realized by cooperation between the processor 110 executing the management program 200 and the storage device 120.

[0020] The processor 110 communicates with one or more cameras 10 via the interface 130. The processor 110 acquires images IMG taken by one or more cameras 10.

[0021] The processor 110 may also communicate with a terminal 20 carried by the target T via the interface 130. Examples of the terminal 20 include a smartphone and a tablet. For example, the processor 110 communicates with the terminal 20 and acquires location information of the terminal 20. The location information of the terminal 20 can be obtained by using a global navigation satellite system (GNSS) or the like.

[0022] The storage device 120 stores camera information 210, an image database 220, a person database 230, and the like.

[0023] The camera information 210 indicates the camera ID, camera position, camera direction, angle of view, etc. for each camera 10 installed in a predetermined area. The camera ID is identification information for the camera 10. The information on the camera position, camera direction, and angle of view is given in advance or provided by the camera 10.

[0024] The image database 220 is a database of images IMG obtained from one or more cameras 10. For example, the image database 220 associates an image IMG with the camera ID of the camera 10 that captured the image IMG.

[0025] The person database 230 indicates various personal information for each target T. For example, the personal information for each target T includes a person ID, a facial image, a terminal ID, location information, a track ID, ReID information, etc. The person ID is identification information of the target T. The person ID and the target T are associated by face recognition processing. The facial image is a facial image of the target T, is registered in advance, and is used in face recognition processing. The terminal ID is identification information of the terminal 20 possessed by the target T. The location information is location information (absolute location information) of the target T. In this embodiment, the location of the terminal 20 is considered to be the location of the target T. Therefore, the location information can be acquired by communicating with the terminal 20. The track ID is identification information of a track indicating the tracking result of the target T. The ReID information is information related to the ReID processing for the target T. For example, the ReID information includes ReID features of the target T, identification information of an image IMG in which the target T appears, the camera ID of the camera 10 that captured the image IMG, the position of the target T in the image IMG (intra-image position), etc.

[0026] The image database 220 and the person database 230 may be in any form and are not particularly limited.

[0027] 2. Issues regarding the combination of face recognition processing and ReID processing The combination of face authentication processing and ReID processing will be examined in more detail below. First, a comparative example will be described with reference to FIG.

[0028] In the comparative example, a camera 10-A dedicated to face recognition processing and a camera 10-B dedicated to ReID processing are separately provided. Face recognition processing is performed based on an image of the face portion rather than the entire person. To perform face recognition processing with high accuracy, it is desirable to obtain a high-resolution image of the face portion. Therefore, the camera 10-A dedicated to face recognition processing is installed in a position where a person can approach the camera. On the other hand, ReID processing generally uses an image of the entire person (whole body). Therefore, the camera 10-B dedicated to ReID processing is installed in a position where it can capture a wide range. For example, the camera 10-B dedicated to ReID processing is installed at a height of several meters.

[0029] As described above, the types of images suitable for use in the person re-identification process and the face authentication process are different. Therefore, in the comparative example, a camera 10-A dedicated to the face authentication process and a camera 10-B dedicated to the ReID process are separately provided.

[0030] In the example shown in FIG. 3, a camera 10-A dedicated to face recognition processing is installed at the entrance to a predetermined area. The camera 10-A captures an image of the face of the target T and its surroundings, and transmits the image IMG-A to the management system 100. The management system 100 performs face recognition processing on the target T based on the image IMG-A captured by the camera 10-A. The target T then moves within the predetermined area. A camera 10-B dedicated to ReID processing is installed within the predetermined area. The management system 100 performs ReID processing based on the image IMG-B captured by the camera 10-B, and tracks the target T.

[0031] However, there is a possibility that tracking of target T may be interrupted midway. Also, there is a possibility that tracking of target T may be mistakenly switched to tracking of another person (ID switching). That is, the association between the person ID and the track ID may disappear, or the association between the person ID and the track ID may change to an incorrect one. In order to restore normal tracking of target T, it is necessary to perform face recognition processing on target T again and restore the association between the person ID and the track ID to normal. However, it is inefficient to have target T return to the entrance of a specified area to perform face recognition processing again.

[0032] To enable face recognition processing in various locations within a predetermined area, it is conceivable to install multiple cameras 10-A dedicated to face recognition processing in various locations within the predetermined area. However, installing multiple cameras 10-A dedicated to face recognition processing separately from multiple cameras 10-B dedicated to ReID processing is also inefficient. Furthermore, the installation costs would be enormous.

[0033] Therefore, this embodiment proposes a technique that can more efficiently realize face authentication processing and person re-identification processing.

[0034] 3. Use of a common camera FIG. 4 is a conceptual diagram illustrating a management system 100 according to this embodiment. According to this embodiment, a "common camera 10-C" that can be used for both face authentication processing and ReID processing is installed in a predetermined area. In other words, one common camera 10-C is used for both face authentication processing and ReID processing. In addition to the common camera 10-C, a camera 10-A and a camera 10-B may also be installed.

[0035] As described above, an image of the entire person (whole body) is used in the ReID process. Therefore, the common camera 10-C is basically installed in a position that allows it to capture a wide range of images, similar to the camera 10-B. The management system 100 communicates with the common camera 10-C and acquires the image IMG-C captured by the common camera 10-C. Basically, the management system 100 performs the ReID process based on the image IMG-C captured by the common camera 10-C. However, if a "predetermined condition" is met, the management system 100 permits the performance of face recognition processing based on the image IMG-C captured by the common camera 10-C.

[0036] The predetermined conditions for permitting the face recognition process are conditions that allow the face recognition process to be performed with high accuracy even when the common camera 10-C is used. Specifically, the predetermined conditions include at least one of the following first and second conditions. The predetermined conditions may be a combination of the first and second conditions. <First condition> The distance D between the common camera 10-C and the target T within the angle of view of the common camera 10-C is less than a predetermined threshold. The predetermined threshold is the maximum distance required to acquire a high-resolution image of the face of a typical person. <Second Condition> It is possible to zoom in on the target T shown in the image IMG-C captured by the common camera 10-C.

[0037] FIG. 5 is a conceptual diagram for explaining an example of face authentication processing when the first condition is met. The camera position, camera direction, and angle of view of the common camera 10-C are obtained from the camera information 210. Position information of the target T is obtained from the terminal 20 of the target T or from the person database 230. The management system 100 determines whether the target T is present within the angle of view of the common camera 10-C based on this information. If the target T is present within the angle of view of the common camera 10-C, the management system 100 calculates the distance D between the common camera 10-C and the target T. Then, the management system 100 determines whether the first condition is met by comparing the distance D with a predetermined threshold.

[0038] The upper part (A) of FIG. 5 shows a situation where the first condition is met. For example, this situation may occur when the target T happens to pass very close to the common camera 10-C. The distance D between the common camera 10-C and the target T is less than a predetermined threshold. The image IMG-C captured by the common camera 10-C shows the target T's face in a sufficiently large size. Therefore, the management system 100 can accurately perform face recognition processing on the target T based on the image IMG-C.

[0039] In the lower part (B) of Fig. 5, the target T is located at a position away from the common camera 10-C. Even in this case, the first condition can be met by prompting the target T to approach the common camera 10-C through the terminal 20. Therefore, the management system 100 provides the position information of the common camera 10-C to the terminal 20 of the target T and requests the target T to approach the common camera 10-C.

[0040] More specifically, the management system 100 transmits request information REQ to the terminal 20 of the target T, prompting the target T to approach the common camera 10-C. The request information REQ includes position information of the common camera 10-C. For example, the position information of the common camera 10-C is absolute position information of the common camera 10-C. The absolute position information of the common camera 10-C is obtained from the camera information 210. As another example, the position information of the common camera 10-C may be relative position information between the target T and the common camera 10-C. The relative position information is calculated from the absolute position information of the common camera 10-C and the absolute position information of the target T.

[0041] The terminal 20 presents the request information REQ received from the management system 100 to the target T. Typically, the terminal 20 displays the request information REQ on a display device. At this time, the terminal 20 may display a map on the display device, and indicate the current position of the target T and the position of the common camera 10-C on the map. Upon seeing the request information REQ, the target T approaches the common camera 10-C in accordance with the request information REQ. When the distance D between the common camera 10-C and the target T becomes less than a predetermined threshold, the first condition is met. This enables the management system 100 to perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0042] FIG. 6 is a conceptual diagram for explaining an example of face recognition processing when the second condition is met. The target T is located at a position away from the common camera 10-C. Therefore, the target T shown in the image IMG-C captured by the common camera 10-C is small. Even in this case, if the position within the image, that is, the position of the target T in the image IMG-C, can be identified, it is possible to zoom in on the target T. If it is possible to zoom in on the target T in the image IMG-C, it becomes possible to perform face recognition processing with high accuracy.

[0043] In the example shown in the upper part (A) of FIG. 6, the management system 100 acquires position information of the target T. The position information of the target T is obtained from the terminal 20 of the target T or the person database 230. The camera position, camera direction, and angle of view of the common camera 10-C are obtained from the camera information 210. The management system 100 determines whether the target T is present within the angle of view of the common camera 10-C based on this information. If the target T is present within the angle of view of the common camera 10-C, the management system 100 calculates the in-image position of the target T in the image IMG-C based on the position information of the target T. Then, the management system 100 zooms in on the target T appearing in the image IMG-C based on the in-image position. For example, the management system 100 remotely controls the common camera 10-C to zoom in on the target T. This enables the management system 100 to perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0044] In the example shown in the lower part (B) of FIG. 6 , the management system 100 communicates with the terminal 20 of the target T and requests the target T to perform a predetermined action. Examples of the predetermined action include raising a hand or jumping in place. The management system 100 transmits request information REQ to the terminal 20 of the target T, prompting the target T to perform the predetermined action. The terminal 20 presents the request information REQ received from the management system 100 to the target T. Typically, the terminal 20 displays the request information REQ on a display device. Upon seeing the request information REQ, the target T performs the predetermined action in accordance with the request information REQ. After transmitting the request information REQ, the management system 100 monitors an image IMG-C captured by the common camera 10-C. The management system 100 then identifies a person performing the predetermined action in the image IMG-C. For example, the management system 100 identifies a person performing the predetermined action within a certain time after transmitting the request information REQ. Identifying a person performing the predetermined action can be achieved, for example, by well-known person detection processing and posture estimation processing. The management system 100 determines that the person performing the specified action identified in this manner is target T. Once target T is identified, the position of target T in image IMG-C can be acquired. Then, based on the position in the image, the management system 100 zooms in on target T appearing in image IMG-C. For example, the management system 100 remotely controls the common camera 10-C to zoom in on target T. This enables the management system 100 to perform face recognition processing on target T with high accuracy based on image IMG-C.

[0045] As a modified example, the target T may perform a predetermined operation on its own initiative. For example, the target T may recognize that the monitoring service has been interrupted and perform a predetermined operation on its own initiative.

[0046] The management system 100 may estimate the posture of the target T from the whole-body image of the target T by a well-known posture estimation process. In this case, the management system 100 can acquire the position of the face in the whole-body image of the target T, and therefore can also zoom in on the face of the target T. This enables the management system 100 to perform face recognition processing with even greater accuracy.

[0047] After the face authentication process is completed, the management system 100 cancels the zoom-in and returns the common camera 10-C to the default state.

[0048] Effects As described above, according to this embodiment, the common camera 10-C is used for both the face authentication process and the ReID process. Therefore, compared to the case where a camera 10-A dedicated to the face authentication process and a camera 10-B dedicated to the ReID process are separately provided, the face authentication process and the ReID process can be realized more efficiently. Furthermore, the installation cost of the camera 10 is also reduced.

[0049] Furthermore, as described above, there is a possibility that the association between the person ID and the track ID will disappear or will change to an incorrect one. In order to restore normal tracking of the target T, it is necessary to perform face recognition processing on the target T again and restore the association between the person ID and the track ID to normal. In the comparative example shown in FIG. 3, it was necessary to have the target T return to the entrance of the specified area to perform face recognition processing again. On the other hand, according to this embodiment, the target T does not need to return to the entrance of the specified area. By using the common camera 10-C near the current position of the target T, it is possible to quickly perform face recognition processing on the target T again. In other words, it is possible to quickly restore the association between the person ID and the track ID to normal. This also contributes to efficiency. [Explanation of symbols]

[0050] 10 Camera 10-C Common Camera 100 Management Systems IMG image T Target

Claims

1. A management system that performs person re-identification processing based on an image captured by a camera, one or more processors; When a predetermined condition is met, the one or more processors permit a facial recognition process to be performed to recognize a target based on the image captured by the camera; the predetermined condition includes a distance between the camera and the target within an angle of view of the camera being less than a predetermined threshold; The one or more processors provide the position information of the camera to the target terminal and request the target to move closer to the camera. Management system.

2. A management system that performs person re-identification processing based on an image captured by a camera, one or more processors; When a predetermined condition is met, the one or more processors permit a facial recognition process to be performed to recognize a target based on the image captured by the camera; the predetermined condition includes being able to zoom in on the target shown in the image; the one or more processors: Communicating with the target terminal and requesting the target to perform a predetermined action; Identifying a person performing the predetermined action in the image; determining that the person performing the predetermined action is the target; Zooming in on the target shown in the image and performing the face recognition process Management system.

Images