Management system

A common camera system efficiently performs face authentication and person re-identification by meeting specific conditions, addressing inefficiencies and cost issues in separate camera setups.

JP2024160495AActive Publication Date: 2024-11-14TOYOTA JIDOSHA KK

Patent Information

Application Number
JP2023075553
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14
Estimated Expiration
2043-05-01

AI Technical Summary

Technical Problem

Existing systems require separate cameras for face authentication and person re-identification, leading to inefficiencies and increased costs due to differing image types needed for each process.

Method used

A management system that uses a common camera for both face authentication and person re-identification, allowing face authentication processing when certain conditions are met, such as proximity or zoom capabilities, to enhance efficiency and reduce installation costs.

Benefits of technology

Enables efficient and cost-effective implementation of both face authentication and person re-identification processes using a single camera, reducing the need for multiple camera installations and minimizing disruptions in tracking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160495000001_ABST
    Figure 2024160495000001_ABST
Patent Text Reader

Abstract

To efficiently achieve face authentication and person re-identification using a camera.SOLUTION: A management system performs person re-identification processing on the basis of an image captured by a camera. When predetermined conditions are satisfied, the management system permits face authentication processing to authenticate a target on the basis of an image captured by the same camera. The predetermined conditions include at least one of a condition that a distance between the camera and the target within a camera view angle is less than a predetermined threshold value and a condition that the target in the image can be zoomed in.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to technology for performing face identification and person re-identification using a camera. [Background technology]

[0002] Patent Documents 1, 2, and 3 disclose face recognition technologies based on images captured by a camera.

[0003] Patent Document 4 discloses a person re-identification technology based on an image captured by a camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-152490 A [Patent Document 2] JP 2016-024498 A [Patent Document 3] JP 2019-219721 A [Patent Document 4] Patent Publication No. 2022-059972 Summary of the Invention [Problem to be solved by the invention]

[0005] Person re-identification is a technology that identifies and tracks the same person from multiple images captured by one or more cameras. A person is detected from the images that make up the video, and the overall features of the detected person are extracted, and person re-identification is performed based on the extracted features. The feature extraction is performed based on a person re-identification model based on machine learning.

[0006] On the other hand, face recognition is performed based on an image of only the face, not the entire person. To perform face recognition with high accuracy, it is desirable to obtain a high-resolution image of the face.

[0007] In this way, the types of images suitable for use in person re-identification and face recognition are different. However, it is inefficient to provide separate cameras for person re-identification and face recognition.

[0008] One objective of the present disclosure is to provide a technology that can efficiently realize face authentication and person re-identification using a camera. [Means for solving the problem]

[0009] One aspect of the present disclosure relates to a management system that performs person re-identification processing based on an image captured by a camera. The management system comprises one or more processors. If a predetermined condition is met, the one or more processors allow a facial recognition process to be performed to recognize a target based on an image captured by the camera. The predetermined conditions are: the distance between the camera and a target within the camera's angle of view is less than a predetermined threshold; Ability to zoom in on targets in the image At least one of the following is included. Effect of the Invention

[0010] According to the present disclosure, the camera used for the person re-identification process is also used for the face authentication process. Therefore, compared with the case where a camera dedicated to the face authentication process and a camera dedicated to the person re-identification process are separately provided, it is possible to realize the face authentication process and the person re-identification process more efficiently. In addition, the installation cost of the camera is also reduced. [Brief description of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram for explaining an overview of a management system according to an embodiment; [Diagram 2] 1 is a block diagram showing an example of a configuration of a management system according to an embodiment; [Diagram 3] FIG. 13 is a conceptual diagram for explaining a comparative example. [Figure 4] FIG. 1 is a conceptual diagram for explaining a management system according to an embodiment. [Diagram 5] 1 is a conceptual diagram for explaining an example of face recognition processing using a common camera according to an embodiment. FIG. [Figure 6] 11 is a conceptual diagram for explaining another example of face recognition processing using the common camera according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0013] 1. Overview of the management system FIG. 1 is a conceptual diagram for explaining an overview of a management system 100 according to the present embodiment. The management system 100 manages a predetermined area. Examples of the predetermined area include a city, a building, and the like. One or more cameras 10 are installed in the predetermined area. The management system 100 acquires and uses images (video) IMG captured by the one or more cameras 10.

[0014] For example, the management system 100 performs "person re-identification processing". In the following description, the person re-identification processing is called "ReID processing". The ReID processing is a technology for identifying and tracking the same person from multiple images IMG captured by one or multiple cameras 10. A person is detected from the image IMG, ReID features of the entire detected person are extracted, and person re-identification is performed based on the extracted ReID features. The extraction of the ReID features is performed based on a person re-identification model based on machine learning. By such ReID processing, the same person can be tracked between multiple cameras 10. The tracking result of the same person is also called a "track". It can also be said that a track is a set of bounding boxes representing the same person in multiple time steps. Note that the ReID processing is a well-known technology, and the method thereof is not particularly limited.

[0015] Furthermore, the management system 100 performs "face identification processing." Face recognition processing is a technology for authenticating (identifying) an individual based on a face image. The face image of the person to be authenticated is registered in advance. The management system 100 performs face recognition processing based on the face image of the person registered in advance, thereby identifying the individual appearing in the image IMG. Note that such face recognition processing is a well-known technology, and the method thereof is not particularly limited.

[0016] By combining the face recognition process and the ReID process, a specific target T within a predetermined area can be tracked. For example, at a certain timing, the target T is individually identified by the face recognition process. The individually identified target T is associated with a track used in the ReID process. Then, the specific target T is tracked by the ReID process.

[0017] The tracking results of a specific target T can be used for various services. For example, an image IMG (video) showing the specific target T can be provided to a user. This can be used for a monitoring service that allows parents to monitor their children.

[0018] FIG. 2 is a block diagram showing a configuration example of a management system 100 according to the present embodiment. The management system 100 includes one or more processors 110 (hereinafter simply referred to as "processor 110"), one or more storage devices 120 (hereinafter simply referred to as "storage device 120"), and an interface 130. The processor 110 executes various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information required for the processes. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory. The interface 130 includes a network interface and a user interface. Examples of the user interface include a display device, a touch panel, a keyboard, and a button.

[0019] The management program 200 is a computer program for managing a predetermined area. The management program 200 is stored in the storage device 120. The management program 200 may be recorded in a computer-readable recording medium. The management program 200 is executed by the processor 110. The functions of the management system 100 are realized by cooperation between the processor 110 executing the management program 200 and the storage device 120.

[0020] The processor 110 communicates with one or more cameras 10 via an interface 130. The processor 110 acquires images IMG captured by the one or more cameras 10.

[0021] Furthermore, the processor 110 may communicate with a terminal 20 carried by the target T via the interface 130. Examples of the terminal 20 include a smartphone and a tablet. For example, the processor 110 communicates with the terminal 20 and acquires location information of the terminal 20. The location information of the terminal 20 can be obtained by using a Global Navigation Satellite System (GNSS) or the like.

[0022] The storage device 120 stores camera information 210, an image database 220, a person database 230, and the like.

[0023] The camera information 210 indicates the camera ID, camera position, camera direction, angle of view, etc. for each camera 10 installed in a specific area. The camera ID is identification information of the camera 10. The information on the camera position, camera direction, and angle of view is given in advance or provided by the camera 10.

[0024] The image database 220 is a database of images IMG obtained from one or more cameras 10. For example, the image database 220 indicates an image IMG in association with the camera ID of the camera 10 that captured the image IMG.

[0025] The person database 230 indicates various personal information for each target T. For example, the personal information of each target T includes a person ID, a face image, a terminal ID, position information, a track ID, ReID information, and the like. The person ID is identification information of the target T. The person ID and the target T are associated by face recognition processing. The face image is a face image of the target T, which is registered in advance and used in face recognition processing. The terminal ID is identification information of the terminal 20 held by the target T. The position information is position information (absolute position information) of the target T. In this embodiment, the position of the terminal 20 is considered to be the position of the target T. Therefore, the position information can be acquired by communicating with the terminal 20. The track ID is identification information of a track indicating the tracking result of the target T. The ReID information is information related to the ReID processing for the target T. For example, the ReID information includes a ReID feature amount of the target T, identification information of an image IMG in which the target T is shown, a camera ID of the camera 10 that captured the image IMG, the position of the target T in the image IMG (position within the image), and the like.

[0026] The image database 220 and the person database 230 may take any form and are not particularly limited.

[0027] 2. Issues regarding the combination of face recognition and ReID processing The combination of face authentication processing and ReID processing will be considered in more detail below. First, a comparative example will be described with reference to FIG.

[0028] In the comparative example, a camera 10-A dedicated to face recognition processing and a camera 10-B dedicated to ReID processing are separately provided. Face recognition processing is performed based on an image of the face portion, not of the entire person. To perform face recognition processing with high accuracy, it is desirable to obtain a high-resolution image of the face portion. Therefore, the camera 10-A dedicated to face recognition processing is installed at a position where a person can approach the camera. On the other hand, in ReID processing, an image of the entire person (whole body) is generally used. Therefore, the camera 10-B dedicated to ReID processing is installed at a position where it can capture a wide range. For example, the camera 10-B dedicated to ReID processing is installed at a height of several meters.

[0029] In this way, the types of images suitable for use in the person re-identification process and the face authentication process are different. Therefore, in the comparative example, a camera 10-A dedicated to the face authentication process and a camera 10-B dedicated to the ReID process are separately provided.

[0030] In the example shown in FIG. 3, a camera 10-A dedicated to face recognition processing is installed at the entrance of a predetermined area. The camera 10-A captures an image of the face of a target T and its surroundings, and transmits the image IMG-A to the management system 100. The management system 100 performs face recognition processing on the target T based on the image IMG-A captured by the camera 10-A. The target T then moves within the predetermined area. A camera 10-B dedicated to ReID processing is installed within the predetermined area. The management system 100 performs ReID processing based on the image IMG-B captured by the camera 10-B, and tracks the target T.

[0031] However, there is a possibility that the tracking of the target T will be interrupted midway. Also, there is a possibility that an event (ID switching) will occur in which the tracking of the target T is mistakenly switched to the tracking of another person. That is, the association between the person ID and the track ID may disappear, or the association between the person ID and the track ID may change to an incorrect one. In order to restore the tracking of the target T to normal, it is necessary to perform face recognition processing on the target T again and restore the association between the person ID and the track ID to normal. However, it is inefficient to have the target T return to the entrance of the specified area to perform face recognition processing again.

[0032] In order to enable face recognition processing in various places within a given area, it is conceivable to install a large number of cameras 10-A dedicated to face recognition processing in various places within the given area. However, installing a large number of cameras 10-A dedicated to face recognition processing separately from a large number of cameras 10-B dedicated to ReID processing is also inefficient. In addition, the installation cost will be enormous.

[0033] Therefore, this embodiment proposes a technique that can more efficiently realize face authentication processing and person re-identification processing.

[0034] 3. Use of common cameras 4 is a conceptual diagram for explaining a management system 100 according to the present embodiment. According to the present embodiment, a "common camera 10-C" that can be used for both face authentication processing and ReID processing is installed in a predetermined area. In other words, one common camera 10-C is used for both face authentication processing and ReID processing. In addition to the common camera 10-C, a camera 10-A and a camera 10-B may also be installed.

[0035] As described above, an image of the entire person (whole body) is used in the ReID process. Therefore, basically, the common camera 10-C is installed in a position where it can capture a wide range, similar to the camera 10-B. The management system 100 communicates with the common camera 10-C and acquires the image IMG-C captured by the common camera 10-C. Basically, the management system 100 performs the ReID process based on the image IMG-C captured by the common camera 10-C. However, if a "predetermined condition" is met, the management system 100 permits face recognition processing to be performed based on the image IMG-C captured by the common camera 10-C.

[0036] The predetermined condition for permitting the face recognition process is a condition that allows the face recognition process to be performed with high accuracy even when the common camera 10-C is used. Specifically, the predetermined condition includes at least one of the following first and second conditions. The predetermined condition may be a combination of the first and second conditions. <First condition> The distance D between the common camera 10-C and the target T within the angle of view of the common camera 10-C is less than a predetermined threshold. The predetermined threshold is the maximum distance required to capture a high-resolution image of the face of a typical person. <Second Condition> It is possible to zoom in on the target T shown in the image IMG-C captured by the common camera 10-C.

[0037] FIG. 5 is a conceptual diagram for explaining an example of face recognition processing when the first condition is satisfied. The camera position, camera direction, and angle of view of the common camera 10-C are obtained from the camera information 210. The position information of the target T is obtained from the terminal 20 of the target T or from the person database 230. The management system 100 determines whether or not the target T is present within the angle of view of the common camera 10-C based on this information. If the target T is present within the angle of view of the common camera 10-C, the management system 100 calculates the distance D between the common camera 10-C and the target T. Then, the management system 100 determines whether or not the first condition is satisfied by comparing the distance D with a predetermined threshold value.

[0038] The upper part (A) of FIG. 5 shows a situation where the first condition is met. For example, such a situation may occur when the target T happens to pass very close to the common camera 10-C. The distance D between the common camera 10-C and the target T is less than a predetermined threshold. The face of the target T is shown sufficiently large in the image IMG-C captured by the common camera 10-C. Therefore, the management system 100 can perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0039] In the lower part (B) of Fig. 5, the target T is located at a position away from the common camera 10-C. Even in this case, the first condition can be satisfied by urging the target T to approach the common camera 10-C through the terminal 20. Therefore, the management system 100 provides the position information of the common camera 10-C to the terminal 20 of the target T and requests the target T to approach the common camera 10-C.

[0040] More specifically, the management system 100 transmits request information REQ to the terminal 20 of the target T, which prompts the target T to approach the common camera 10-C. The request information REQ includes position information of the common camera 10-C. For example, the position information of the common camera 10-C is absolute position information of the common camera 10-C. The absolute position information of the common camera 10-C is obtained from the camera information 210. As another example, the position information of the common camera 10-C may be relative position information between the target T and the common camera 10-C. The relative position information is calculated from the absolute position information of the common camera 10-C and the absolute position information of the target T.

[0041] The terminal 20 presents the request information REQ received from the management system 100 to the target T. Typically, the terminal 20 displays the request information REQ on a display device. At this time, the terminal 20 may display a map on the display device and indicate the current position of the target T and the position of the common camera 10-C on the map. Upon seeing the request information REQ, the target T approaches the common camera 10-C in accordance with the request information REQ. When the distance D between the common camera 10-C and the target T becomes less than a predetermined threshold, the first condition is met. This enables the management system 100 to perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0042] 6 is a conceptual diagram for explaining an example of face recognition processing when the second condition is satisfied. The target T is located at a position away from the common camera 10-C. Therefore, the target T reflected in the image IMG-C captured by the common camera 10-C is small. Even in this case, if the position within the image, which is the position of the target T in the image IMG-C, can be identified, it is possible to zoom in on the target T. If it is possible to zoom in on the target T in the image IMG-C, it is possible to perform face recognition processing with high accuracy.

[0043] In the example shown in the upper part (A) of FIG. 6, the management system 100 acquires the position information of the target T. The position information of the target T is obtained from the terminal 20 of the target T or the person database 230. The camera position, camera direction, and angle of view of the common camera 10-C are obtained from the camera information 210. The management system 100 determines whether the target T is present within the angle of view of the common camera 10-C based on these pieces of information. If the target T is present within the angle of view of the common camera 10-C, the management system 100 calculates the in-image position of the target T in the image IMG-C based on the position information of the target T. Then, the management system 100 zooms in on the target T shown in the image IMG-C based on the in-image position. For example, the management system 100 remotely controls the common camera 10-C to zoom in on the target T. This allows the management system 100 to perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0044] In the example shown in the lower part (B) of FIG. 6, the management system 100 communicates with the terminal 20 of the target T and requests the target T to perform a predetermined action. Examples of the predetermined action include raising a hand and jumping on the spot. The management system 100 transmits request information REQ to the terminal 20 of the target T, which prompts the target T to perform a predetermined action. The terminal 20 presents the request information REQ received from the management system 100 to the target T. Typically, the terminal 20 displays the request information REQ on a display device. Upon seeing the request information REQ, the target T performs a predetermined action according to the request information REQ. After transmitting the request information REQ, the management system 100 monitors an image IMG-C captured by the common camera 10-C. Then, the management system 100 identifies a person performing a predetermined action in the image IMG-C. For example, the management system 100 identifies a person performing a predetermined action within a certain time after transmitting the request information REQ. Identification of a person performing a predetermined action is possible, for example, by well-known person detection processing and posture estimation processing. The management system 100 determines that the person performing the specified action identified in this manner is the target T. When the target T is identified, the in-image position of the target T in the image IMG-C can be acquired. Then, the management system 100 zooms in on the target T appearing in the image IMG-C based on the in-image position. For example, the management system 100 remotely controls the common camera 10-C to zoom in on the target T. This enables the management system 100 to perform face recognition processing on the target T with high accuracy based on the image IMG-C.

[0045] As a modified example, the target T may perform a predetermined operation on its own initiative. For example, the target T may recognize that the monitoring service has been interrupted and perform a predetermined operation on its own initiative.

[0046] The management system 100 may estimate the posture of the target T from the whole-body image of the target T by a well-known posture estimation process. In this case, since the management system 100 can acquire the position of the face in the whole-body image of the target T, it is also possible to zoom in on the face of the target T. This enables the management system 100 to perform face recognition processing with even greater accuracy.

[0047] After the face authentication process is completed, the management system 100 cancels the zoom-in and returns the common camera 10-C to the default state.

[0048] Effects As described above, according to this embodiment, the common camera 10-C is used for both the face authentication process and the ReID process. Therefore, compared with the case where the camera 10-A dedicated to the face authentication process and the camera 10-B dedicated to the ReID process are separately provided, it is possible to realize the face authentication process and the ReID process more efficiently. In addition, the installation cost of the camera 10 is also reduced.

[0049] Also, as described above, there is a possibility that the association between the person ID and the track ID will disappear or will change to an incorrect one. In order to restore the tracking of the target T to normal, it is necessary to perform face recognition processing on the target T again to restore the association between the person ID and the track ID to normal. In the comparative example shown in FIG. 3, it was necessary to have the target T return to the entrance of the specified area to perform face recognition processing again. On the other hand, according to this embodiment, the target T does not need to return to the entrance of the specified area. By using the common camera 10-C in the vicinity of the current position of the target T, it is possible to quickly perform face recognition processing on the target T again. That is, it is possible to quickly restore the association between the person ID and the track ID to normal. This also contributes to efficiency. [Explanation of symbols]

[0050] 10. Camera 10-C Common Camera 100 Management Systems IMG Image T Target

Claims

1. A management system that performs a person re-identification process based on an image captured by a camera, One or more processors; When a predetermined condition is satisfied, the one or more processors permit a face recognition process to be performed to recognize a target based on the image captured by the camera; The predetermined condition is: the distance between the camera and the target within the camera's angle of view is less than a predetermined threshold; being able to zoom in on the target shown in the image; Includes at least one of Management system.

2. The management system according to claim 1, the predetermined condition includes the distance between the camera and the target being less than the predetermined threshold; The one or more processors provide the position information of the camera to the terminal of the target and request the target to move closer to the camera. Management system.

3. The management system according to claim 1, the predetermined condition includes being able to zoom in on the target shown in the image; The one or more processors: acquiring position information of the target, and calculating an in-image position that is a position of the target in the image based on the position information; Based on the position in the image, the face recognition process is performed by zooming in on the target shown in the image. Management system.

4. The management system according to claim 1, the predetermined condition includes being able to zoom in on the target shown in the image; The one or more processors: Identifying a person performing a predetermined action in the image; determining that the person performing the predetermined action is the target; Zooming in on the target shown in the image and performing the face recognition process. Management system.

5. The management system according to claim 4, The one or more processors communicate with the target terminal and request the target to perform the predetermined action. Management system.

Citation Information

Patent Citations

  • Individual authentication system

    JP2005275869A

  • Face authentication database construction method, face authentication device, and face authentication program

    JP2013069155A

  • Positional detection system and method for detecting position

    JP2019152490A

  • Authentication device, computer program, and authentication system

    JP2021147954A

  • Seating position management system and seating position management method

    JP2021149621A

Cited By

  • Game machine

    JP2024166283A

  • Game machine

    JP2024166284A

  • Game machine

    JP2024166285A

  • Game machine

    JP2024169538A

  • Game machine

    JP2024169539A