Facial recognition device and facial recognition method

The face authentication device and method address security and feedback gaps in conventional systems by displaying live images and progress notifications, enhancing user experience and data security.

JP7595268B2Active Publication Date: 2024-12-06PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023092439
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06
Estimated Expiration
2039-03-04

AI Technical Summary

Technical Problem

Conventional face recognition systems require users to carry IC cards for authentication, posing security risks for personal information management and lack real-time feedback on recognition progress.

Method used

A face authentication device and method that includes a facial feature matching unit, displaying live images and progress status on a screen, providing notifications for standby, in-progress, and completed authentication states, and securely managing facial data.

Benefits of technology

Enables users to easily track face recognition progress while ensuring safe management of personal information by minimizing data storage and providing real-time authentication feedback.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595268000001
    Figure 0007595268000001
  • Figure 0007595268000002
    Figure 0007595268000002
  • Figure 0007595268000003
    Figure 0007595268000003
Patent Text Reader

Abstract

To provide a face authentication machine to simply make a user recognize a progress status of a face authentication based on an initial screen and a notification screen as an authentication screen by collation processing of face feature value.SOLUTION: A face authentication machine 1 comprises: a face image acquisition part (a camera 11) for acquiring a live image of a prescribed photographic area in a facility including a user face image; a display part (a display 12) for displaying an authentication screen including the live image acquired by the face image acquisition part; and a processor (a control part 15) for controlling the face image acquisition part and the display part. When performing face authentication by a face feature value collation part 86, the processor displays an initial screen indicating that it is in a standby state of the face authentication when detecting the approach of the user, as the authentication screen, and further causes the display part to display a notification screen indicating a progress status of the face authentication when detecting the user's face from the live image, as the authentication screen.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a face recognition device and a face recognition method that perform face recognition processing based on image data obtained by photographing a subject. [Background technology]

[0002] Conventionally, a system has been developed in which face recognition of a subject is applied to the opening and closing control of a gate through which the subject passes. For example, a face recognition system is known that includes a reading unit that receives an identifier transmitted from a wireless tag (remote IC card) carried by a user, an acquisition unit that acquires a facial feature amount registered corresponding to the identifier received from the wireless tag, an imaging unit that images the user, an extraction unit that extracts the facial feature amount from the image data captured by the imaging unit, a face matching unit that checks whether the facial feature amount extracted by the extraction unit matches the facial feature amount acquired by the acquisition unit, and an opening and closing control unit that opens or closes the exit of the gate according to the result of the matching by the face matching unit (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. WO2018 / 181968 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in the above conventional technology, it is possible to narrow down the subjects of face recognition (i.e., registered facial features) by the identifier transmitted from the user's wireless tag, but since users who pass through the gate must always carry their IC cards, it becomes difficult to safely manage personal information (i.e., information that can identify a specific individual) contained in the IC cards. Moreover, the above conventional technology does not take into consideration a mechanism for safely managing both the facial image data and the feature data by storing them on a cloud server or the like.

[0005] Furthermore, in a facial recognition system, it may take a certain amount of time from photographing the user's face to obtaining the final recognition result, but the above-mentioned conventional technology does not take into consideration any mechanism for informing the user of the progress of facial recognition in such a case.

[0006] Therefore, the main objective of the present disclosure is to provide a facial recognition device and a facial recognition method that can safely manage personal information of users who use a facility while allowing the users to easily recognize the progress of facial recognition. [Means for solving the problem]

[0007] The face recognition device disclosed herein requests a face feature amount matching unit, which performs a matching process of face features for face recognition of a user of a facility, when an event occurs based on an action of the user in the facility, and A screen showing the progress of face recognition is presented to the user. A face authentication device, comprising: a face image acquisition unit for acquiring a live image of a predetermined photographing area in the facility, the live image including a face image of the user; a screen display area in which the live image is displayed and an information display area in which information indicating the progress is displayed; A display unit that displays an authentication screen, and a processor that controls the face image acquisition unit and the display unit, the processor instructing the user who approaches the face authentication device to: a display of the information display area on the authentication screen is controlled to present an initial screen indicating that the face authentication is in a standby state; and, when the face of the user is detected from the live image and a face image is acquired, a display of the information display area on the authentication screen is controlled to present a first notification screen indicating that the face authentication of the user is in progress; and, when the face feature matching process by the face feature matching unit is completed, a display of the information display area on the authentication screen is controlled to present a second notification screen indicating that the face authentication of the user has been completed. The composition is as follows.

[0008] In addition, the face recognition method disclosed herein is a method of recognizing a face of a facility user by requesting a face feature amount matching unit that performs a matching process of a face feature amount for face recognition of the user of the facility when an event occurs based on an action of the user in the facility. Present the user with a screen showing the progress of face recognition. A face authentication method using a face authentication device, comprising: acquiring a live image of a predetermined photographing area in the facility, the live image including a face image of the user; an image display area in which the live image is displayed and an information display area in which information indicating the progress is displayed; When the authentication screen is displayed on the display unit, the user who approaches the face authentication device is a display of the information display area on the authentication screen is controlled to present an initial screen indicating that the face authentication is in a standby state; and, when the face of the user is detected from the live image and a face image is acquired, a display of the information display area on the authentication screen is controlled to present a first notification screen indicating that the face authentication of the user is in progress; and, when the face feature matching process by the face feature matching unit is completed, a display of the information display area on the authentication screen is controlled to present a second notification screen indicating that the face authentication of the user has been completed. The composition is as follows. Effect of the Invention

[0009] According to the present disclosure, The authentication screen for face recognition has an image display area where a live image is displayed and an information display area where progress information is displayed. By displaying each area, the relationship between the state of the live image and the progress of face recognition can be understood. This allows the user to easily recognize it. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an overall configuration diagram of a face authentication system according to an embodiment of the present invention. [Diagram 2] An explanatory diagram showing examples of events that occur at a facility [Diagram 3] An overview of match groups [Figure 4] An explanatory diagram showing an overview of the face matching inquiry performed by the face recognition device 1. [Diagram 5] An explanatory diagram showing an overview of a face matching unit [Figure 6] Block diagram showing the schematic configuration of face recognition device 1 [Figure 7] A block diagram showing the schematic configuration of the management terminal 2. [Figure 8] A block diagram showing a schematic configuration of a registration device 4. [Figure 9] A block diagram showing the schematic configuration of the face management server 5. [Figure 10] A block diagram showing a schematic configuration of the face matching server 6. [Figure 11] A block diagram showing a schematic configuration of an external device. [Figure 12] FIG. 1 is an explanatory diagram showing an overview of a face image extraction process; [Figure 13] An explanatory diagram showing an overview of face recognition inquiry based on matching group information [Figure 14] FIG. 13 is an explanatory diagram showing an example of a registration screen displayed on the registration device 4. [Figure 15] A sequence diagram showing the steps of the user registration process [Figure 16] A sequence diagram showing a first example of a process for face authentication of a user. [Figure 17] A sequence diagram showing a second example of a process for face authentication of a user. [Figure 18] FIG. 1 is an explanatory diagram showing an example of an authentication screen displayed on the face authentication device 1. [Figure 19] A sequence diagram showing a third example of a process for face authentication of a user. [Figure 20] An explanatory diagram showing the login screen displayed on the management terminal 2. [Figure 21] A screen displayed on the management terminal 2 when registering, referencing, updating, or deleting the association between the face recognition device 1 and the matching group. [Figure 22] A screen displayed on the management terminal 2 when registering, referencing, updating, or deleting the association between the face recognition device 1 and the matching group. [Diagram 23] FIG. 13 is an explanatory diagram showing an example of a user management screen displayed on the management terminal 2. [Figure 24] FIG. 13 is an explanatory diagram showing an example of a conference room search screen displayed on the management terminal 2. [Diagram 25] FIG. 13 is an explanatory diagram showing an example of an attendee display screen displayed on the management terminal 2. [Figure 26] FIG. 13 is an explanatory diagram showing an example of a personal connection search screen displayed on the management terminal 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The first invention made to solve the above problems is a method for identifying a face of a user of a facility by issuing a request for the matching process to a face feature amount matching unit that performs a matching process for face authentication of the user of the facility when an event based on an action of the user occurs in the facility, Present the user with a screen showing the progress of face recognition.A face authentication method using a face authentication device, comprising: acquiring a live image of a predetermined photographing area in the facility, the live image including a face image of the user; an image display area in which the live image is displayed and an information display area in which information indicating the progress is displayed; When the authentication screen is displayed on the display unit, the user who approaches the face authentication device is a display of the information display area on the authentication screen is controlled to present an initial screen indicating that the face authentication is in a standby state; and, when the face of the user is detected from the live image and a face image is acquired, a display of the information display area on the authentication screen is controlled to present a first notification screen indicating that the face authentication of the user is in progress; and, when the face feature matching process by the face feature matching unit is completed, a display of the information display area on the authentication screen is controlled to present a second notification screen indicating that the face authentication of the user has been completed. The composition is as follows.

[0012] According to this, The authentication screen for face recognition has an image display area where a live image is displayed and an information display area where progress information is displayed. By displaying each area, the relationship between the state of the live image and the progress of face recognition can be understood. This allows the user to easily recognize the information.

[0013] In addition, the second invention is a method for processing the above-mentioned As the first notification screen , the user The face recognition The display device is configured to display characters or graphics indicating that a job is in progress.

[0014] This makes it possible for the user to more reliably recognize the progress of face authentication by the face authentication server.

[0015] In addition, the third invention is a method for detecting a plurality of kinds of data by using the above-mentioned processor. As a second notification screen , the user The face recognition The display device is configured to display characters or graphics indicating that the process has been completed.

[0016] This makes it possible for the user to more reliably recognize the progress of face authentication by the face authentication server.

[0017] In addition, a fourth invention is configured such that the processor deletes the face image for authentication after the face authentication process is completed.

[0018] This will prevent facial images from being stored for long periods of time in a facial recognition device used by unspecified persons, making it possible to manage users' facial images more safely.

[0019] In addition, the fifth invention is configured to further include a history information generation unit that requests the face authentication processing when the user enters a conference room in the facility as the event, and generates entry history information that associates multiple users who have entered the conference room.

[0020] This makes it possible to obtain information indicating the relationships between multiple users from information related to face authentication.

[0021] In a sixth aspect of the present invention, a facial feature amount matching unit that performs a matching process of facial features for face authentication of a user of a facility is requested to perform the matching process when an event occurs based on an action of the user in the facility, Present the user with a screen showing the progress of face recognition. A face authentication method using a face authentication device, comprising: acquiring a live image of a predetermined photographing area in the facility, the live image including a face image of the user; an image display area in which the live image is displayed and an information display area in which information indicating the progress is displayed; When the authentication screen is displayed on the display unit, the user who approaches the face authentication device is a display of the information display area on the authentication screen is controlled to present an initial screen indicating that the face authentication is in a standby state; and, when the face of the user is detected from the live image and a face image is acquired, a display of the information display area on the authentication screen is controlled to present a first notification screen indicating that the face authentication of the user is in progress; and, when the face feature matching process by the face feature matching unit is completed, a display of the information display area on the authentication screen is controlled to present a second notification screen indicating that the face authentication of the user has been completed. The composition is as follows.

[0022] According to this, The authentication screen for face recognition has an image display area where a live image is displayed and an information display area where progress information is displayed. By displaying each area, the relationship between the state of the live image and the progress of face recognition can be understood. This allows the user to easily recognize the information.

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0025] FIG. 1 is a diagram showing the overall configuration of a face authentication system according to this embodiment.

[0026] This face authentication system includes a face authentication device 1, a management terminal 2, a face authentication server 3 (cloud server), and a registration device 4 (face authentication registration device). The face authentication server 3 includes a face management server 5 (face authentication management server) and a face matching server 6.

[0027] The face authentication device 1, management terminal 2, registration device 4, face management server 5, and face matching server 6 are connected via a network such as the Internet. A face authentication system is constructed for each face authentication service provider (business operator). Multiple face authentication devices 1 are installed at each location where face authentication is required, such as building entrances and room entrances. The registration device 4 is placed at a reception counter that receives visitors to the facility, etc. A required number of face matching servers 6 are provided according to the number of face authentication devices 1, etc.

[0028] The face authentication device 1 is equipped with a camera 11, which captures images of users (visitors to the facility, employees of companies or stores within the facility, etc.) using the camera 11. The face authentication device 1 also has a display 12, which displays the face authentication results acquired from the face matching server 6 and notifies the user of the face authentication results.

[0029] The management terminal 2 is operated by an administrator of the face authentication system, and is configured as a PC, on which a management application is installed that manages the operations of the face authentication device 1, the registration device 4, the face management server 5, and the face matching server 6. This management application allows the administrator to perform various management tasks. The management application is implemented as a Web application. For the administrator, multiple administrator groups are set up, each of which has different access rights to each piece of information in the face authentication system.

[0030] The registration device 4 acquires information used for processing the face authentication of users (hereinafter, referred to as authentication information). The authentication information may include not only the face image for registration of each user, but also identification information other than the face image that is used to identify the user.

[0031] Examples of the specific information include information displayed on the user's business card, information displayed on a payment card (e.g., a credit card or a cash card) held by the user, and information indicating the user's location within the facility. The specific information may include information that can permanently identify the user, such as the user's name. However, the specific information may also be information that can temporarily identify the user, such as a user number assigned to a user of the facility, the number of a conference room (including any space available for interviews) used by the user within the facility, or the number of the room in which the user stays at a lodging facility, etc.

[0032] The face management server 5 centrally manages user information (including personal information). Specifically, the face management server 5 stores and manages data of face images for registration and identification information for each user in association with each other. The face management server 5 acquires the face images for registration and the identification information from the registration device 4 when registering a user. The face management server 5 may also acquire at least a portion of the information included in the face images for registration and the identification information from the management terminal 2 (i.e., the administrator).

[0033] During face authentication, the face matching server 6 acquires face image data for authentication of the person to be subjected to face authentication from the face authentication device 1, generates face feature data of the person from the face image data, and performs face matching by comparing the face feature data of the person to the face feature data of a registered person (registered user) stored in the device, thereby performing face authentication to determine whether the person is a registered person or not.

[0034] Furthermore, prior to face authentication, when a user is registered, face matching server 6 acquires face image data for registration of the user from face management server 5 (or registration device 4), generates face feature amount data of the user from the face image data, and stores the data in its own device. In some cases, face matching server 6 can also acquire a photographed image of the user from management terminal 2 (i.e., the administrator), and acquire face image data from the photographed image.

[0035] In this embodiment, the face management server 5 (face image management unit) and the face matching server 6 (face image matching unit) are located in physically different information processing devices, but they may also be located in a single information processing device.

[0036] In addition, in this embodiment, the management terminal 2 and the face management server 5 are provided, but the management terminal 2 and the face management server 5 can be configured as a single information processing device. For example, by installing a management application in the face management server 5, the face management server 5 can also serve as the management terminal 2. In addition, an administrator (e.g., an employee of a company or store in the facility) with limited access rights to the information of the face authentication system can access the face management server 5 from his / her own information processing terminal (e.g., a PC or a tablet) and refer to information that needs to be kept less secret than the user's personal information (e.g., information about a conference attended by the user, which will be described later, and information about the user's personal connections).

[0037] In addition, in the present embodiment, facial feature matching is performed, but the facial matching is not limited to facial feature matching, and a matching method to which machine learning etc. is applied may be adopted. Furthermore, the present embodiment can be applied to biometric authentication other than facial authentication.

[0038] Next, an example of an event that occurs due to an action of a user of a facility to which a face authentication system is applied will be described. Fig. 2 is an explanatory diagram showing an example of an event that occurs in the facility.

[0039] As shown in Fig. 2, a user 10 of the facility can register as a user of the facility (i.e., register information to be used in face matching processing) using a registration device 4 arranged at the reception desk. For registered users who revisit the facility within a specified period of time, registration using the registration device 4 can be omitted.

[0040] In the facility, facial recognition is performed for each of the various events (see the area within the two-dot chain line in Figure 2) that occur due to the actions of people (including registered users) within the facility.

[0041] For example, a registered user 10 can pass through a security gate that manages the passage of the user 10 within a facility by undergoing facial authentication in front of the security gate. A facial authentication device 1 that cooperates with a gate device 16 (i.e., controls the open / closed state of the gate) can be installed at the security gate.

[0042] Also, for example, a registered user 10 can enter a conference room in a facility by undergoing facial authentication in front of the conference room. In front of the conference room, a facial authentication device 1 can be placed that works with a locking device that locks the door of the conference room (i.e., controls the locking and unlocking of the door). Note that, for example, in a lodging facility such as a hotel, a facial authentication device 1 that works with a locking device of the door of the room where the user is staying can be installed instead of a conference room.

[0043] For example, a registered user 10 can undergo face authentication in a restaurant in the facility, allowing automatic payment of the restaurant's bill. A camera 11 capable of photographing the face of the user 10 is installed in the restaurant, and the face authentication machine 1 works in conjunction with a payment terminal device (or payment system) used to pay the bill at the restaurant.

[0044] Also, for example, a registered user 10 can undergo face authentication at a gym in the facility, allowing automatic payment of the gym's usage fee. The gym is equipped with a camera 11 capable of photographing the user's face, and the face authentication machine 1 works in conjunction with a payment terminal device (or payment system) used to pay the gym usage fee.

[0045] Events occurring within the facility are not limited to the examples mentioned above, but may include various other occurrences such as payment of fees for the facility's parking lot, payment of fees for the laundry service within the facility, and payment of fees for the bathhouse within the facility.

[0046] Furthermore, for example, if employees of a company that provides services within a facility are registered as users, security in the back yard can be ensured by placing a facial recognition device 1 at the entrance / exit of the facility's back yard (e.g., a warehouse, workshop, etc.).

[0047] Next, the comparison group will be described with reference to Fig. 3, which is an explanatory diagram showing an overview of the comparison group.

[0048] In this embodiment, users are grouped according to the installation location of the face authentication device 1 where the users undergo face authentication, and a matching group is set for each user. Similarly to the users, the face authentication devices 1 are grouped according to the installation location of the face authentication device 1, and a matching group is set for each face authentication device 1. Furthermore, the face matching servers 6 corresponding to the face authentication devices 1, i.e., the face matching servers 6 that accept inquiries (requests) for face matching from the face authentication devices 1, are grouped in the same way as the face authentication devices 1, and a matching group is set for each face matching server 6.

[0049] For example, in a face authentication device 1 installed at the entrance of a multi-tenant building in which multiple companies are located, users belonging to multiple companies undergo face authentication. On the other hand, information on users belonging to different organizations cannot be managed in the same database. In such a case, therefore, a matching group is formed for each of the multiple companies, and the registered information of users is stored in matching group units, and the face authentication device 1 makes a face matching inquiry to the face matching server 6 of the matching group for each company.

[0050] A user can only be authenticated by a face authentication device 1 that corresponds to the matching group to which the user belongs. Therefore, even a registered user cannot be authenticated by a face authentication device 1 that does not belong to the user's matching group, for example, a face authentication device 1 in a building that the user is not allowed to enter. For example, if a user who works at one business office visits another business office, the user cannot be authenticated by the face authentication device 1 at that business office.

[0051] Next, a description will be given of a face matching inquiry from the face authentication device 1 to the face matching server 6. FIG.

[0052] The face authentication device 1 makes an inquiry about face matching to a face matching server 6 that belongs to the same matching group as the device itself, and sends a face matching request (face authentication processing request) to the face matching server 6 that is the inquiry destination (request destination). The face authentication device 1 also holds the network address (IP address) of the face matching server 6 as matching group information, and makes an inquiry about face matching to the face matching server 6 that corresponds to the device itself based on this network address.

[0053] The face authentication inquiry method includes a simultaneous inquiry shown in FIG. 4(A), a random inquiry shown in FIG. 4(B), and a sequential inquiry (switching inquiry destination) shown in FIG. 4(C).

[0054] As shown in Fig. 4(A), in the case of simultaneous inquiry, the face authentication device 1 makes a face matching inquiry to the face matching servers 6 corresponding to each of the multiple matching groups at the same time. For example, when the face authentication device 1 is installed at the entrance of a multi-tenant building and users of multiple matching groups undergo face authentication, the face authentication device 1 sends a face matching request to the face matching servers 6 of each group at the same time.

[0055] In order to reduce the load on one face matching server 6, it is also possible to split and store the facial feature amount data of users belonging to one matching group among multiple face matching servers 6. In this case, the face authentication device 1 simultaneously queries all face matching servers 6 in the same matching group as itself and that have different registered contents.

[0056] As shown in FIG. 4(B), in the case of random inquiry, the face authentication device 1 randomly selects a face matching server 6 to be queried for face matching from among multiple face matching servers 6 in the same matching group as the face authentication device 1. The multiple face matching servers 6 have the same registered contents. In other words, the same user is the target of matching, and the facial feature amount data of the same user is stored. In this inquiry method, inquiries from the face authentication device 1 are distributed to each face matching server 6, so the load on each face matching server 6 can be reduced and the load can be distributed.

[0057] As shown in FIG. 4(C), in the case of sequential inquiry, the face authentication device 1 sequentially selects the inquiry destination for face matching from among the multiple face matching servers 6 in the same matching group as itself. Specifically, an order (priority) is assigned to the face matching servers 6, and the face authentication device 1 selects the face matching server 6 according to the order, and if there is no response from the face matching server 6 with a higher order, the transmission destination is switched to the face matching server 6. That is, the face authentication device 1 selects the face matching server 6 with the first order in the matching group and requests face authentication from that face matching server 6. Then, if there is no response from the face matching server 6, the face authentication device 1 selects another face matching server 6 with the next order in the same matching group and requests face authentication from that face matching server 6. This type of control can achieve redundancy (backup).

[0058] Next, the face matching unit will be described with reference to Fig. 5, which is an explanatory diagram showing an overview of the face matching unit.

[0059] The face matching server 6 can be provided with a plurality of face matching units for performing face matching. The face matching units have a face matching process for executing a face matching program, and a face matching database in which information (facial feature amount data) of users to be compared for face matching is registered.

[0060] This face matching unit is provided for each matching group, and performs face matching in response to a request for face matching from a face authentication device 1 that belongs to the matching group. Therefore, one face matching server 6 can handle a plurality of matching groups.

[0061] Here, the face authentication device 1 holds the network address (IP address) of the face matching server 6 as matching group information, and can make a face matching inquiry to the face matching server 6 corresponding to the device itself based on this network address. Furthermore, in this embodiment, the face authentication device 1 holds the network address (IP address) of each face matching unit, and can make a face matching inquiry to the face matching unit corresponding to the device itself based on this network address.

[0062] In addition, since a face matching database is provided for each matching group, the face feature amount data of the user can be stored for each matching group. The face matching process executes a matching process between the face feature amount data stored for each group and the face feature amount data generated from the face image data acquired from the face authentication device 1 of the same group.

[0063] Here, there are various modes of the correspondence relationship between the face authentication device 1 and the face matching unit.

[0064] In the example shown in FIG. 5(A), the face matching server 6 is provided with one face matching unit of the matching group to which the face authentication device 1 belongs, and the face authentication device 1 and the face matching server 6 have a one-to-one correspondence.

[0065] In the example shown in FIG. 5(B), the face authentication device 1 and the face matching server 6 correspond one-to-one, but the face matching server 6 is provided with two face matching units for the matching group to which the face authentication device 1 belongs. In this case, in the case of a face authentication device 1 installed at the entrance of a multi-tenant building, information of users who belong to different organizations (e.g., user A, user B) can be managed in different matching databases.

[0066] 5(C), the face authentication device 1 and the face matching server 6 correspond to each other in a one-to-two relationship, and the face authentication device 1 makes face matching inquiries (user matching requests) to the two face matching servers 6. In this case, by providing two or more identical face matching units, that is, face matching units that perform face matching using the facial feature amount data of the same user, in different face matching servers 6, it is possible to distribute the load of the face matching server 6 and handle vital matters.

[0067] Next, a description will be given of a schematic configuration of the face authentication device 1. Fig. 6 is a block diagram showing a schematic configuration of the face authentication device 1.

[0068] The face authentication device 1 includes a camera 11 (face image acquisition unit), a display 12 (display unit), a communication unit 13 (transmission unit, reception unit), a memory unit 14, a control unit 15, and a control signal transmission unit 17.

[0069] The camera 11 constantly captures a predetermined shooting area, and when a person enters the shooting area, the person is photographed, and a photographed image of the subject of face authentication (including registered users) can be obtained. Note that, to save power, a human presence sensor (not shown) may be provided to detect the approach of a person and activate the camera 11.

[0070] The display 12 displays the progress of face authentication and the face authentication results, so that the person undergoing face authentication can check them. Note that a speaker may be provided as an output unit that outputs the authentication results of the subject, so that the face authentication results are notified by voice. Also, the display 12 may be omitted depending on the installation location of the face authentication device 1 (for example, when the face authentication device 1 is installed at a security gate).

[0071] The communication unit 13 communicates with the face matching server 6 via the network. In this embodiment, the communication unit 13 transmits face image data to the face matching server 6. The communication unit 13 also receives a user authentication result from the face matching server 6. The communication unit 13 also communicates with the face management server 5 via the network. In this embodiment, the communication unit 13 receives matching group information and the like from the face management server 5.

[0072] The storage unit 14 stores data of a captured image (face image) of a user, collation group information, a control program executed by a processor constituting the control unit 15, and the like.

[0073] The control unit 15 includes a startup processing unit 21, a face image extraction unit 22, a face matching request unit 23, an authentication result notification unit 24, an authentication result adjustment unit 25 (history information generation unit), and an operation status monitoring unit 26. The control unit 15 is configured with a processor, and each unit of the control unit 15 is realized by causing the processor to execute a program stored in the storage unit 14.

[0074] When the face authentication device 1 is started up, the startup processing unit 21 acquires the matching group information from the face management server 5 and stores it in the storage unit 14. Note that the acquisition of the matching group information is not limited to this, and the matching group information may be distributed from the face management server 5 to the target face authentication device 1 along with an update of the matching group information.

[0075] The face image extraction unit 22 can acquire a photographed image of a subject of face authentication from the camera 11 (camera image capture), detect the face of the person from the photographed image (face detection), determine whether the size of the detected face is appropriate (face size check), cut out the face area from the photographed image (face cut-out), and acquire data of the subject's face image (face image for authentication). Note that the face image data may be data of only the image of the face area, but may also be a combination of data of the photographed image (image of a specified photographed area) and position information (face frame information) of the face area on the photographed image data.

[0076] The face matching request unit 23 transmits a face matching request via the communication unit 13 to the face matching server 6 that is in the matching group matching the own device.

[0077] The authentication result notification unit 24 displays the face authentication result acquired from the face matching server 6 by the communication unit 13 on the display 12, thereby notifying the user of the face authentication result.

[0078] The authentication result adjustment unit 25 performs control for linking with the external device 16 based on the face matching result of the user acquired from the face matching server 6. Examples of the external device 16 that can be used include a gate device (see FIG. 2) that manages the passage of users in the facility, a door lock device that manages entry and exit to a conference room or a specified area in the facility, and a payment terminal device for making payments for services received or purchased items by users in the facility. When the authentication result adjustment unit 25 links with the door lock device of the conference room, it can generate information on an entry history that associates multiple users who have entered the conference room.

[0079] The face authentication device 1 and the external device 16 are directly connected by a communication cable or the like, or are communicatively connected via a known network. The authentication result adjustment unit 25 is configured as a connection application, and can notify the user's face matching result to a server of an external system including the external device 16.

[0080] The operation status monitoring unit 26 monitors the operation status of its own device, and notifies the face management server 5 of the operation status of its own device.

[0081] The control signal transmitting unit 17 transmits a control signal to the external device 16 for controlling the operation thereof.

[0082] Next, a description will be given of the management terminal 2. FIG.

[0083] The management terminal 2 includes a communication unit 31, a display 32 (display unit), an input device 33 (operation unit), a storage unit 34, and a control unit 35.

[0084] The communication unit 31 communicates with the face management server 5 via the network. In this embodiment, the communication unit 31 receives screen information and the like from the face management server 5, and transmits to the face management server 5 operation information and the like of the administrator in response to the screen information and the like.

[0085] The display 32 displays various screens. The input device 33 is a mouse, a keyboard, etc., and is used to operate the screen displayed on the display 32.

[0086] The storage unit 34 stores a program (management application) executed by a processor constituting the control unit 35 and the like.

[0087] The control unit 35 includes a GUI control unit 38. The control unit 35 is configured with a processor, and each unit of the control unit 35 is realized by executing a program (management application) stored in the storage unit 34 by the processor.

[0088] The GUI control unit 38 displays various operation screens delivered from the face management server 5 on the display 32. In addition, in response to an input operation of an administrator using the input device 33, the GUI control unit 38 acquires input information and controls the screen. In this embodiment, the GUI control unit 38 performs display input control on a screen related to login, specifically, a login screen. In addition, the GUI control unit 38 performs display input control on a screen related to user management, specifically, a screen related to registration (individual registration, batch registration), reference, update, and deletion of user information. In addition, the GUI control unit 38 performs display input control on a screen related to matching group management, specifically, a screen related to registration (individual registration, batch registration), reference, update, and deletion of matching groups. In addition, the GUI control unit 38 performs display input control on a screen related to authentication device management, specifically, a screen related to registration, reference, update, and deletion of association between authentication devices and matching groups. In addition, the GUI control unit 38 performs display input control on a display input screen for reference to an authentication log (history information of face authentication).

[0089] Next, a description will be given of a schematic configuration of the registration device 4. FIG 8 is a block diagram showing a schematic configuration of the registration device 4.

[0090] The registration device 4 includes a camera 41A for photographing a face, a camera 41B for acquiring information (personal information acquiring section), a display 42 (display section), a communication section 43, a storage section 44, and a control section 45.

[0091] The face photographing camera 41A can obtain a photographed image of a user by photographing the face of the user who stops by the reception desk of a facility, for example.

[0092] The information acquisition camera 41B can capture an image of a business card presented by the user by capturing the business card. However, the information acquisition camera 41B can capture not only business cards but also the user's belongings on which information that can identify the user is displayed (for example, the user's payment card, a card on which the user's place of stay information is recorded, etc.) and capture the captured image. In order to ensure that the information acquisition camera 41B captures images reliably, the registration device 4 can be provided with a placement stand (placement surface) 116 (see FIG. 14(C)) on which the user can easily place business cards, etc., within the capture area of ​​the camera 41B.

[0093] In addition, in the registration device 4, the cameras 41A and 41B for photographing a face and acquiring information may be configured by one camera. Furthermore, the timing of photographing by the camera 41A for photographing a face and the camera 41B for acquiring information can be determined by the user's operation (for example, pressing a photographing button) on the registration device 4. Alternatively, the timing of photographing may be determined by the receptionist of the facility.

[0094] Furthermore, the registration device 4 may be provided with an information reading device (eg, an RFID reader) capable of reading specific information of a user from an information recording medium (eg, an RFID tag) carried by the user via wireless communication.

[0095] Display 42 can display to the user the registration procedure (see Figs. 14(A)-(F)) for authentication information (including the user's personal information). Also, by providing a speaker to display 42, the registration procedure and the like can be guided to the user by voice. Also, by configuring display 42 as a touch panel, it becomes possible to obtain information input by the user through touch operations.

[0096] The communication unit 43 communicates with the face management server 5 via the network. In this embodiment, the communication unit 43 transmits face image data and identification information to the face management server 5.

[0097] The storage unit 44 stores data of a captured image (face image) of the user, specific information of the user, a control program executed by a processor constituting the control unit 45, and the like.

[0098] The control unit 45 includes a face image extraction unit 46, a specific information acquisition unit 47, a registration request unit 48, a GUI control unit 49, and an operation status monitoring unit 50. The control unit 45 is configured with a processor, and each unit of the control unit 45 is realized by causing the processor to execute a program stored in the storage unit 44.

[0099] The face image extraction unit 46 can acquire a photographed image of a person to be subjected to face authentication from the face photographing camera 41A, detect the person's face from the photographed image, and determine whether the face image of the user has been appropriately acquired. If the acquired face image is not appropriate, the face image extraction unit 46 can display a message on the display 42 prompting the user to retake the image. The face image extraction unit 46 may have the functions of camera image capture, face detection, face size check, and face cutout, similar to the face image extraction unit 22 of the face authentication device 1 described above.

[0100] The specific information acquisition unit 47 has an OCR (optical character recognition) function, and generates specific information (here, information written on a business card) from characters, symbols, and the like extracted from an image captured by the information acquisition camera 41B.

[0101] The registration request unit 48 transmits a request for registration of the user to the face management server 5 via the communication unit 43.

[0102] The GUI control unit 49 displays various guidance screens for the user, which are distributed from the face management server 5, on the display 42. Also, in response to an input operation by the user using the input function of the display 42 (here, a touch panel), the GUI control unit 49 acquires input information and controls the screen.

[0103] The operation status monitoring unit 50 monitors the operation status of its own device, and notifies the face management server 5 of the operation status of its own device.

[0104] Next, we will explain the schematic configuration of the face management server 5. Fig. 9 is a block diagram showing the schematic configuration of the face management server 5.

[0105] The face management server 5 includes a communication unit 51, a storage unit 52, and a control unit 53.

[0106] The communication unit 51 communicates with the management terminal 2 via the network. The communication unit 51 also communicates with the face authentication device 1 via the network. The communication unit 51 also communicates with the registration device 4. The communication unit 51 also communicates with the face matching server 6.

[0107] The memory unit 52 stores a face information database, a database regarding association information between the face authentication device 1 and a matching group, a database regarding association information between the face matching server 6 and a matching group, an administrator access log, a control program executed by the processor constituting the control unit 53, and the like.

[0108] In the face information database, a face image for registration is stored as information about each registered user. In addition, in the face information database, specific information of each user is stored in association with the face image for registration. In addition, a matching group and the like are registered in the face information database. Note that the face image and specific information of the user may be stored in an encrypted state to protect privacy. Furthermore, in the face information database, a part of the face image and specific information of the user may be replaced with anonymized information and stored.

[0109] The control unit 53 includes an administrator access management unit 61, a user management unit 62, a matching group management unit 63, a device management unit 64, a face matching server management unit 65, a face information management unit 66, a database management unit 67, an operation status monitoring unit 68, an authentication log presenting unit 69, an encryption unit 70, an anonymized information generation unit 80, and an image generation unit 90. The control unit 53 is configured with a processor, and each unit of the control unit 53 is realized by the processor executing a program stored in the storage unit 52. Each unit of the control unit 53 is configured as a WebAPI (Web Application Programming Interface).

[0110] The administrator access management unit 61 permits or denies access (login) by an administrator according to the access authority of the administrator who accesses its own device from the management terminal 2. The administrator access management unit 61 monitors the status of access (login) from the management terminal 2 to the face management server 5 and face matching server 6, and upon detecting access from the management terminal 2, records information about the access (such as the administrator who accessed and the date and time) as an administrator access log (history information). As a result, the face management server 5 provides the administrator access log to the management terminal 2 in response to a request from the management terminal 2 to view the administrator access log, allowing the administrator to view the administrator access log.

[0111] Furthermore, the administrator access management unit 61 manages access to its own device from the management terminal 2, and when the administrator operates the management terminal 2 to access its own device, the administrator access management unit 61 accumulates the information at that time as an administrator access log (history information) in the storage unit 52. Furthermore, in response to a reference request from the management terminal 2, the administrator access log is presented to the management terminal 2.

[0112] The user management unit 62 manages information about users, such as face images for registration and specific information, and performs necessary processing related to users in response to requests from the management terminal 2. In this embodiment, requests for registration, reference, update, and deletion related to users are made from the management terminal 2, and the user management unit 62 performs necessary processing in response to the requests. The user management unit 62 also manages information about conferences attended by users and information about the users' personal connections.

[0113] The matching group management unit 63 manages information related to matching groups, and performs necessary processing related to matching groups in response to requests from the management terminal 2. In this embodiment, requests for registration, reference, update, and deletion related to matching groups are made from the management terminal 2, and necessary processing is performed in response to the requests. In addition, matching group information, i.e., information required for the face authentication device 1 to request face authentication from the face matching server 6 that matches its own matching group, is generated for each face authentication device 1, and the matching group information is provided to the face authentication device 1.

[0114] The device management unit 64 manages information relating to the face authentication device 1 and the registration device 4, and performs necessary processing relating to the face authentication device 1 and the registration device 4 in response to a request from the management terminal 2. In this embodiment, a request is made from the management terminal 2 to register, refer to, update, and delete associations between the face authentication device 1 and a matching group, and the device management unit 64 performs necessary processing in response to the request.

[0115] The face matching server management unit 65 manages information relating to the face matching server 6, and performs necessary processing relating to the face matching server 6 in response to a request from the management terminal 2. In this embodiment, a request is made from the management terminal 2 for registration, reference, and deletion of associations between the face matching server 6 and matching groups, and the necessary processing is performed in response to the request.

[0116] The face information management unit 66 synchronizes the face information so that the face information (face image, identification information, etc.) of the user stored in its own device and the face information (user's face feature amount data) of the user stored in the face matching server 6 are maintained in a consistent state. The face information management unit 66 also copies the face information (user's face feature amount data).

[0117] The database management unit 67 manages the database provided in its own device, and backs up and restores the database.

[0118] The operation status monitoring unit 68 monitors the operation status of its own device, and also receives notifications of operation status from the face authentication device 1, the registration device 4, and the face matching server 6, and displays the operation status of the face authentication device 1, the registration device 4, its own device (face management server 5), and the face matching server 6 on the screen of the management terminal 2 in accordance with the administrator's operations on the management terminal 2.

[0119] The authentication log presenting unit 69 acquires the authentication log from the face matching server 6 in response to a reference request from the management terminal 2, and presents the authentication log to the management terminal 2.

[0120] The encryption unit 70 can encrypt at least a part of the user's facial image and specific information. The encrypted information is stored in the storage unit 52. Note that the face management server 5 may delete from the storage unit 52 the original information (information before encryption processing) corresponding to the information encrypted by the encryption unit 70.

[0121] The anonymized information generating unit 80 can perform a process of anonymizing at least a part of the user's face image and specific information. As the anonymization process, the anonymized information generating unit 80 can, for example, replace a part of the user's face image and specific information with dummy information. In addition, in the anonymization process, dummy information may be used for information missing from the user's face image and specific information (for example, information refused to be provided by the user). The anonymized information is stored in the storage unit 52. Note that the encryption unit 70 may encrypt only original information (information before the anonymization process) corresponding to the information anonymized by the anonymized information generating unit 80.

[0122] The image generating unit 90 generates a management screen for an administrator to confirm or change the state of information management by the user management unit 62 and the device management unit 64. The management screen is distributed to the management terminal 2 and the like.

[0123] Next, a description will be given of a schematic configuration of face matching server 6. Fig. 10 is a block diagram showing a schematic configuration of face matching server 6.

[0124] The face matching server 6 includes a communication unit 71, a storage unit 72, and a control unit 73.

[0125] The communication unit 71 communicates with the face authentication device 1 via the network. In this embodiment, it receives face image data and the like from the face authentication device 1. It also transmits user authentication results and the like to the face authentication device 1. The communication unit 71 also communicates with the face management server 5 via the network. In this embodiment, it receives requests for various processes and the like from the face management server 5, and transmits corresponding responses and the like to the face management server 5.

[0126] The storage unit 72 stores a face matching database, association information between users and feature amounts, information about the matching group of the own device, an authentication log, a control program executed by a processor constituting the control unit 73, and the like.

[0127] In the face matching database, facial feature data of each registered user is registered as information about each registered user. A face matching database is provided for each matching group, and the facial feature data of users is stored on a group basis. Here, the facial feature data of registered users may be backed up and stored in a non-volatile memory such as an HDD or SSD provided in face matching server 6 in case of unexpected loss from the face matching database.

[0128] The control unit 73 includes a matching group management unit 81, an image quality check unit 82, a facial image extraction unit 83, a facial feature amount generation unit 84, a facial feature amount management unit 85, a facial feature amount matching unit 86, an authentication log management unit 87, a database management unit 88, and an operation status monitoring unit 89. The control unit 73 is configured with a processor, and each unit of the control unit 73 is realized by the processor executing a program stored in the storage unit 72. Each unit of the control unit 73 is configured as a WebAPI.

[0129] The match group management unit 81 manages the match group to which the device itself belongs, and performs registration and deletion processes related to the match group in response to a request from the face management server 5.

[0130] The image quality check unit 82 judges whether the image of the face area in the captured image satisfies a predetermined quality. Specifically, it detects whether a mask and sunglasses are worn from the target image, and calculates the face recognition suitability (evaluation value based on the face direction and expression).

[0131] The face image extraction unit 83 extracts a face image from a captured image of a user acquired by the registration device 4 during user registration. If necessary, the face image extraction unit 83 detects a person's face from the captured image (face detection), determines whether the size of the detected face is appropriate (face size check), cuts out a face area from the captured image (face cutout), and acquires the person's face image. Note that the face image extraction unit 83 can also extract a face image from a captured image of a subject acquired by the face authentication device 1 during face authentication.

[0132] The facial feature amount generating unit 84 detects facial feature points from the data of the facial image for registration and the facial image for authentication at the time of user registration and face authentication, and generates facial feature amount data, respectively.

[0133] When a user is registered, the facial feature amount management unit 85 registers the facial feature amount data of the user generated by the facial feature amount generation unit 84 in a face matching database corresponding to the user's matching group. When updating or deleting, the facial feature amount management unit 85 deletes the facial feature amount data registered in the face matching database in response to a request from the face management server 5. When a program related to the facial feature amount generation or face matching algorithm is upgraded, the facial feature amount management unit 85 updates the facial feature amount data registered in the face matching database in response to a request from the face management server 5 so as to correspond to the new program.

[0134] During face authentication, the facial feature amount matching unit 86 compares the facial feature amount data of the subject generated from the data of the face image for authentication acquired from the face authentication device 1 with the facial feature amount data of a registered person (registered user) stored in the device itself to determine whether the subject is a registered person or not. This facial feature amount matching unit 86 corresponds to a matching group associated with the device itself, and when multiple matching groups are associated with the device itself, the device has multiple facial feature amount matching units 86 for each matching group. This facial feature amount matching unit 86 (face matching process) constitutes a face matching unit in combination with a face matching database.

[0135] The facial feature amount matching unit 86 calculates the similarity (matching score) between the subject and the registrant. By comparing this similarity with a predetermined threshold, it is possible to determine whether the facial authentication has been successful. In addition to notifying the face authentication device 1 of the result of the success or failure as the matching result, the face authentication device 1 may also be notified of the person ID and similarity of the registrant with a high similarity as the matching result.

[0136] When the face authentication device 1 requests the authentication log management unit 87 to perform face authentication, the authentication log management unit 87 accumulates the information at that time, i.e., the device ID of the face authentication device 1 that requested the face authentication, the face matching result, and other information, as an authentication log, in the storage unit 72. In addition, in response to a request from the face management server 5, the authentication log is provided to the face management server 5.

[0137] The database management unit 88 manages the database provided in its own device, and backs up and restores the database.

[0138] The operation status monitoring unit 89 monitors the operation status of its own device, and notifies the face management server 5 of the operation status of its own device.

[0139] In this embodiment, the face matching server 6 has the functions of face image extraction, face feature amount generation, and face feature amount matching as its main functions, but these functions can be configured in separate information processing devices independent of each other. For example, the face image extraction function can be configured in a separate information processing device independent of other functions such as face feature amount generation and face feature amount matching.

[0140] Next, a description will be given of a schematic configuration of the external device 16. Fig. 11 is a block diagram showing a schematic configuration of the external device 16.

[0141] The external device 16 includes a control signal receiving unit 91 , a driving unit 92 , a power supply unit 93 , and a control unit 94 .

[0142] The control signal receiving unit 91 receives a control signal transmitted from the control signal transmitting unit 17 of the face authentication device 1 (an operation command from the face authentication device 1 that cooperates with the external device 16).

[0143] The drive unit 92 is controlled by the control unit 94 based on a control signal from the face authentication device 1. For example, if the external device 16 is a gate device, the drive unit 92 supplies power to open (or close) a door of the gate device. Also, if the external device 16 is a door lock device, the drive unit 92 supplies power to lock (or unlock) the lock device. Note that if the external device 16 is a device that does not have a driven part (for example, a payment terminal device), the drive unit 92 can be omitted.

[0144] The power supply unit 93 supplies power to each part of the external device 16. The power supply unit 93 is also electrically connected to a power supply unit 97 of the face authentication device 1, and can also supply power to the power supply unit 97. The power supply unit 97 supplies power to each part of the face authentication device 1. By being connected to the power supply unit 97 of the face authentication device 1 via a LAN cable, power can be supplied to the power supply unit 97 by PoE power supply.

[0145] In addition, the power supply unit 93 can be supplied with power from an uninterruptible power supply (or an emergency power supply) that is independent of the power supply line for the entire facility. This allows, for example, a gate device as the external device 16 to perform normal operations even if a power outage occurs within the facility, which has the advantage of maintaining the security of the facility.

[0146] The control unit 94 controls the operation of each unit of the external device 16. The control unit 94 can also execute processes required for the external device 16. The control unit 94 is composed of a processor, and is realized by causing the processor to execute a program stored in a storage unit (memory), not shown.

[0147] Next, the facial image extraction process will be described with reference to Fig. 12, which is an explanatory diagram showing an overview of the facial image extraction process.

[0148] As described above, the facial image extraction unit 22 of the facial authentication device 1 performs a facial image extraction process, i.e., a process of generating facial image data from the captured image data of the user by performing each of the processes of face detection, face size check, and face cut-out, and this facial image extraction process is also performed in the facial image extraction unit 83 of the face matching server 6.

[0149] When a user is registered, photographed image data of the user is sent from the registration device 4 to the face matching server 6 via the face management server 5, and a face image extraction process is performed by the face image extraction unit 83 of the face matching server 6. In some cases, photographed image data of the user may be sent from the management terminal 2 to the face matching server 6 via the face management server 5.

[0150] On the other hand, during face authentication, it is preferable to perform face image extraction processing only in the face authentication device 1 and not in the face matching server 6. In this case, the face authentication device 1 is given a highly accurate face detection function. Also, in the face authentication device 1, the camera 11 constantly captures the image of the shooting area, and sends face image information (captured image data and face frame information) to the face matching server 6 when a face is detected. This distributes the load of the face image extraction processing to multiple face authentication devices 1, and reduces the load on the face matching server 6. In addition, the amount of communication can be reduced, and therefore the load on the network can be reduced. And, since the face authentication response can be performed at high speed, the face authentication of subjects who appear one after another can be performed efficiently.

[0151] That is, by sharing the user's face authentication process between the face authentication device 1 and the face matching server 6, it is no longer necessary to provide a large number of expensive authentication devices that centralize all of the user's face authentication process in the face authentication device 1 as in the past. Furthermore, even if updates to the face feature amount data occur, large-scale maintenance work on the authentication devices is not required, and the work can be done in the face matching server 6. Therefore, according to this embodiment, a face authentication system with an inexpensive configuration and excellent operability can be constructed.

[0152] However, the face authentication device 1 may be configured not to have the function of extracting a face image, that is, the face authentication device 1 may not be provided with the face image extraction unit 22.

[0153] FIG. 13 is an explanatory diagram showing an outline of a face authentication inquiry based on matching group information.

[0154] The face management server 5 holds information about the matching group to which the user belongs, association information between the face authentication device 1 and the matching group, and association information between the face matching server 6 and the matching group. In addition, the matching group management unit 63 of the face management server 5 generates matching group information for each face authentication device 1 based on the association information between the face authentication device 1 and the matching group and the association information between the face matching server 6 and the matching group.

[0155] This matching group information is information required for the face authentication device 1 to request face authentication from the face authentication server 6 that matches its own matching group. This matching group information includes identification information (group number) of the matching group to which the face authentication device 1 belongs, and the face authentication server 6 to which the face authentication device 1 requests face authentication, i.e., destination information of the face authentication server 6 corresponding to the matching group with the face authentication device 1. Here, the destination information is specifically the network address (e.g. IP address) of the face authentication server 6, and this destination information specifies the face authentication server 6 to which the face authentication request is to be made, and the face authentication device 1 and the face authentication server 6 to which the face authentication request is to be made are associated with each other. Note that, if the face authentication device 1 belongs to multiple matching groups, the address (IP address) of each of the corresponding multiple face authentication servers 6 is included in the matching group information.

[0156] In the face authentication device 1, at the time of startup, the startup processing unit 21 acquires matching group information from the face management server 5 as operation setting information and stores it in the device itself. When the face authentication device 1 detects a person's face, it sends a face matching request to the face matching server 6 that matches the matching group to which the device itself belongs. This face matching request includes information on the matching group to which the device itself belongs.

[0157] In addition, in the face authentication device 1, the timing for obtaining setting information regarding matching group information, etc. from the face management server 5 may be at startup, or may be periodically at a specified timing or at specified intervals, or a configuration in which matching group information is distributed from the face management server 5 may be adopted.

[0158] The face matching server 6 holds association information between the face matching process of its own device and the matching group. When the face matching server 6 receives a face recognition request from the face authentication device 1, it identifies the face authentication process corresponding to the face authentication device 1 based on the association information between the matching group and the face authentication process and the matching group acquired from the face authentication device 1, and causes the face authentication process to perform face matching. As a result, the face matching process is performed in the face authentication process corresponding to the specified matching group.

[0159] In the present embodiment, the face matching server 6 provides a face matching database for each matching group, and stores face feature data for each matching group. Therefore, when matching faces, face matching may be performed using face feature data registered in the face matching database corresponding to the matching group of the face authentication device 1. On the other hand, the face matching server 6 may perform matching regardless of the matching group of the face authentication device 1, and then filter the matching results. That is, face matching may be performed using the face feature data of all users stored in the face matching server 6, and only the matching results with users who belong to the matching group of the face authentication device 1 may be extracted. In this case, it is not necessary to provide a separate face matching database for each matching group.

[0160] Next, a description will be given of a registration operation by a user on the registration device 4. Fig. 14 is an explanatory diagram showing an example of a registration screen displayed on the registration device 4.

[0161] When a user starts using a facility (or when the user does not revisit the facility within a specified period after registration), the user can perform a registration operation for authentication using the registration device 4. In this embodiment, the user can perform the registration operation by following a guide screen displayed on the display 42 (here, a touch panel) of the registration device 4. Note that part of the registration operation for authentication described below (such as pressing a button) may be performed by a receptionist with the user's consent.

[0162] An initial screen showing a registration start button 101 is displayed on the display 42 of the registration device 4 (see FIG. 14(A)). When the user presses (touches) the registration start button 101, a screen (first screen) is displayed that explains how personal information is handled in the face authentication system and asks the user for their consent (see FIG. 14(B)).

[0163] The user can consent to the handling of personal information in the face authentication system by pressing the consent button 102 on the screen of Fig. 14(B). This causes the registration device 4 to start acquiring authentication information for the user. Then, a screen (second screen) showing a method for acquiring specific information used to identify the user is displayed on the display 42 (see Fig. 14(C)). The user can cancel the registration operation by pressing the cancel button 103.

[0164] In the example shown in FIG. 14(C), a screen (second screen) is displayed showing a method for photographing the user's business card (here, guidance for placing the user's business card within the photographing area of ​​information acquisition camera 41B). When the user places the business card on the placement stand 116 of registration device 4 and presses photographing button 104, registration device 4 photographs the business card. The user can stop photographing the business card by pressing stop button 105.

[0165] After the business card is photographed, the registration device 4 displays a screen (second screen) showing a method for photographing the user's face image for registration (guidance for positioning the user's face within the photographing area of ​​the face photographing camera 41A) (see FIG. 14(D)). At this time, the registration device 4 can urge the user to align the face by blinking the photographing frame 106 corresponding to the photographing area. A live image (a real-time moving image) photographed by the face photographing camera 41A is displayed within the photographing frame 106. When the user positions his / her face within the photographing frame and presses the photographing button 107, the registration device 4 photographs the user's face. At this time, the registration device 4 can generate a shutter sound when the photographing button 107 is pressed. The user can stop photographing the face by pressing the stop button 108.

[0166] After the face has been photographed, the registration device 4 then displays a screen (second screen) for the user to confirm the photographed face image (see FIG. 14(E)). The photographed face image (still image) is displayed on this screen, and if the user determines that there is no problem with the photographed face image, he or she presses the registration button 111. On the other hand, if the user determines that there is a problem with the photographed face image, he or she can press the re-photograph button 112. This causes the registration device 4 to display the screen shown in FIG. 14(D), making it possible to photograph the face again. The user can cancel photographing the face by pressing the cancel button 113.

[0167] When the photographing of the face is completed, the acquired face image and specific information (here, including the business card image and the business card information extracted from the business card image by OCR) are transmitted to the face management server 5. In this case, for a user who does not wish to provide both a face image and a business card image, at least one of the face image and the business card image may be transmitted to the face management server 5. In this way, the face management server 5 executes a registration process (see FIG. 15) for the user. When the registration process of the face management server 5 is normally completed, the registration device 4 displays a screen (third screen) indicating that the registration is completed (see FIG. 14(F) and “Screen display of processing result” in FIG. 15). Then, when the user presses the end button 115, the registration operation in the registration device 4 is completed. The order of obtaining the face image and the business card image may be reversed.

[0168] After the registration operation is completed, the registration device 4 can delete the face image and the specific information stored in the storage unit 44. This prevents personal information from being stored for a long period of time in the registration device 4 used by unspecified persons, and enables the user's personal information to be managed more safely.

[0169] Note that instead of registering a user with the above-mentioned registration device 4, the administrator can prepare a file of a face image for registration of the user in advance and then register the user using the management terminal 2. Furthermore, in order to complement the registration with the registration device 4, the administrator can also modify the authentication information registered by the registration device 4 or add new information from the management terminal 2.

[0170] Next, a user registration process executed in response to the above-mentioned user registration operation will be described below. Fig. 15 is a sequence diagram showing the procedure of the user registration process.

[0171] When a user's registration operation is performed on the registration device 4 as described above, the face management server 5 starts registration processing for the user as shown in Fig. 15. At this time, the face management server 5 receives the captured face image and identification information along with the user's operation information from the registration device 4. In this registration processing, the face management server 5 first sends a request for face image extraction to a face matching server 6 whose matching group matches the target user. This request includes the captured face image of the user acquired from the registration device 4. At this time, if there are multiple face matching servers 6 whose matching group matches the target user, one face matching server 6 is selected, and a request for face image extraction is sent to that one face matching server 6.

[0172] When the face matching server 6 receives a request for face image extraction from the face management server 5, it performs face image extraction processing. In this face image extraction processing, it performs processing such as face detection and face cut-out on the photographed image of the user's face acquired from the face management server 5 to extract the user's face image. Then, it sends a face image extraction response to the face management server 5. This response includes the user's face image.

[0173] When the face management server 5 receives a response to the facial image extraction from the face matching server 6, it sends a facial registration request to the face matching server 6 whose matching group matches the target user. This request can include the facial image of the user acquired from a specific face matching server 6. In this case, if there are multiple face matching servers 6 whose matching group matches the target user, the face registration request is sent to all (two in FIG. 15 ) face matching servers 6, including the facial image of the user acquired from the specific face matching server 6.

[0174] In addition, the face management server 5 may omit the request for facial image extraction to the face matching server 6 described above, and instead add the photographed image of the user's face obtained from the registration device 4 together with the operation information to the face registration request to the face matching server 6.

[0175] When the face matching server 6 receives a face registration request from the face management server 5, it performs face registration processing. In this face registration processing, face features are generated from the face image of the user and the face features are registered in a database. At this time, a face registration ID is assigned in association with the face features of the user. Then, a face registration response is sent to the face management server 5. This response includes a result indicating whether the face registration has been completed normally or not and the face registration ID. Note that, upon completion of registration of the face features in the database, the face image of the user in the face matching server 6 is deleted. Furthermore, the face feature data may be backed up and stored in a non-volatile memory such as an HDD or SSD managed separately from the face matching DB of the face matching server 6.

[0176] The face management server 5 receives a face registration response from the face matching server 6, and if the process has been completed normally, performs registration processing for the user. In this registration processing, the face image of the user acquired from the registration device 4 is registered in the face information database. The face management server 5 also associates the specific information of the user acquired from the face matching server 6 with the face image and registers it in the face information database. The face management server 5 also registers the face registration ID issued by the face matching server 6 in the face information database as user information.

[0177] Next, a description will be given of the process of face authentication of a user in the face authentication device 1. Fig. 16 shows a first example of the process of face authentication regarding a user. Here, an example is shown in which a gate device (security gate) that manages the passage of users is used as the external device 16 that cooperates with the face authentication device 1.

[0178] When the face authentication device 1 detects a person's face from the image captured by the camera 11 and acquires the face image, it transmits a face matching request to the face matching server 6. This face matching request includes the device ID of the requesting face authentication device 1, the matching group of the face authentication device 1, data of the captured image of the person to be face authenticated (face image for authentication), face frame information, and matching conditions.

[0179] When the face matching server 6 receives the face matching request, first, the face feature generating unit 84 generates face features of the subject from the face image of the subject acquired from the face authentication device 1. Next, the face feature matching unit 86 of the face authentication unit corresponding to the matching group of the face authentication device 1 matches the face features of the subject with the face features of the user registered in the face authentication database. At this time, the face matching process is performed in a face authentication process whose matching group matches the face authentication device 1 that made the request. When the face matching process is completed, the face matching server 6 transmits a face matching response to the face authentication device 1 that made the request. This face matching response includes the matching result (success, failure), the matching score, and the user code.

[0180] The facial feature amount matching unit 86 calculates a matching score that indicates the degree of similarity between the subject of facial authentication and the registered user, and if this matching score is equal to or greater than a predetermined reference value, the subject of facial authentication is deemed to be the registered user, and a matching result is generated indicating that facial matching has been successful.On the other hand, if the matching scores for all users are not equal to or greater than the reference value, it is determined that the subject of facial authentication is not the registered user, and a matching result is generated indicating that facial matching has failed.

[0181] In addition, the processing conditions (parameters for face matching) performed by the face matching server 6 may be added to the matching request. This allows the face authentication device 1 to instruct the processing contents of face matching performed by the face matching server 6. For example, a threshold value for the matching score may be specified as the processing condition for face matching, and matching results with matching scores equal to or greater than the predetermined threshold may be included in the response. Also, the number of matching results may be specified as the processing condition for face matching, and a predetermined number of matching results starting from the highest matching scores may be included in the response.

[0182] In addition, in the face matching server 6, when face matching is completed, the authentication log management unit 87 stores information such as the matching result acquired in the face matching in a database as an authentication log (face recognition history information). At this time, in addition to storing only the face recognition result (success or failure) as the authentication log, the matching score may also be included in the authentication log. Also, the facial image of the subject acquired from the face recognition device 1 may be included in the authentication log. In this case, it is preferable to encrypt and store the facial image of the subject.

[0183] In addition, when many valid matching results whose matching scores exceed the threshold are obtained, a predetermined number of matching results may be narrowed down in order from the highest matching scores and saved as an authentication log. In addition, when face matching does not end normally or when no valid matching results whose matching scores exceed the threshold are obtained, only information included in the request from the face authentication device 1 may be saved as an authentication log.

[0184] Furthermore, when the face authentication device 1 receives a face matching response indicating success of face authentication from the face matching server 6, it transmits an opening / closing control signal for the door section to the associated gate device. In other words, the face authentication device 1 converts the face matching response into an opening / closing control signal for the door section, thereby functioning as a control device (gate opening / closing control device) that controls the operation of the door section of the associated gate device. As a result, the gate device performs an operation to open the door section (i.e., to allow the user to pass) by the drive unit 92. Note that, upon completion of transmission of the control signal (opening / closing control signal), the face authentication device 1 deletes the face image of the user stored in the storage unit 14.

[0185] Fig. 17 shows a second example of the face authentication process for a user. Fig. 18 is an explanatory diagram showing an example of an authentication screen (screen at the time of authentication) displayed on the face authentication device 1. Here, an example is shown in which a lock device provided on the door of a conference room is used as the external device 16 that cooperates with the face authentication device 1. Note that, regarding the second example of face authentication process, detailed explanations of matters similar to those in the above-mentioned first example are omitted.

[0186] The face authentication machine 1 is installed, for example, on a wall of a corridor facing a conference room. When the face authentication machine 1 senses the approach of a person (i.e., detects a user) by a motion sensor or the like, it displays a standby screen (initial screen) of the authentication screen on the display 12 (see FIG. 18(A)). Here, the authentication screen (notification screen) displayed on the display 12 includes an image display area 121 that displays a live image (real-time moving image) captured by the camera 11 of a predetermined shooting area, and an information display area 122 that displays information indicating the progress of the face authentication process.

[0187] Thereafter, when the face authentication device 1 detects a person's face from the image captured by the camera 11 and acquires a face image, it sends a face matching request to the face matching server 6. At this time, characters 123 and a figure 124 indicating that authentication of the user is in progress are displayed in the information display area 122 (see FIG. 18(B)). Note that in the information display area 122, display of either the characters 123 or the figure 124 may be omitted.

[0188] When the face matching server 6 receives the face matching request, it generates face features and executes face matching processing in the same manner as in the above-mentioned first example. When the face matching processing is completed, the face matching server 6 transmits a face matching response to the face authentication device 1 that originated the request.

[0189] Furthermore, when the face authentication device 1 receives a face matching response indicating success of face authentication from the face matching server 6, it displays text 127 and a figure 128 indicating that authentication of the user has been completed in the information display area 122 of the authentication screen (see FIG. 18(C)). At the same time, the face authentication device 1 transmits a control signal (a command to unlock the door) to the lock device of the associated door. In other words, the face authentication device 1 functions as a control device that controls the operation of the associated lock device. As a result, the lock device performs an operation to release the lock (i.e., allow the user to enter). Note that, upon completion of transmission of the control signal, the face authentication device 1 deletes the face image of the user stored in the memory unit 14.

[0190] The face authentication device 1 can also be linked to an external device that only processes information without mechanical operation, such as the gate device or door lock device described above. Figure 19 shows a third example of the face authentication process for a user.

[0191] Here, an example is shown in which a payment terminal device is used as the external device 16 that cooperates with the face authentication device 1. Note that, regarding the third example of the face authentication process, detailed explanations of matters similar to those of the first or second example described above will be omitted.

[0192] In this third example of the face authentication process, when the face authentication device 1 receives a face matching response indicating success of face authentication from the face matching server 6, it transmits a control signal to the associated payment terminal device (or payment system). As a result, the payment terminal device executes a process of linking the authenticated user with the data of the usage fee for the service used by the corresponding user (e.g., the provision of a meal at a restaurant), and as a result, the usage fee is added to the facility expense billing data for the corresponding user.

[0193] Next, the management work by the administrator of the face authentication system will be described. Fig. 20 is an explanatory diagram showing a login screen displayed on the management terminal 2.

[0194] When an administrator launches a management application on the management terminal 2 and accesses the face management server 5, a login screen is displayed. On this login screen, the administrator can enter his or her own user ID and password. When the administrator enters a user ID and password on this login screen and operates the login button, user authentication is performed on the face management server 5. If login is successful, it becomes possible to display various management screens. If login is unsuccessful, an error is displayed.

[0195] Next, registration, reference, update, and deletion of the association between the face authentication device 1 and the matching group in this system will be described. Figures 21 and 22 are explanatory diagrams showing the management screen displayed on the management terminal 2 when registering, referencing, updating, and deleting the association between the face authentication device 1 and the matching group.

[0196] When the management terminal 2 logs in to the face management server 5, the reference screen shown in Figure 21(A), the registration screen shown in Figure 21(B), and the deletion screen shown in Figure 22 can be displayed when registering, referencing, updating, and deleting the association between the face authentication device 1 and the matching group.

[0197] The reference screen shown in Fig. 21(A) displays a list of association settings between the registered face authentication device 1 and the matching group. By viewing this reference screen, the administrator can check the registered contents of the matching group.

[0198] This reference screen is provided with a list display section 211 and a face authentication machine designation section 212. In the list display section 211, the device ID of each authentication machine, the matching group (number) associated with each authentication machine, and an explanation of the matching group are displayed. In the face authentication machine designation section 212, the target face authentication machine 1 (device ID) can be selected using a pull-down menu. This allows the display content of the list display section 211 to be narrowed down to a specific face authentication machine 1, and only the matching group associated with the specific face authentication machine 1 is displayed in the list display section 211. Note that if a face authentication machine 1 is not designated in the face authentication machine designation section 212, information on all face authentication machines 1 is displayed in the list display section 211.

[0199] Also, on this reference screen, sorting can be performed by specifying items (device ID, face authentication device 1, description). Note that it may be possible to specify the number of items to be displayed in the list display section 211 (the number of face authentication devices 1) and the display range of the face matching server 6 and matching group. It may also be possible to specify search conditions related to each item (device ID, face authentication device 1, description) for searching.

[0200] In the example shown in FIG. 21(A), the face recognition devices 1 are grouped according to the locations where the face recognition devices 1 are installed (for example, the locations of business establishments).

[0201] The registration screen (second screen) shown in Fig. 21(B) is used to set up an association between the face authentication device 1 and a matching group. Information that associates the authentication device with a matching group can be registered.

[0202] This registration screen is provided with an authentication device designation section 213, a matching group designation section 214, and a registration button 215. In the authentication device designation section 213, a target face authentication device 1 (device ID) can be selected using a pull-down menu. In the matching group designation section 214, a matching group of the target face authentication device 1 can be selected using a pull-down menu. This pull-down menu displays matching groups that have already been registered. When the face authentication device 1 and matching group are entered on this registration screen and then the registration button 215 is operated, a process of registering the matching group information based on the entered contents is performed on the face management server 5.

[0203] In some cases, multiple matching groups may be associated with one face authentication device 1. In this case, it is sufficient to repeat the registration operation on the registration screen to associate one matching group with one face authentication device 1. Also, multiple matching group designation units 214 may be provided on the registration screen.

[0204] The deletion screen shown in FIG. 22 is for deleting the association setting between the face recognition device 1 and the matching group.

[0205] This deletion screen is provided with a list display section 216, a face authentication device designation section 217, and a delete button 218. In the list display section 216, the device ID of each authentication device, the matching group (number) associated with each face authentication device 1, and an explanation of the matching group are displayed. In addition, the list display section 216 is provided with a check box for each association setting. The association setting can be selected using the check box. When the registration item to be deleted is selected using the check box and the delete button 218 is operated, a process of deleting the selected registration item is performed in the face management server 5. In the face authentication device designation section 217, the face authentication device 1 can be selected using a pull-down menu. As a result, the display contents of the list display section 216 are updated to be narrowed down to a specific face authentication device 1.

[0206] Here, by selecting "reference" on the menu screen (not shown), the screen transitions to a reference screen shown in FIG. 21(A). By selecting "registration" on the menu screen, the screen transitions to a registration screen shown in FIG. 21(B). By selecting "delete" on the menu screen, the screen transitions to a deletion screen shown in FIG. 22. By selecting "update" on the menu screen, the screen transitions to an update screen (not shown). By selecting "association" of the face authentication device 1 with a matching group on the reference screen, the screen transitions to an update screen (not shown).

[0207] In the example shown in Fig. 22, the deleted screen displays a list of registered associations, but the deletion may be performed on an individual edit screen. Also, the update screen (not shown) is similar to the registration screen (Fig. 21(B)).

[0208] FIG. 23 is an explanatory diagram showing an example of a user management screen displayed on the management terminal 2. As shown in FIG.

[0209] When managing information related to a user, the management terminal 2 displays a user management screen shown in FIG.

[0210] 23 is for an administrator to perform operations of registering, referring to, updating, deleting, and searching face images and specific information (business card information in this case) related to the user. Information (including images) input to the user management screen includes information acquired by the registration device 4.

[0211] The user management screen includes an operation selection area 131, a user information area 132, and a business card information area 133.

[0212] In the operation selection area 131, the administrator can select the item to be executed (such as registering, referencing, updating, deleting, and searching information). In Fig. 23, business card registration is selected as the item to be executed. This allows the administrator to newly input information written on the business card displayed on the user's management screen.

[0213] The user information area 132 includes the user's face image and name, and information used for face authentication. Here, the user information area 132 includes a face image display section 141, a name input section 142, a user ID input section 143, an identification ID input section 144, a matching group input section 145, an access permission group input section 146, an additional information input section 147, an activation date input section 148, and an invalidation date input section 149.

[0214] The facial image of the user is displayed on the facial image display section 141. The administrator can delete the facial image on the facial image display section 141 by pressing the delete button 151. When the administrator is unable to acquire a facial image by the registration device 4, the administrator can select a prepared facial image (image file) as the display target by pressing the image selection button 152. Alternatively, the administrator can display the facial image selected by the image selection button 152 on the facial image display section 141 in place of the facial image acquired by the registration device 4. The facial image prepared by the administrator is not limited to the actual facial image of the user, and may be a dummy image (for example, an avatar image of the user).

[0215] The user's first and last name are input in the name input section 142. The user ID (such as an employee number) is input in the user ID input section 143. An ID capable of identifying the user (such as a series of unique numbers assigned in the order of user registration) is input in the identification ID input section 144. A comparison group is input in the matching group input section 145. An access permission group indicating an administrator group that can access the user's information is input in the access permission group input section 146. Additional information can be input in the additional information input section 147. A date on which the face matching of the person can be enabled is input in the activation date input section 148. A date on which the face matching can be disabled is input in the invalidation date input section 149. By setting these activation and invalidation dates, services for multiple users can be started or ended simultaneously.

[0216] The business card information area 133 includes photographed images of business cards registered by the registration device 4 and information extracted from the photographed images of the business cards. Here, the business card information area 133 includes a business card image display section 161, an affiliation information input section 162, an activation date input section 163, and an invalidation date input section 164.

[0217] The business card image of the user is displayed in the business card image display section 161. As in the case of the face image described above, the administrator can delete the business card image or select a prepared business card image (image file) by pressing the delete button 171 or the image selection button 172.

[0218] The affiliation information input section 162 is input with the name of the company to which the user belongs, the name of the department of the user, the job title, telephone number, fax number, email, postal code, address, etc. The validity date input section 163 is input with the date on which the information on the person's business card can be valid. The invalidation date input section 164 is input with the date on which the information on the person's business card can be invalid.

[0219] When the administrator presses the registration button 167 after completing input into the desired input section, the input (added or changed) information becomes valid.

[0220] In addition, the user management screen can also manage other specific information (for example, information displayed on a payment card or information indicating the user's location within the facility) in addition to or instead of the information written on the business card.

[0221] Furthermore, the administrator can search for the required information by selecting a search tab (e.g., match group search) in the operation selection area 131 of the user management screen. On such an information search screen, each input section on the above-mentioned user management screen is displayed as blank. For example, the administrator can input the match group to be searched for into the blank match group input section 145 to search for users who belong to the input match group.

[0222] When an administrator enters search conditions in the required fields on the search screen and presses a search button (not shown), the screen transitions to a list display screen reflecting the search results (e.g., a screen showing a list of extracted users).

[0223] Next, the use of information about users will be described. Fig. 24 is an explanatory diagram showing an example of a conference room search screen (conference search screen) displayed as a management screen on the management terminal 2, etc. Fig. 25 is an explanatory diagram showing an example of an attendee display screen displayed as a management screen on the management terminal 2.

[0224] The conference room search screen shown in Fig. 24 is for the administrator to use (reference) information about the conference acquired in the face authentication process. By accessing the face management server 5 from the management terminal 2, the administrator can appropriately add information about the conference held at the facility that cannot be acquired by the face authentication device 1.

[0225] The conference room search screen displays information such as the date, time, and room number (conference room number) of conferences that have been held in the past in conference rooms of the facility. The conference room search screen can be generated using information on the entry history acquired by the face authentication device 1 linked to the locking device of the conference room door. When the administrator selects the desired conference (clicking on the check box for the conference shown at the top in FIG. 24) and presses the select button 173, the screen transitions to a search result display screen reflecting the search results.

[0226] As the search result display screen, for example, an attendee display screen including an attendee list as shown in FIG.

[0227] The attendee list includes an attendee information area 175 and a respondent information area 176. Attendee information area 175 displays face images and specific information of the attendees of the conference (here, the name of the company to which the user belongs and the user's name). Also, respondent information area 176 displays information (here, the department and name) about the person who responded to the attendee (for example, an employee of a company located in the facility).

[0228] FIG. 26 is an explanatory diagram showing an example of a personal connection search screen displayed as a management screen on the management terminal 2. As shown in FIG.

[0229] 26 is for the administrator to use (reference) information about the user 10's personal connections acquired in the face recognition process (here, information about human connections estimated from information about meeting attendees). By accessing the face management server 5 from the management terminal 2, the administrator can add necessary information about the user's personal connections as appropriate.

[0230] The personal connection search screen includes a correlation diagram display area 181 and a list display area 182. The correlation diagram display area 181 displays a diagram that visualizes personal connections related to a person selected by the administrator. Furthermore, the list display area 182 displays a personal connection list 185 related to the person (user 10) selected by the administrator. This personal connection list 185 displays the person's facial image, affiliation information (company name, department, etc.), a numerical value indicating the strength of the personal connection (personal connection strength), etc.

[0231] Such a personal connection search screen may be displayed, for example, by the administrator selecting (clicking) an area of ​​a desired attendee on the attendee display screen of FIG.

[0232] It should be noted that use of the above-mentioned conference room search screen and personal connection search screen does not have to be limited to via the management terminal 2. For example, by granting access rights as an administrator to facility users (e.g., employees of a company located in the facility), the users may be able to refer to the conference room search screen and personal connection search screen from their own information processing terminals (PCs, tablets, etc.).

[0233] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments.

[0234] In recent years, in the world of IoT (Internet of Things), attention has been focused on CPS (Cyber-Physical Systems), a new concept that creates new added value by information cooperation between physical space and cyberspace. In this context, the present embodiment can also adopt the CPS concept. That is, as a basic configuration of CPS, for example, an edge device arranged in a physical space and a cloud server arranged in a cyberspace are connected via a network, and face authentication processing can be distributed and processed by processors mounted on the edge device and the cloud server. In this face authentication processing, the edge device acquires face image data of a user captured by a camera of an authentication machine. Next, the cloud server executes a process of generating feature data from the face image data received from the edge device via the network, collates the data with the feature data of the user registered in advance, executes authentication processing of the user, and takes over the authentication result related to the authentication processing by face authentication application software, and displays the result on a display or the like in an output format defined on the application. Here, it is preferable that each piece of data generated in the edge device or cloud server is generated by a web application or the like mounted on a standardized platform. By using such a standardized platform, it is possible to improve the efficiency of building a system that includes a variety of different sensors and IoT application software. [Industrial Applicability]

[0235] The facial recognition device and facial recognition method disclosed herein have the effect of enabling users to easily recognize the progress of facial recognition while safely managing personal information of users who use a facility, and are useful as a facial recognition device and facial recognition method that executes facial recognition processing based on image data of a subject. [Explanation of symbols]

[0236] 1: Face recognition machine 2: Management terminal 3: Face recognition server 4: Registration device 5: Face management server 6: Face matching server 10: User 11: Camera (face image acquisition unit) 12: Display (display unit) 15: Control unit (processor) 16: External device 25: Authentication result adjustment unit (history information generation unit) 41A: Face camera (personal information acquisition section) 41B: Information acquisition camera (personal information acquisition section) 42: Display (display unit) 45: Control unit (processor) 62:User Management Department 64:Equipment management department 90: Image generation unit

Claims

1. A face authentication device that requests a face feature matching unit that performs a matching process of a face feature for face authentication of a user of a facility when an event occurs based on an action of the user in the facility, and presents a screen showing a progress status of the face authentication to the user, a face image acquisition unit that acquires a live image of a predetermined photographing area in the facility, the live image including a face image of the user; a display unit that displays an authentication screen having an image display area in which the live image acquired by the face image acquisition unit is displayed and an information display area in which information indicating the progress is displayed; a processor for controlling the face image acquisition unit and the display unit, The processor, a face authentication device which controls the display of the information display area on the authentication screen to present an initial screen indicating that the face authentication is in a standby state to the user approaching the face authentication device, and further, when the user's face is detected from the live image and a face image is acquired, controls the display of the information display area on the authentication screen to present a first notification screen indicating that the face authentication of the user is in progress, and further, when the face feature matching process by the face feature matching unit is completed, controls the display of the information display area on the authentication screen to present a second notification screen indicating that the face authentication of the user has been completed.

2. The processor, 2. The face authentication device according to claim 1, wherein the first notification screen displays a character or a graphic indicating that the face authentication of the user is in progress.

3. The processor, 3. The face authentication device according to claim 1, wherein the second notification screen displays a character or a graphic indicating that the face authentication of the user has been completed.

4. The processor, 4. The face recognition device according to claim 1, wherein the face image for authentication is deleted after the face recognition process is completed.

5. As the event, a request for processing the face authentication is made when the user enters a conference room in the facility; 5. The face authentication device according to claim 1, further comprising a history information generating unit that generates information on an entry history that associates a plurality of the users who have entered the conference room.

6. A face authentication method using a face authentication device, comprising: a face feature amount matching unit that performs a matching process of a face feature amount for face authentication of a user of a facility, and a screen showing a progress status of the face authentication is presented to the user when an event occurs based on an action of the user in the facility, the face authentication method comprising: Acquire a live image of a predetermined photographing area in the facility, including a facial image of the user; When an authentication screen having an image display area in which the acquired live image is displayed and an information display area in which information indicating the progress is displayed is displayed on a display unit, a display area on the authentication screen for displaying an initial screen indicating that the facial authentication is in a standby state for the user approaching the facial authentication device, and when the user's face is detected from the live image and a facial image is acquired, a display area on the authentication screen for displaying a first notification screen indicating that the facial authentication of the user is in progress is controlled, and when the facial feature matching process by the facial feature matching unit is completed, a display area on the authentication screen for displaying a second notification screen indicating that the facial authentication of the user has been completed.

Citation Information

Patent Citations

  • Access control system and access control method

    JP2010003009A

  • Server system for electronic authentication, program, electronic authentication method and electronic authentication system

    JP2018018481A

  • Mobile terminal and method for controlling same

    US20160364561A1

  • Facial recognition system, device, method and program

    WO2018181968A1