Facial recognition system

Abstract

We provide a facial recognition system that allows for flexible implementation of facial recognition while minimizing the impact on the specifications and operations of existing systems. [Solution] In the facial recognition system F, the server device comprises a first facial database 12 as an ID storage unit in which a user ID or a user code, which is a code encoded by the user ID, which is information that the service provider uses to identify the user, is registered, and a first facial data storage unit in which facial data, which is biometric data for facial recognition of the user, is registered; a first authentication program 11 as an authentication means that compares the face captured by the camera 25 of a dedicated tablet 20, which is a shooting means for capturing the face of the user on-site, with the facial data; and a screen 26 as an ID display means that displays the user ID or user code of the user on-site who has successfully been authenticated.

Description

Technical Field

[0001] The present invention relates to a face recognition system.

Background Art

[0002] In recent years, with the penetration of DX (Digital Transformation), the application range of face recognition technology is expanding. The main advantages of face recognition are the improvement of convenience and security in the authentication process. Face recognition, which is biometric authentication, does not require memorization of a password or carrying an item, unlike knowledge authentication or possession authentication. And the shapes of people's faces are very different, making it difficult for others to imitate. Also, since it can be used without contact, it is excellent in terms of hygiene and is particularly effective as a measure against infectious diseases. Furthermore, technologies for dealing with changes in environmental illuminance and age have reached a practical level, ensuring stable authentication accuracy. Due to these characteristics, face recognition is expected to be applied in various fields.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, in order to replace the conventional authentication means with face recognition, it is necessary to replace the authentication device and change the operations related to authentication. Therefore, when changing the authentication means of an already operating system, depending on the scale of the system, a large-scale operation may be required.

[0005] In view of such problems, an object of the present invention is to provide a mechanism that can flexibly introduce face recognition while suppressing the influence on the specifications and operations of an existing system. [Means for solving the problem]

[0006] To solve the above problems, the facial recognition system of the present invention comprises: an ID storage unit in which a user ID or a user code, which is information used by a service provider to identify a user, is registered; a facial data storage unit in which facial data, which is biometric data for facial recognition of the user, is registered; a shooting means for capturing the user's face; an authentication means for comparing the face captured by the shooting means with the facial data; and an ID display means for displaying the user ID or user code of the user who has successfully been authenticated.

[0007] For example, retail stores, rental shops, libraries, etc., may use membership cards to identify individuals and provide services tailored to those members, as well as manage their usage history and points. A system equipped with a memory unit that stores an ID (user ID, user code) that identifies service users and their facial data, displays the user's ID or code after facial recognition, and then allows for conventional operations using the ID or code. In other words, by presenting the ID or code, normally written on a membership card, to the service provider via facial recognition, the service provider can verify the user's identity through facial recognition (preventing impersonation) while providing conventional services using the membership card. This mechanism allows for the flexible introduction of facial recognition into existing services without modifying the specifications or operations of the service provider's current system. Furthermore, service users benefit from being able to prove their identity without carrying or taking out their membership card.

[0008] In this case, the service provider has a provider system, which is another information processing system operated by the service provider that is different from the facial recognition system, and the provider system has a user database in which the user IDs are registered, and the service provider can identify the user in the provider system by reading the encoded user ID displayed by the ID display means with a scanner, or by inputting the user ID displayed by the ID display means into the provider system.

[0009] Furthermore, it is desirable that the aforementioned imaging means and the aforementioned ID display means are implemented in a facial recognition device provided by the service provider. This is to enhance the convenience of facial recognition for users and to ensure the objectivity of user identity verification.

[0010] Furthermore, the facial recognition system of the present invention may include a server device connected to the facial recognition device via a network, wherein the facial recognition device has the imaging means and the ID display means, and the server device has the ID storage unit, the face data storage unit and the authentication means. By centralizing the facial recognition process on the server device, that is, by creating a so-called cloud-type facial recognition system, the functions of the facial recognition device are simplified, and cost efficiency is increased when using a large number of facial recognition devices. Alternatively, the facial recognition system of the present invention may be configured such that the facial recognition device has the ID storage unit, the face data storage unit, the imaging means, the authentication means and the ID display means. By completing the facial recognition process on equipment at the service provision site, that is, by creating a so-called edge-type facial recognition system, smoother operation can be achieved.

[0011] Furthermore, the facial recognition system of the present invention is equipped with registration means for acquiring the user's facial image when registering the facial data, and it is desirable that the registration means be provided using a user terminal, which is an information terminal owned by the user. This is because it is inconvenient for the user to go to a service location and have their face photographed in order to use the facial recognition system.

[0012] Furthermore, it is preferable that the user be able to associate and register multiple user IDs or user codes for multiple services with a single facial image or single facial data. This is because it is inconvenient to individually photograph one's face at multiple service locations in order to use the facial recognition system.

[0013] Furthermore, the facial recognition system of the present invention comprises a facial recognition device that captures the user's face, and a server device connected to the facial recognition device via a network. The facial recognition device and the server device each have an ID storage unit, a facial data storage unit, and an authentication means. The face captured by the facial recognition device is first verified by the authentication means of the facial recognition device, and if verification fails, it is verified by the authentication means of the server device. This configuration allows for both the advantages of a cloud-based facial recognition system, which enables the registration and verification of a vast amount of facial data, and the advantages of an edge-based facial recognition system, which offers high-speed processing.

[0014] In this case, when authentication is successful by the authentication means of the server device, it is desirable for the facial recognition device to download the authenticated user's facial image or facial data, and the user ID or user code from the server device or other device. This is because, generally, users of the service tend to use the service repeatedly in the same location. [Effects of the Invention]

[0015] Thus, the facial recognition system of the present invention makes it possible to flexibly introduce facial recognition while minimizing the impact on the specifications and operations of existing systems. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram outlining the facial recognition system F. [Figure 2]It is an explanatory diagram of the usage method of the face recognition system F in the store of a retail store. [Figure 3] It is an arrangement diagram showing the functional configurations of the face recognition system F and the retail store system 90. [Figure 4] It is a use case diagram of the face recognition system F from the customer's perspective and the store clerk's perspective. [Figure 5] It is a sequence diagram showing the operation process of the face recognition system F. [Figure 6] It is a block diagram explaining the functional configuration of a modified example of the face recognition system F. [Figure 7] It is an arrangement diagram showing the functional configuration of the face recognition system Fb according to another modified example. [Figure 8] It is a sequence diagram explaining the face recognition procedure using the face recognition system Fb and the input procedure of the membership number N.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The face recognition system F described below is an example of the face recognition system of the present invention introduced into a retail store such as a shopping mall. The retail store in this example issues membership cards to customers. The store clerk of the retail store obtains the membership number of the customer by scanning the barcode written on the membership card, and provides services according to the customer, such as applying discounts, issuing coupon tickets, and confirming adults and minors. The purchase history of each customer and the points of the point program are also managed by the membership number.

[0018] The face recognition system F in this example is a system independent of existing systems such as the POS system, CRM system, and OMS (Order Management System) operated by this retail store. The face recognition system F identifies customers by face recognition in the store of the retail store and displays the membership number of the customer as a barcode for the store staff. The store staff inputs the membership number of the customer into the company's system by scanning the barcode and then provides services according to the conventional procedures. That is, the face recognition system F is a system for performing the act of presenting a membership card by a customer through face recognition. In this way, the retail store has introduced the face recognition mechanism into its own services while suppressing the impact on the specifications and operations of the existing systems. Hereinafter, embodiments will be described as examples for this feature of the face recognition system F and each associated feature.

[0019] <System Overview> FIG. 1 is a block diagram showing an overview of the face recognition system F of this embodiment. The face recognition system F of this embodiment mainly includes a server device 10, a dedicated tablet 20 (face recognition device), and a user app 30 (registration means). As described above, the face recognition system F is a system independent of various retail store systems 90 (provider systems) including the POS system operated by a retail store (service provider).

[0020] FIG. 2 is an image diagram for explaining the usage method of the face recognition system F in the store of the retail store. As shown in FIG. 2(a), a POS terminal 91 and a dedicated tablet 20 of the face recognition system F are installed at the cash register of the retail store. The dedicated tablet 20 of this embodiment is a two-sided tablet terminal and has a surface facing the customer side and a surface facing the store staff side. This dedicated tablet 20 is a device prepared in advance by the retail store side and installed at the cash register. Note that the face recognition system F of this embodiment is not a system developed by this retail store for its own use, but a service provided by a third party other than this retail store. The retail store is in the position of using that service. And the dedicated tablet 20 is a device lent by that third party to the retail store.

[0021] As shown in Figures 2(a) and 2(b), the customer faces the camera 25 (shooting means) of the dedicated tablet 20 at checkout and has their face photographed. If customer authentication is successful, a barcode B (user code) is displayed on the store clerk's screen 26 (ID display means), and the store clerk reads the barcode B with the scanner 911. This inputs the member number N (user ID), which is information that identifies the customer, into the POS terminal 91. The store clerk then provides service according to the conventional procedure. Note that the hardware configuration of the dedicated tablet 20 in this form is just one example, and it can be replaced with a combination of general tablet devices, such as smartphones. The dedicated tablet 20 is not limited to a two-screen type; a single-screen tablet device could be mounted on a rotating stand, supported by a simple, non-fixed stand that the store clerk can easily change the orientation of, the tablet device could be pointed towards the store clerk and a separate camera pointed towards the customer, the tablet device could be positioned so that both the customer and the store clerk can see the screen, or the store clerk could lean in to scan the tablet device pointed towards the customer. Furthermore, with tablet devices that have foldable screens, such as OLED displays, or screens that are pre-folded, it's possible to have one side facing the staff and the other side facing the customer.

[0022] <Functional Configuration> Figure 3 is a layout diagram showing the functional configuration of the facial recognition system F and the retail store system 90. The functions of the facial recognition system F and the retail store system 90 will be described below with reference to Figure 3.

[0023] As shown in Figure 3(a), the dedicated tablet 20 and user application 30 of the facial recognition system F are connected to the server device 10. In this configuration, the dedicated tablet 20 and user application 30 send and receive data with the server device 10 via the internet. As mentioned above, the dedicated tablet 20 is equipped with a camera 25 and a screen 26. The user application 30 is a tool for retail store customers to upload their membership information, including their membership number N, and a facial image P for registration to the server device 10. The server device 10 is a server that performs facial recognition processing. In other words, the facial recognition system F in this configuration is a so-called cloud-based facial recognition system. This relaxes the functional and performance requirements for the dedicated tablet 20, and improves cost efficiency when deploying the dedicated tablet 20 at multiple cash registers in a store.

[0024] The server device 10 in this configuration is internally divided into an authentication server 10a specialized for facial recognition processing and an extension server 10b for extending and customizing its functions. The authentication server 10a has a first face database 12 (ID storage unit, face data storage unit) in which face data T, which is biometric data for facial recognition of customers, is registered, and a first authentication program 11 (authentication means) that compares the face captured by the camera 25 with the face data T. Face data T here refers to comparison model data in which features are extracted from the customer's face image P acquired by the user application 30, and is also sometimes called a face template, face vector, feature quantity, face model, or face map. In addition, the first face database 12 also registers an image of a barcode B on which each customer's membership number N is encoded, associated with that face data T. The extension server 10b has a user database 14 (ID storage unit) in which each customer's membership information, including membership number N, and face image P are registered, and an encoding program 13 that converts membership number N into barcode B. The customer's membership information and face image P, obtained by the user application 30 and uploaded to the server device 10, are registered in the user database 14. Subsequently, the first authentication program 11 extracts features from the face image P using the same algorithm as during authentication to generate face data T, which is registered in the first face database 12. Then, the encoding program 13 converts the customer's membership number N into a barcode B and registers the image in the first face database 12.

[0025] In this configuration, the extension server 10b is a virtual server built using a cloud hosting service. On the other hand, the authentication server 10a in this configuration is a server of an external service that provides facial recognition functionality. In other words, in the facial recognition system F, the facial recognition processing is delegated to an external service. The extension server 10b is not limited to a cloud server; it may also be an on-premises server installed within the LAN of its provider (in this case, a third party, not a retail store).

[0026] When the server device 10 successfully authenticates a face captured by the camera 25 of the dedicated tablet 20, it transmits an image of the customer's barcode B to the dedicated tablet 20. The dedicated tablet 20 then displays the barcode B on the screen 26.

[0027] The functional distribution between the server device 10 and the dedicated tablet 20 in the facial recognition system F is an example, and the arrangement of these functions is not limited to the configuration shown in this example. These functions can be appropriately changed according to the scale of the facial recognition system F, the number of users, the budget, and other implementation and operational constraints and requirements. For example, the encoding program 13 of the server device 10 may be located on the dedicated tablet 20. In this case, the server device 10 transmits the customer's membership number N to the dedicated tablet 20 upon successful authentication, and the dedicated tablet 20 converts the membership number N into a barcode B each time. Alternatively, for example, the first authentication program 11 could be located on the dedicated tablet 20, allowing remote access to the first facial database 12 of the server device 10. Or, all the functions of the authentication server 10a could be located on the dedicated tablet 20, creating an edge-type facial recognition system. In other words, it is possible to complete the customer's facial recognition process using only in-store equipment, enabling smoother operation. Furthermore, in this edge-type facial recognition system, instead of barcode B, a membership number N is linked to the facial data T, and an encoding program 13 is also provided to convert the customer's membership number N into barcode B each time authentication is successful. In this case, it is not necessary to generate barcode B in advance.

[0028] The user application 30 is a dedicated application installed on a smartphone (user terminal) owned by a customer of a retail store. The user application 30 has a registration form 32 for entering member information, including a member number N, and uses the smartphone's camera 31 to acquire the customer's facial image P. The acquired member information and facial image P are uploaded to the server device 10 and registered in the user database 14. The user application 30 may also be dedicated software installed on the customer's PC or a web application. In this configuration, customers do not need to go to a retail store to have their face photographed or fill out an application form in order to use facial recognition; they can complete the preparation using their smartphone.

[0029] As shown in Figure 3(b), the retail store system 90 in this configuration includes a POS terminal 91 and a customer database 92 (user database). The customer database 92 is the database of the CRM system operated by the retail store, and the same member number N registered in the facial recognition system F is registered therein. Note that the functions shown in Figure 3(b) are only a small part of the many functions of the retail store system 90. In addition, in this configuration of the retail store system 90, a one-dimensional code, barcode B, is used to represent the member number N, but this could be a two-dimensional code, or any other code.

[0030] Furthermore, the facial recognition system F is not a system used only in the retail store in this example, but is implemented in multiple different retail stores. Each customer can use the user app 30 to associate and register multiple retail store membership numbers N with facial data T generated from a single facial image P. To achieve this, the user database 14 of the facial recognition system F stores information for each customer, i.e., each user of the facial recognition system F, in addition to the facial image P, including a dedicated ID and password for the facial recognition system F, name, contact email address, phone number, token to identify the customer's device, information of the retail stores that have indicated their intention to use facial recognition (so-called opt-in), membership number N for each opt-in retail store, retail store-specific membership information other than membership number N, and facial recognition usage history. The dedicated tablet 20 of each retail store also transmits information identifying that retail store when transferring the facial image P. Each customer does not need to take separate photos of their face at each retail store to use facial recognition at multiple different stores; they can prepare to use facial recognition at all stores with just one photo taken using the user app 30.

[0031] Figure 4 is a use case diagram of the facial recognition system F from the customer's perspective and the store clerk's perspective. From the customer's perspective in a retail store, the interface of the facial recognition system F consists of two parts: a user application 30 in which the customer enters their membership information and takes a photo of their face, and a dedicated tablet 20 in which the customer takes a photo of their face instead of presenting a membership card. From the store clerk's perspective, the interface of the facial recognition system F consists only of the dedicated tablet 20, and specifically only the screen 26 that displays barcode B. The scanner 911 (see Figure 2) that the clerk uses to read barcode B displayed on screen 26 is the same scanner 911 that reads barcode B written on a membership card. In other words, the clerk's operation is almost the same as when a customer presents a membership card.

[0032] More specifically, the facial recognition system F does not use facial recognition to input the membership number by scanning the barcode B written on the membership card, but rather uses facial recognition to display the barcode B written on the membership card, which is one step earlier. It is possible to identify the membership number N using facial recognition alone, and in fact, facial recognition, being a biometric authentication method, can identify individuals with a higher degree of accuracy than the membership card, which is an owner authentication method. However, in order to allow the existing input method (scanner 911) of the retail store system 90 to be used as is, the degree of coupling between the facial recognition system F and the retail store system 90 is deliberately kept loose by limiting the process to displaying the barcode B.

[0033] In this way, the facial recognition system F independently stores a barcode B representing the customer's membership number N and the customer's facial data T. After facial recognition of the customer, the system displays the customer's barcode B, meaning that the barcode B printed on the membership card is presented to the store clerk via facial recognition, allowing the clerk to perform the same operations as traditional operations using a membership card. Furthermore, as mentioned above, since the facial recognition system F is functionally separate from the retail store system 90, there is no need to modify the specifications of the retail store system 90. In addition, customers benefit from being able to prove their identity without carrying or taking out their membership card. This convenience is expected to increase the frequency of customer identification, leading to synergistic effects such as improved accuracy in so-called one-to-one marketing and targeted marketing.

[0034] <Operation Process> Figure 5 is a sequence diagram showing the operation process of facial recognition system F. The following explanation will refer to Figure 5 to describe the customer's facial registration procedure, facial recognition procedure, and the procedure for entering the member number N.

[0035] When a customer of a retail store begins using the facial recognition system F, it is necessary to obtain the customer's facial image P using the user application 30. There are two main methods for registering a customer's face. (Face registration 1) First, the customer installs the user app 30 on their smartphone. Then, they enter their membership information, including their membership number N, into the registration form 32 and take a picture of their face with the camera 31. This is uploaded to the server device 10 and registered in the user database 14. Subsequently, face data T and barcode B are generated. Here, to prevent errors in entering the membership number N, the input of the membership number N may be performed by reading the barcode B written on the membership card with the camera 31, and the encoding program 13 reconstructs the membership number N from that barcode B. (Face registration 2) Another method of facial registration is to prepare a separate user application for the retail store, the retail store app 39, and distribute a registration form 32 or web form with the member number N pre-filled to the customer from the retail store. Specifically, the retail store uses the retail store app 39 to pre-enter the customer's member number N and contact information and sends a registration invitation to the customer. The server device 10 receives this and sends a registration guide to the customer's smartphone via email or SMS. This registration guide contains a download link for the user app 30 and a link to the registration form 32 with the member number N pre-filled. The customer receives the registration guide, completes the remaining member information in the registration form 32, etc., and takes a photo of their face. This is uploaded to the server device 10 and registered in the server device 10's user database 14, and facial data T and barcode B are generated.

[0036] As mentioned above, customers may use the facial recognition system F at multiple retail stores. If a customer has already installed the user app 30 and has entered basic membership information and uploaded a facial image P, they can use facial recognition at that retail store simply by opting in.

[0037] Next, we will explain the facial recognition procedure after facial registration is complete, as well as the procedure for entering member number N. (Facial recognition) When a customer's face is captured by the dedicated tablet 20, the tablet 20 transfers the face image P to the server device 10. The server device 10's first authentication program 11 then compares this with the face data T in the first face database 12. If authentication is successful, the server device 10 sends an image of the customer's barcode B to the dedicated tablet 20. Upon receiving the image of barcode B, the dedicated tablet 20 displays it on the screen 26. (Restoring membership number) The retail store employee reads barcode B displayed on screen 26 with scanner 911 and enters member number N into the retail store system 90.

[0038] <Variation> Figure 6 is a block diagram showing the functional configuration of a modified version of the facial recognition system F. Of these, Figure 6(a) is an example of a configuration in which the image of barcode B is directly captured from the customer's membership card, etc., without using the membership number N. Figure 6(b) is an example of a configuration in which only the membership number N is used without using barcode B.

[0039] (Variation 1) In the above embodiment, the facial recognition system F obtains the customer's membership number N from the registration form 32 of the user application 30 and converts it into a barcode B using the encoding program 13. However, in environments where all customers' barcodes B are known, such as when all customers using the facial recognition system F have their own membership cards, the membership number N and the encoding program 13 can be omitted (Figure 6(a)). In this configuration, each customer is asked to take a picture of the barcode B on their membership card using the user application 30, and that image is associated with facial data T and registered in the first facial database 12. When facial recognition of a customer is successful, the image of the barcode B is sent to a dedicated tablet 20 and displayed.

[0040] (Modification 2) In the facial recognition system F of the above embodiment, a store employee reads barcode B with scanner 911 and inputs member number N into POS terminal 91 (retail store system 90). However, in smaller retail stores, for example, barcode B may not be displayed on the member card, and the employee may manually input the member number N from the member card into the store's system. In this case, since barcode B is not used, the encoding program 13 can be omitted (Figure 6(b)). In this configuration, instead of barcode B, the member number N is associated with the facial data T and registered in the first facial database 12. When facial recognition of a customer is successful, the member number N can be sent to a dedicated tablet 20 and displayed.

[0041] (Variation 3) Figure 7 is a layout diagram showing the functional configuration of a facial recognition system Fb according to another modification. In this modification, the facial recognition device installed at the cash register counter of a retail store is replaced from a dedicated tablet 20 to an authentication device 20b. The authentication device 20b further includes a second facial database 22, which is a proprietary ID storage unit and facial data storage unit, and a second authentication program 21, which is a proprietary authentication means. The second facial database 22, like the first facial database 12, is a database in which customer facial data T is registered in association with barcode B, and the authentication algorithm of the second authentication program 21 is the same as that of the first authentication program 11.

[0042] Figure 8 is a sequence diagram illustrating the facial recognition procedure using the facial recognition system Fb and the procedure for entering the member number N. Note that the procedure for registering the face with the server device 10 is the same as for the facial recognition system F. Facial recognition using the facial recognition system Fb is first performed at the in-store authentication device 20b (edge ​​authentication). If authentication fails there, the facial image P is transferred to the server device 10, where facial recognition is performed again (cloud authentication). The specific processing flow will be explained below with reference to Figure 8.

[0043] When the authentication device 20b captures a customer's face with the camera 25, the authentication device 20b uses its second authentication program 21 to compare the features of the face image P with the face data T in the second face database 22. If authentication is successful, the barcode B is displayed on the screen 26 of the authentication device 20b. The retail store employee reads the barcode B displayed on the screen 26 with the scanner 911 and enters the member number N into the retail store system 90.

[0044] If the authentication device 20b fails to authenticate, that is, if the customer's face data T is not registered in the second face database 22, it transfers the face image P to the server device 10. The first authentication program 11 in the server device 10 then compares its features with the face data T in the first face database 12. If authentication is successful, the server device 10 sends the customer's face data T and barcode B image to the authentication device 20b. The authentication device 20b displays the received (downloaded) barcode B on the screen 26 and registers the customer's face data T and barcode B in the second face database 22. The retail store employee reads the barcode B displayed on the screen 26 with the scanner 911 and enters the member number N into the retail store system 90.

[0045] In this modified version, the advantages of a cloud-based facial recognition system, such as the ability to register and match a vast amount of facial data T, and the advantages of an edge-based facial recognition system, such as high processing speed, are combined. On the other hand, there is a limit to the amount of facial data T that can be registered in the desktop authentication device 20b. Therefore, the server device 10 registers only the facial data T of customers that it has successfully authenticated in the second facial database 22, so that only the facial data T necessary for the authentication device 20b is registered. This is because, generally, customers of retail stores tend to use services repeatedly at the same location.

[0046] <Other Embodiments> In the above embodiment, an example of introducing the facial recognition system F to a retail store was described. The facial recognition system of the present invention can be introduced to places other than retail stores, such as various rental shops, libraries, and hospitals. In this case as well, it is conceivable that the membership number N and barcode B of a membership card could be displayed via facial recognition, or that the medical card number could be displayed via facial recognition.

[0047] Furthermore, while the facial recognition system F in the above embodiment is primarily intended to improve customer convenience in retail stores, it also has the effect of strengthening the traditional possession authentication using membership cards with biometric authentication. In other words, it also has the effect of preventing impersonation of service users for service providers. By applying this effect, for example, driver's license numbers, passport numbers, residence card numbers, individual numbers (My Number), insurance card numbers, etc., can be registered in association with facial data T, and by users opting in to use these numbers, these numbers can be presented or obtained through facial recognition. This can prevent impersonation, for example, when the police enforce traffic violations or during immigration checks. Facial recognition devices such as dedicated tablets 20 capable of displaying these numbers should, of course, be kept under strict management and only permitted to be possessed by facilities and personnel of authorized institutions. In addition, to prevent fraud during facial registration, restrictions should be placed on the facial registration procedure when registering these numbers with the facial recognition system F. For example, for these numbers, it is conceivable to prohibit facial registration by the user app 30 and instead have staff at the nearest city hall or ward office verify the user's identity before taking a photograph, or to restrict registration to only via a time-limited link in the registration instructions sent via SMS, i.e., a time-limited link verified by phone number.

[0048] Although embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and various modifications can be made without departing from the spirit of the invention. [Explanation of symbols]

[0049] F, Fb: Facial recognition system, N: Membership number (User ID), B: Barcode (User code), P: Facial image, T: Facial data, 10: Server device, 10a: Authentication server, 10b: Expansion server, 11: First authentication program (Authentication means), 12: First facial database (ID storage unit, facial data storage unit), 13: Encoding program, 14: User database (ID storage unit), 20: Dedicated tablet (Facial recognition device), 20b: Authentication device (Facial recognition device), 21: Second authentication program (Authentication means), 22: Second facial database (ID storage unit, facial data storage unit), 25: Camera (Shooting means), 26: Screen (ID display means), 30: User application (Registration means), 31: Camera, 32: Registration form, 39: Retail store application, 90: Retail store system (Provider system), 91: POS terminal, 911: Scanner, 92: Customer DB (User database)

Claims

1. An ID storage unit in which a user ID, which is information that identifies the user provided by the service provider, or a user code, which is a code that encodes the user ID, is registered. A face data storage unit in which face data, which is biometric data for the user's face recognition, is registered, A means for photographing the user's face, Authentication means for comparing the face captured by the aforementioned shooting means with the aforementioned face data, The system includes an ID display means that displays the user ID or user code of the user who has successfully authenticated, Facial recognition system.

2. The service provider has a provider system, which is another information processing system operated by the service provider, that is different from the facial recognition system. The provider system has a user database in which the user ID is registered. The service provider identifies the user in the provider system by scanning the user code displayed by the ID display means, or by inputting the user ID displayed by the ID display means into the provider system. The facial recognition system according to claim 1.

3. The aforementioned shooting means and the aforementioned ID display means are implemented in a facial recognition device, which is equipment provided by the service provider. The authentication system according to claim 1.

4. The facial recognition device is connected to a server device via a network, The aforementioned facial recognition device is The aforementioned photographic means, The aforementioned ID display means is provided, The server device is The aforementioned ID storage unit, The aforementioned face data storage unit, Having the aforementioned authentication means, The facial recognition system according to claim 3.

5. The aforementioned facial recognition device is The aforementioned ID storage unit, The aforementioned face data storage unit, The aforementioned photographic means, The authentication means, Having the aforementioned ID display means, The facial recognition system according to claim 3.

6. The system includes a registration means for acquiring the user's facial image when registering the aforementioned facial data. The registration means is provided using a user terminal, which is an information terminal owned by the user. The facial recognition system according to claim 1.

7. The user may register by associating a single facial image or single facial data with the user ID or user code of multiple services. The facial recognition system according to claim 1.

8. A facial recognition device that captures the user's face, The facial recognition device and the server device connected via a network are provided. The facial recognition device and the server device each have the ID storage unit, the facial data storage unit, and the authentication means, The face captured by the facial recognition device is first verified by the authentication means of the facial recognition device, and if authentication fails, it is verified by the authentication means of the server device. The facial recognition system according to claim 1.

9. When authentication is successful by the authentication means of the server device, the facial recognition device downloads the facial image or facial data, and the user ID or user code of the authenticated user from the server device or other device. The facial recognition system according to claim 8.

Images