Facial recognition system

Abstract

By making various facial recognition systems available in a unified and cross-cutting manner, we aim to make facial recognition more accessible and convenient. [Solution] This is solved by a facial recognition system comprising a facial recognition engine, a user terminal that acquires an image of the user's face or scan data of its three-dimensional shape (hereinafter referred to as "facial image, etc."), and a provider tool used by the provider of each service that uses facial recognition, wherein some services require the service provider to perform an operation to authorize a specific user to use the service using the provider tool (hereinafter referred to as "authorization operation"), and others can be used without requiring the authorization operation.

Description

Technical Field

[0001] The present invention relates to a face recognition system.

Background Art

[0002] In recent years, with the penetration of DX (Digital Transformation), the application range of face recognition technology has been expanding. The main advantages of face recognition are the improvement of convenience and security in the authentication process. Face recognition, which is a biometric authentication, does not require memorizing a password or carrying an item unlike knowledge authentication or possession authentication. And the shapes of people's faces are very different, making it difficult for others to imitate. Also, since it can be used without contact, it is excellent in terms of hygiene and is particularly effective as a measure against infectious diseases. Furthermore, technologies that take into account changes in environmental illuminance and age have reached a practical level, ensuring stable authentication accuracy. Due to these characteristics, face recognition is expected to be applied in various fields.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, conventional face recognition systems require taking pictures of faces and registering them for each device (such as smartphones) or facility or organization in which they are introduced, and there were also various ways to start using them. Therefore, when using face recognition, users had to receive explanations or guides each time or investigate the setting method by themselves. This effort has raised the hurdle for face recognition and hindered its spread.

[0005] In view of such problems, an object of the present invention is to make face recognition more accessible and easier by enabling various face recognition systems to be used in a unified and cross-cutting manner. [Means for solving the problem]

[0006] To solve the above problems, the facial recognition system of the present invention comprises a facial recognition engine, a user terminal that acquires an image of a user's face or scan data of its three-dimensional shape (hereinafter referred to as "facial image, etc."), and a provider tool used by the provider of each service that uses facial recognition, wherein the service provider of the service includes those for which the provider of the service must perform an operation to authorize a specific user to use the service using the provider tool (hereinafter referred to as "authorization operation"), and those that can be used without requiring the authorization operation. [Effects of the Invention]

[0007] Thus, the facial recognition system of the present invention makes facial recognition more accessible and easier to use. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram outlining the facial recognition system F. [Figure 2] This is a layout diagram showing the functional configuration of facial recognition system F. [Figure 3] This is an illustrative diagram explaining the 20 types of facial recognition engines. [Figure 4] This is a use case diagram of facial recognition system F from a user's perspective. [Figure 5] This is a sequence diagram showing the process by which a user registers their face in the facial recognition system F. [Figure 6] This is a sequence diagram showing a modified version of the pre-check process for facial image P. [Figure 7] This is a schematic diagram illustrating how users can select services. [Figure 8] This is a sequence diagram showing the process of registering and unregistering services in the facial recognition system F. [Figure 9]Figure 9(a) is a sequence diagram showing the registration process for a service that requires approval from the service provider. Figure 9(b) is a sequence diagram showing the registration process for a service that requires prior invitation from the service provider. [Figure 10] This is a sequence diagram showing the facial recognition process in each facial recognition engine 20. [Figure 11] This is a sequence diagram illustrating another embodiment of the facial recognition system of the present invention. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The facial recognition system F described below is a system that centrally wraps facial recognition engines used in various services and provides users with a unified UI for using them. Below, this feature of the facial recognition system F and each of its associated features will be described using embodiments as examples.

[0010] In this invention, "services" refer to, for example, sales services, food and beverage services, transportation services, accommodation services, real estate services, payment services, childcare and nursing care services, entertainment services, etc., provided by businesses or public institutions. The services of this invention are not limited to comprehensive services, but also include partial services, such as payment services within food and beverage services provided by businesses. Furthermore, they are not limited to external services, but also include services such as locking and unlocking provided by access control systems for company offices or homes. In other words, the meaning of "services" as used here is extremely broad.

[0011] Furthermore, the term "face recognition engine" in this invention refers to a system that identifies an individual from input facial data, and includes not only on-premise engines provided as independent products, but also cloud service engines, engines embedded in devices such as smartphones, or engines that combine these.

[0012] <System Overview> Figure 1 is a block diagram illustrating the overview of the facial recognition system F. This embodiment of the facial recognition system F mainly consists of a management server 10, multiple facial recognition engines 20, a user application 30, and a service provider tool 39. The management server 10 holds data about users of the facial recognition system F and sends and receives data with each facial recognition engine 20. The user application 30 is a UI in which users of the facial recognition system F take pictures of their faces and input personal information. The service provider tool 39 is a UI in which service providers input data and configure settings necessary for their services. In the facial recognition system F, service providers do not directly operate the facial recognition engines 20 used in their services, but rather operate them via the management server 10 using the service provider tool 39.

[0013] <Functional Configuration> Figure 2 is a layout diagram showing the functional configuration of the facial recognition system F. As shown in Figure 2, each facial recognition engine 20 and user application 30 of the facial recognition system F is connected to the management server 10 so as to be able to communicate with it. Although the service provider tool 39 is omitted in Figure 2, the service provider tool 39 is also connected to the management server 10 so as to be able to communicate with it. In this configuration, each facial recognition engine 20, user application 30, and service provider tool 39 send and receive data with the management server 10 via the internet.

[0014] (User app) The user app 30 is a dedicated application installed on the smartphone (user terminal) owned by the user. The user app 30 has a registration form 32 for the user to input user information 111 including the user's personal information and information regarding payment means, a face registration program 33 for uploading the face image P captured by the camera 31 of the smartphone to the management server 10, and a service addition button 34 for the user to register a service that uses face authentication. The user app 30 is not limited to the form of a smartphone app, and may be dedicated software installed on the user's PC or a web application. In this embodiment, the face image P is described as a general digital photo image, but depending on the type of face authentication engine 20 used, it is also conceivable to acquire scan data capturing the three-dimensional shape of the face with the user app 30. In the present invention, acquiring the scan data of the user's face is also referred to as "shooting".

[0015] (Face authentication engine) Each face authentication engine 20 includes a camera 21 which is a photographing device for photographing the face of the person facing it, and a screen 22 which is a general-purpose touch panel. The screen 22 in this embodiment displays the face being photographed by the camera 21, messages to the user, input forms, etc. Each face authentication engine 20 also includes a face database 25 in which a face template T which is biometric data for user face authentication is stored, and an authentication program 26 (authentication means) for comparing the face photographed by the camera 21 with the face template T. The face template T referred to here is model data for comparison obtained by extracting the features from the user's face image P photographed by the user app 30, and may also be called, for example, a face vector, feature quantity, face model, face map, etc. Since the face authentication engine 20 is individually selected to be suitable for the nature and environment of the service provided using this, there are various products and services with different generation algorithms for the face template T mixed in the face authentication engine 20 of this embodiment.

[0016] The face recognition engine 20 of this form is roughly divided into two types depending on its usage, scale of use, etc. FIG. 3 is an image diagram for explaining the types of the face recognition engine 20. As shown in FIG. 3(a), the face recognition engine 20 of this form includes a so-called edge-type face recognition engine in which an authentication program 26 and a face database 25 are also built in a housing in which a camera 21 and a screen 22 are mounted, and as shown in FIG. 3(b), a so-called cloud-type face recognition engine in which a front end such as a camera 21 and a screen 22 and a backend such as an authentication program 26 and a face database 25 are connected via the Internet. The edge type in this form includes those in which a device holding the authentication program 26 and the face database 25 is connected to the same local network as the camera 21. The "local network" mentioned here means a LAN to which the camera 21 is connected or that the camera 21 is connected by a PAN. Further, the cloud type in this form includes those in which the authentication program 26 is in the same housing as the camera 21 and remotely accesses the face database 25 on the cloud.

[0017] (Management server) Returning to FIG. 2, the explanation will be continued. The management server 10 has a user database 11 in which user account data including the above-mentioned user information 111 and face image P is registered, and a face recognition engine database 12 in which identification information, address information, product name and version of each face recognition engine 20, and other information necessary for the management of each face recognition engine 20 are registered. The user database 11 of this form also includes a user ID 113 (described later), which is unique identification information linked to the user account.

[0018] Furthermore, the management server 10 has a pre-check program 16 that checks the data quality of face images P uploaded from the user application 30, and a distribution program 17 that distributes face images P that have passed the quality check to each face recognition engine 20. Face images P that have passed the quality check are registered in the user database 11. When each face recognition engine 20 receives a face image P from the management server 10, an authentication program 26 extracts features from the face image P using the same algorithm as during authentication and generates a face template T. The generated face template T is registered in the face database 25.

[0019] In this configuration, the management server 10 is a virtual server built using a cloud hosting service. The management server 10 is not limited to a cloud server; it may also be an on-premise server located within the operator's LAN. Furthermore, the management server 10 may be composed of a group of multiple servers that are physically or functionally separated. Also, the functions and data of the management server 10 shown in Figure 2 represent only a small part of the functions and data that the management server 10 possesses. Moreover, it is not necessary to develop all of the various functions of the management server 10 in-house; they can also be implemented using external services. For example, the management server 10 in this configuration has a payment program 18 for processing user payment requests and a PIN code 112 for identity verification during payment, but the actual payment processing may be delegated to an external service specializing in that. Functions delegated to external services are also included in the functions of the management server 10.

[0020] (Use case) Figure 4 is a use case diagram of the facial recognition system F from the user's perspective. The interface of the facial recognition system F from the user's point of view consists of two parts: a user application 30 in which the user inputs their user information 111, takes a photo of their face, and selects (registers) the services they will use, and a facial recognition engine 20 (camera 21) that takes a photo of the user's face at the service provision site. Distribution of facial images P to each facial recognition engine 20 and coordination with each service provider regarding registration of services to be used are performed via the management server 10. In this way, by setting up a management server 10 that wraps various facial recognition engines 20 and services, and by providing a single UI, the user application 30, users can use facial recognition for various services in a unified and cross-platform manner. This makes facial recognition system F in this form more accessible and easier to use.

[0021] <Facial image pre-check function> Figure 5 is a sequence diagram showing the process by which a user registers their face in the facial recognition system F. The pre-check function of the facial image P in the user's face registration process will be explained below with reference to Figure 5.

[0022] (Face registration 1) As shown in Figure 5(a), when a user registers their face with the facial recognition system F, the user enters the necessary user information 111 into the registration form 32 of the user application 30 and takes a picture of their face with the camera 31. The personal information 111 and facial image P entered here are transferred to the management server 10. The management server 10 uses a pre-check program 16 to evaluate whether the data quality of the captured facial image P is at a level that allows each facial recognition engine 20 to generate a facial template T. If it is confirmed that the quality of the facial image P meets this level, the management server 10 registers the facial image P in the user database 11 and notifies the user that registration is complete. On the other hand, if the quality of the facial image P is evaluated as insufficient, the management server 10 requests the user to retake a picture of their face (not shown). The facial image P registered in the user database 11 is retained permanently or for a certain period of time and is distributed to each facial recognition engine 20 when the service is registered, as will be explained later. In this form, the process of verifying the quality of the facial image P is simplified by having the management server 10 centrally verify the quality of the facial image P on behalf of the various facial recognition engines 20. This minimizes the number of transfers and transactions for face images P, shortening the turnaround time after a user uploads a face image P.

[0023] In this embodiment, the pre-check program 16 performs preprocessing, face detection, face landmark detection, face alignment and cropping, noise reduction, image normalization, feature extraction (face template generation), and saving of the face template on the captured face image P. The program determines that the data quality of the face image P is at an acceptable level once the face template generation process is complete. In this way, this embodiment reduces the discrepancy between the evaluation result and the acceptance or rejection by each face recognition engine 20 by evaluating the quality of the face image P by mimicking the actual face template generation process performed by each face recognition engine 20. The degree of checking by the pre-check program 16 is adjustable, but it is desirable to at least confirm whether or not face detection is possible.

[0024] In this form of facial recognition system F, the management server 10 has a pre-check program 16, but the pre-check program 16 can also be implemented as a function of the user application 30. In that case, a quality check of the facial image P is performed before the facial image P is transferred to the management server 10. Alternatively, a mechanism could be considered in which the user application 30 performs a simple check of the facial image P, and then the management server 10 performs a more precise check.

[0025] Furthermore, as mentioned above, the facial recognition engine 20 of the facial recognition system F contains a mix of various products and services with different facial template T generation algorithms. Depending on the type of facial recognition engine 20, not only frontal images of the face may be required, but also images of the face at an angle or with the head tilted. Therefore, the user's facial image P, taken in accordance with the guidance of the user application 30, may contain images that are unnecessary for any of the facial recognition engines 20.

[0026] (Re-registration of facial image) Figure 5(b) is a sequence diagram showing the process when a user re-registers a face image P. Users of the face recognition system F can re-register their face image P at any time using the user application 30. This is because a person's face is not permanently unchanged, and the accuracy of the registered face image P may be insufficient. If the user has not yet selected a service, the re-registration process is the same as the face registration 1 process described above. On the other hand, if the user has already registered for one of the services, after a pre-check of the face image P, the new face image P is redistributed to the face recognition engine 20 used for the service that the user has registered for, and the face template T is updated in each face recognition engine 20.

[0027] (Face registration 2) Figure 6 is a sequence diagram showing a modified version of the pre-check process for face image P. In the example in Figure 5, the management server 10 checked the quality of face image P using its own pre-check program 16, but the quality check of face image P can also be performed using any of the face recognition engines 20. In the example in Figure 6(a), the engine that is most strict about the quality of face image P among the face recognition engines 20 is used as the pre-check engine 20'. In this case, the management server 10 transfers the face image P received from the user application 30 to the pre-check engine 20' and receives the check result from the pre-check engine 20'. The check result may be information indicating whether the pre-check engine 20' succeeded in generating the face template T, or information indicating the data quality of face image P. The process after face image P passes the quality check is the same as in the example in Figure 5(a). If the pre-check engine 20' evaluates the data quality of face image P as insufficient, it requests the user to retake a photo of their face (not shown).

[0028] (Face registration 3) Figure 6(b) shows an example where the face image P is directly distributed to each face recognition engine 20 without performing a pre-check, and the face template T is created. In this example, the management server 10 forwards the face image P received from the user application 30 directly to each face recognition engine 20 and receives the result of generating the face template T from each face recognition engine 20. The process after all face recognition engines 20 have successfully generated the face template T is the same as in the example in Figure 5(a). If any of the face recognition engines 20 fails to generate the face template T due to the data quality of the face image P, the user is requested to retake a photo of their face (not shown).

[0029] The data quality check performed by the pre-check program 16 in this configuration differs from the actual face template T generation algorithm executed in each face recognition engine 20. While setting the acceptance standard by the pre-check program 16 as strict as possible can prevent failures in the subsequent face template T generation process, this cannot be guaranteed. As shown in the example in Figure 6(b), it is possible to reliably prevent discrepancies between the evaluation result of the face image P and the acceptance or rejection by each face recognition engine 20. As will be explained in more detail later, in this configuration of face recognition system F, after the face image P is registered with the management server 10, the user selects the service they will use, and the face image P is transferred to the face recognition engine 20 used by the selected service. In other words, when the user first takes a picture of their face, the service (face recognition engine 20) the user will use is unknown. Therefore, the example in Figure 6(b) can be used when the service the user will use is known in advance, when the number of services the user can use is small, or when the user retakes the face image P after registering the service.

[0030] <Service registration function> Figure 7 is a schematic diagram illustrating how a user selects services. As shown in Figure 7(a), the user application 30 is equipped with a service addition button 34 (service selection means) for the user to register the services they will use. In this configuration, the management server 10 distributes the user's face image P only to the face recognition engine 20 used for the services that the user has registered. Distributing the face image P to the face recognition engine 20 of services that the user does not use would not only waste resources but also increase unnecessary security risks.

[0031] When the user taps the service add button 34 in the user app 30, a list of services that the user can add is displayed. For example, if the user selects service A (341), an input component 342 is displayed where the user can enter service-specific information, such as membership number or the type of tickets they possess. The input component 342 is not limited to a text box; it may also be a dropdown list, radio button, checkbox, etc. The data entered here is saved in the user database 11 as user information 111. Depending on the selected service, the input component 342 may not be displayed. In other words, the user database 11 contains information that is referenced when providing a specific service, but not when providing other services. As will be explained in more detail later, some services that use facial recognition are provided to the general public or the entire nation, while others are provided only to subscribers or certain related parties. If the service is directed to a specific individual, an input component 342 may be displayed where the user can enter a passcode or other information that has been notified in advance by the service provider.

[0032] When a user enters the required information and taps the registration button 343, the management server 10 distributes the user's face image P, which it had stored, to the face recognition engine 20 of service A. The face recognition engine 20, having received the face image P, generates and registers a face template T from that face image P. This prepares the user to use the face recognition of service A.

[0033] If service A requires approval from the service provider, then the application button 344 will be displayed instead of the registration button 343. When the user enters the necessary information and taps the application button 344, the management server 10 sends an approval request to the service provider.

[0034] Alternatively, if Service A is a service that has an information processing system that uses its own user accounts, the sign-in form 345 for Service A may be displayed in order to associate the face template T of the face recognition system F with the user account of Service A. Or, the user may be redirected to the sign-in screen of Service A. By signing in from this sign-in form 345, the identification information of each is associated. In this case, the face recognition system F may have the identification information of the user account of Service A, or Service A may have the user ID 113 of the face recognition system F. When face recognition is successful by the face recognition engine 20 of Service A, the user ID 113 or information that can uniquely identify the user account of Service A is sent to the system of Service A.

[0035] Furthermore, the OAuth framework can also be used for this inter-service communication process. Since the OAuth protocol specification is publicly known, a detailed explanation will be omitted here, but the general flow is as follows. In the following example, the OAuth access token is information that uniquely identifies the user account of service A. (1) When a user of the facial recognition system F selects service A in the user app 30, the user is redirected to the authorization server (sign-in screen) for service A. (2) The user signs in to Service A at the redirect destination and authorizes the authorization server to link Service A with facial recognition system F. (3) As a result, the authorization server issues an authorization code to the facial recognition system F, and the facial recognition system F requests an access token from the authorization server along with the issued authorization code. (4) An access token is issued from the authorization server to the facial recognition system F. (5) When a user successfully undergoes facial recognition by the facial recognition engine 20 at the service provision site of Service A, a message containing an access token is sent from the facial recognition system F or the facial recognition engine 20 to the Service A system. (6) This allows Service A to identify which user's facial recognition on Service A's system was successful.

[0036] Note that this linking operation may also be performed on the service A side rather than the user application 30. Figure 7(b) shows the facial recognition usage settings screen 90 on the service A side. By signing in here with the user account of the facial recognition system F, the identification information of each is associated, and service A is registered on the user application 30.

[0037] (Service registration 1) Figure 8 is a sequence diagram showing the process by which a user registers and unregisters services in the facial recognition system F. When a user registers a service they will use, the user taps the service addition button 34 on the user app 30 and selects the service they will use from the displayed list of services. This selection includes inputting service-specific requirements. Once a service is selected by the user, the management server 10 distributes the user's facial image P to the facial recognition engine 20 used for that service. The facial recognition engine 20 generates a facial template T from the facial image P distributed by the management server 10 and registers it in the facial database 25. Upon receiving a success code for the facial template T from the facial recognition engine 20, the management server 10 notifies the user app 30 that the service registration is complete. On the other hand, if the facial recognition engine 20 fails to generate the facial template T due to the data quality of the facial image P, the management server 10 requests the user to retake a facial image (not shown).

[0038] (Service cancellation) When a user wants to unregister for a service, the user taps the service addition button 34 in the user app 30 and then taps the service they are registered for from the displayed list of services. This displays a message asking the user to confirm whether they want to unregister for that service. The unregistration process begins when the user responds to this message indicating that they wish to continue unregistering. Once the user has unregistered for a service, the management server 10 sends an instruction to the facial recognition engine 20 used for that service to delete the user's facial template T. The facial recognition engine 20 deletes the user's facial template T according to the instructions from the management server 10. Here, the facial recognition engine 20 may choose not to delete the facial template T and instead disable it so that it cannot be used. Upon receiving a success code for the deletion process from the facial recognition engine 20, the management server 10 notifies the user app 30 that the service unregistration is complete.

[0039] (Service registration 2) Figure 9(a) is a sequence diagram showing the registration process for a service that requires the approval of a service provider. When registering a service that requires the approval of a service provider, the user first taps the service addition button 34 on the user app 30, selects the service from the displayed list of services, enters the service-specific requirements, and then taps the application button 344. When the application button 344 is tapped, the management server 10 sends an approval request to the service provider (service provider tool 39). The management server 10 then waits for the service provider's approval operation and distributes the user's face image P to the face recognition engine 20 used for the service. Here, the service provider's approval operation may not only approve the application but also set service-specific information for the user. For example, if the service is a locking / unlocking service for various facilities in an apartment building, this may include the user's room number. The data set by the service provider is registered in the user database 11 as user information 111. The subsequent process is the same as service registration 1 described above, but the registration completion notification is sent not only to the user but also to the service provider.

[0040] (Service registration 3) Figure 9(b) is a sequence diagram showing the registration process for a service that requires prior invitation from a service provider. When registering a service that requires prior invitation, the service provider first sends an invitation to a specific user from the service provider tool 39. The user receives the invitation and selects the service. The subsequent process is the same as service registration 2 described above. The registration invitation sent to the user may include a link or passcode for selecting the service. The passcode here refers to a sequence of numbers or characters, as well as one-dimensional or two-dimensional codes such as barcodes read by a scanner. Alternatively, the service provider's invitation operation may correspond to a prior approval operation for the user, and the registration invitation may simply be a notification of that approval.

[0041] <Facial Recognition Process> Figure 10 is a sequence diagram showing the facial recognition process in each facial recognition engine 20. The following explanation will describe each pattern of the facial recognition process with reference to Figure 10.

[0042] When a user captures their face with the camera 21 of the facial recognition engine 20 at the service provision site, the authentication program 26 compares that face with the facial template T. If authentication is successful, the intended request is accepted. On the other hand, if facial recognition fails, a message prompting the user to check whether the service is registered is displayed on the screen 22. As mentioned above, in this form of facial recognition system F, users need to register the services that will use facial recognition in advance. To sufficiently reduce the risk of false positives (false acceptance rate) in facial recognition, it is effective to break down the service into as many small components as possible. However, if the service is broken down into extremely small components, such as when users are asked to select stores from the same chain, the actual registration scope may be narrower than the user perceives. In such cases, having the user check the service registration status allows for the quick identification of the cause of authentication failure. Note that the method of prompting the user to check is not limited to displaying a message on the screen 22; for example, an audio message to that effect may also be emitted.

[0043] If the service that successfully performs facial recognition involves a user payment process, the facial recognition engine 20 further requests the user to enter a PIN code 112. The user enters the PIN code using the keypad displayed on screen 22, and if it matches the registered PIN code 112, the payment process is performed. If the entered PIN code does not match, the user is requested to re-enter it multiple times, and if it still does not match, the payment process fails. Facial recognition is biometric authentication and can be said to have a higher authentication strength compared to other authentication methods, but as mentioned above, the probability of a false positive is not zero. In payment processing, where attempts at misuse are naturally expected, the security of payments can be further enhanced by making facial recognition part of multi-factor authentication.

[0044] <Other Embodiments> (Service Selection 4) Figure 11 is a sequence diagram showing another embodiment of the facial recognition system of the present invention. In the above embodiment, a configuration was mainly described in which the management server 10 is equipped with a pre-check program 16 and a single facial image P is distributed to multiple facial recognition engines 20. On the other hand, the pre-check program 16 and the commonization of facial images P are not essential elements of the facial recognition system of the present invention and can be omitted. In this case, for example, as shown in Figure 11, a facial image will be required each time a service is selected. If the number of facial recognition engines 20 used is small, or if the specifications of the facial images P required by them differ significantly, such a configuration may be considered. Alternatively, the mechanism of the above embodiment may be used for facial recognition engines 20 with similar facial image P requirements, and the mechanism of Figure 11 may be used for services that require special facial images P.

[0045] Although embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and various modifications can be made without departing from the spirit of the invention. [Explanation of Symbols]

[0046] F: Face recognition system, P: Face image, T: Face template, 10: Management server, 11: User database, 111: User information, 112: PIN code, 113: User ID, 12: Face recognition engine database, 16: Pre-check program (pre-check method), 17: Distribution program, 18: Payment program, 20: Face recognition engine, 20': Pre-check engine, 21: Camera (shooting device), 22: Screen, 25: Face database, 26: Authentication program (authentication method), 30: User app (user terminal), 31: Camera, 32: Registration form, 32: Face registration program, 34: Add service button (service selection method), 341: Service A, 342: Input component, 343: Registration button, 344: Application button, 345: Sign-in form, 39: Service provider tool, 90: Face recognition usage settings screen 90, 91: Sign-in form, 92: Start integration button

Claims

1. Facial recognition engine and A user terminal that acquires a scan data of the user's face or its three-dimensional shape (hereinafter referred to as "face image, etc."), The provider tools used by the providers of each service that uses facial recognition, Equipped with, Among the aforementioned services, The provider of that service needs to perform an operation (hereinafter referred to as "permission operation") to allow a specific user to use it using the provider tool, This includes those that can be used without requiring the aforementioned permission operation. Facial recognition system.

2. Furthermore, it includes a management server that holds information about the user's payment method, In services involving payment by the aforementioned payment method, in addition to facial recognition by the aforementioned facial recognition engine, additional authentication by other authentication methods is required. The facial recognition system according to claim 1.

3. The system further includes a management server that holds a user ID, which is information that can uniquely identify a user. Among the services provided using the aforementioned facial recognition engine are independent services that have an information processing system using their own user accounts. When facial recognition by the facial recognition engine is successful in the independent service, the user ID or information that uniquely identifies the user account of the independent service is transmitted to the information processing system of the independent service. The facial recognition system according to claim 1.

4. The management server holds information about each user, including personal information, payment method information, ticket information, or information set by the provider of each service provided using the facial recognition engine for that user (hereinafter, these pieces of information are collectively referred to as "user information"). The user information includes information that is referenced when providing a specific service but not when providing other services. The facial recognition system according to claim 1.

Images