Management device, management method, and management program

Abstract

To provide a management apparatus, a management method, and a management program, configured to flexibly use results of collation made by face authentication engines.SOLUTION: A management apparatus according to the present disclosure includes: an acquisition unit which acquires, from a user, a face image on which a plurality of face authentication engines execute authentication processing; a transmission unit which transmits the face image acquired by the acquisition unit to each of the face authentication engines; and a result determination unit which determines whether to permit a request from the user on the basis of authentication results of the face authentication engines that have executed authentication on the face image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a management device, a management method, and a management program for managing authentication processing.

Background Art

[0002] As a means for performing personal identification, face authentication processing is known. In face authentication processing, a face image of a user is registered in advance, and personal identification is performed by comparing the face image acquired at the time of authentication with the registered image.

[0003] The prior registration of a face image in face authentication processing is performed, for example, by the user himself / herself using a terminal such as a smartphone. As a technique related to the registration of a face photo, a technique for supporting efficient registration by determining the suitability of a face image and then processing the face photo according to the medium to be registered is known (for example, Patent Document 1). Also, in face authentication processing, a technique for improving authentication accuracy by using a plurality of face authentication engines is known (for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to the technique of Patent Document 2 above, authentication accuracy can be improved by using a plurality of different face authentication engines.

[0006] However, with the technology described in Patent Document 2, if authentication fails with any of the registered facial recognition engines, an error may occur and the authentication process may stop. In other words, with the technology described in Patent Document 2, it is difficult to perform flexible authentication processing that takes advantage of the strengths of each facial recognition engine.

[0007] Therefore, this disclosure proposes a management device, a management method, and a management program that can flexibly utilize the matching results obtained by a facial recognition engine. [Means for solving the problem]

[0008] To solve the above problems, one form of management device according to this disclosure includes: an acquisition unit that acquires a facial image from a user for multiple facial recognition engines to perform authentication processing; a transmission unit that transmits the facial image acquired by the acquisition unit to each of the multiple facial recognition engines; and a result determination unit that determines whether or not to permit the user's request based on the authentication result of each of the multiple facial recognition engines that authenticated the facial image. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram schematically shows the flow of the registration process according to the embodiment. [Figure 2] This diagram schematically illustrates the flow of the facial recognition process according to the embodiment. [Figure 3] Figure (1) illustrates the registration process according to the embodiment. [Figure 4] Figure (2) illustrates the registration process according to the embodiment. [Figure 5] This is a sequence diagram showing the procedure for the registration process according to the embodiment. [Figure 6] This is a sequence diagram showing the procedure for facial recognition processing according to the embodiment. [Figure 7] This is a flowchart showing the determination procedure for facial recognition processing according to the embodiment. [Figure 8] This figure shows an example configuration of a management device according to an embodiment. [Figure 9] It is a diagram showing an example of a user information storage unit according to an embodiment. [Figure 10] It is a diagram showing an example of a quality check item storage unit according to an embodiment. [Figure 11] It is a diagram showing an example of a face photo storage unit according to an embodiment. [Figure 12] It is a diagram showing an example of an external service information storage unit according to an embodiment. [Figure 13] It is a hardware configuration diagram showing an example of a computer that realizes the functions of the management device. **[Embodiments for Carrying Out the Invention]**

[0010] Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. In the following embodiments, the same parts are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] (1. Embodiment) (1-1. An Example of Management Processing According to the Embodiment) FIG. 1 and FIG. 2 schematically show the flow of management processing according to the embodiment. The management processing according to the embodiment is executed by the management device 100 included in the management system 1 shown in FIG. 1. As shown in FIG. 1, the management system 1 includes a management device 100, a user terminal 20, and a face authentication provider 50.

[0012] The management device 100 is an information processing device that executes management processing according to the embodiment. For example, the management device 100 is a cloud server or the like that can communicate with the user terminal 20 and the face authentication provider 50 via a network.

[0013] The user terminal 20 is an information processing terminal used by the user 10. For example, the user terminal 20 is a smartphone, a tablet terminal, a PC (Personal Computer), or the like. The user terminal 20 according to the embodiment is provided with a camera and can capture a face image of the user 10.

[0014] The face authentication provider 50 generally refers to an operator that provides a face authentication engine for executing face authentication processing. The face authentication provider 50 includes, for example, a first provider that provides a first face authentication engine 60, a second provider that provides a second face authentication engine 70, a third provider that provides a third face authentication engine 80, and the like.

[0015] The first provider, the second provider, and the third provider are operators that provide different face authentication engines. The face authentication engines provided by each provider have different features in various aspects, such as the detection of the face range in an image, the extraction of face feature amounts, and the method of matching feature amounts. That is, the face authentication engines have different features in the features of the images they are good at (features such as the brightness of the image, the ratio of the range occupied by the face in the image, the skin color of the user to be authenticated, and the depth of engraving). In the following description, when there is no need to distinguish the first provider, the second provider, and the third provider, they may be collectively referred to as the face authentication provider 50.

[0016] The management device 100 according to the embodiment enhances the authentication accuracy and improves the reliability of authentication by performing registration and collation with these multiple face authentication engines when the user 10 registers a face image or performs authentication. That is, the management device 100 serves as a platform for executing authentication processing using a multi-face authentication engine.

[0017] Specifically, for the execution of authentication processing by a plurality of face authentication engines, the management device 100 receives a face image that is the source of correct data used for collation by each of the plurality of face authentication engines from the user 10. In addition, the management device 100 provides processing means for the user 10 to process the received face image. Then, the management device 100 determines whether the processed image, which is the processed face image, has a quality that can be used as correct data for the authentication processing of the plurality of face authentication engines.

[0018] In this way, the management device 100 prevents user 10 from having to redo the process by determining whether the processed image created by user 10 can be used by each facial recognition engine. Furthermore, the management device 100 may verify the quality of the facial image for each of the multiple facial recognition engines and register the facial image only if it has been verified to be of usable quality for all of the facial recognition engines used for authentication. This allows the management device 100 to perform highly accurate authentication using multiple facial recognition engines while smoothly managing the often cumbersome process of registering facial images.

[0019] Regarding this management process, we will first explain the process of registering facial images for multiple facial recognition engines in a step-by-step manner using Figure 1. Figure 1 is a schematic diagram showing the flow of the registration process according to the embodiment.

[0020] In the example shown in Figure 1, user 10 accesses the management device 100 using user terminal 20 and requests registration of a facial image to be used for facial recognition (step S1). For example, user 10 requests registration of a facial image via a website provided by the management device 100.

[0021] When the management device 100 receives a request, it provides the user terminal 20 with a web page or similar for registering a facial image. The user 10 takes a picture of themselves using the camera on the user terminal 20 and sends the facial image to be registered to the management device 100. Details of facial image registration using the camera will be described later with reference to Figure 3.

[0022] The management device 100 stores the facial image acquired from the user 10 as facial image data 30 in its storage unit. At this time, the management device 100 may determine whether the facial image meets the quality requirements of each facial recognition engine, and only if it determines that the quality is met may it store the facial image as facial image data 30. If the management device 100 determines that the facial image does not meet the quality requirements, it may request the user 10 to retake the photo or make adjustments by processing the facial image.

[0023] If the facial image meets the quality requirements of each facial recognition engine and is eligible for registration, the management device 100 notifies the user 10 accordingly (step S2).

[0024] When the management device 100 acquires facial image data 30, it registers the facial image data 30 with each facial recognition engine. For example, the management device 100 sends the facial image data 30 to the first provider having the first facial recognition engine 60 and requests registration to make the first facial recognition engine 60 available for user 10's facial recognition (step S3).

[0025] The first provider extracts features from the acquired face photograph data 30 through a feature extraction process performed by the first face recognition engine 60 and stores them in the feature memory unit 62. For example, the first face recognition engine 60 extracts the facial features of user 10 as parameters by hashing the face images contained in the face photograph data 30, and stores the extracted values ​​as features in the feature memory unit 62. After this, the first provider may discard the face photograph data 30 itself. As a result, the first provider can reduce the resources and effort involved in data management by retaining only the features without retaining personal information such as the face photograph data 30.

[0026] Similarly, the management device 100 transmits the facial image data 30 to the second provider and requests registration to make the second facial recognition engine 70 available for user 10 facial recognition (step S4). The second provider, like the first provider, extracts features from the acquired facial image data 30 and stores them in the feature storage unit 72.

[0027] Similarly, the management device 100 transmits the facial image data 30 to the third provider and requests registration to make the third facial recognition engine 80 available for user 10 facial recognition (step S5). The third provider, like the first and second providers, extracts features from the acquired facial image data 30 and stores them in the feature storage unit 82.

[0028] The management device 100 registers the facial image data 30 of user 10 for each facial recognition engine to be used. Once registration is complete, the management device 100 links the user 10, the registered facial image data 30, the registered facial recognition engine, and other information and stores it in the memory unit.

[0029] Next, using Figure 2, we will explain the flow of verifying the identity of user 10 using facial recognition (matching process). Figure 2 is a schematic diagram showing the flow of the facial recognition process according to the embodiment. The management system 1 shown in Figure 2 further includes an authentication terminal 40 and a service provider 200.

[0030] The authentication terminal 40 is an information processing terminal installed in locations where identity verification is required. The authentication terminal 40 can communicate with the management device 100 via a network, and for example, it transmits images captured by its own camera to the management device 100. The authentication terminal 40 then obtains the authentication result from the management device 100 and notifies the person who requested authentication whether their identity has been verified (whether it has been matched with the registered facial image). The authentication terminal 40 also determines whether or not to provide a predetermined service based on the matching result.

[0031] Service provider 200 is a business that provides various services. For example, service provider 200 manages authentication terminals 40 and provides various services to users whose identity has been verified. In this disclosure, "service" is not limited to commercial services, but is a general term for any mechanism or system that user 10 intends to use, such as factory access control gates. For example, services include permission to enter and exit specific facilities, attendance management, and settlement of financial products. Service provider 200 has user information 212, which is information about users who use the service, and business data 214, which stores the type and content of the service.

[0032] In the example shown in Figure 2, when user 10 attempts to use a service provided by service provider 200, he or she takes a picture of himself or she attempts facial recognition with the authentication terminal 40. For example, when user 10 enters a facility where entry is controlled, he or she requests authentication from the authentication terminal 40 to verify his or her identity.

[0033] When the authentication terminal 40 photographs the user 10, it transmits the authentication face image obtained from the photograph to the management device 100 (step S11). The management device 100 transmits the acquired authentication face image to each face authentication provider.

[0034] For example, the management device 100 transmits an authentication face image to the first provider and causes the first face recognition engine 60 to perform face recognition based on the authentication face image (step S12). Specifically, the first face recognition engine 60 compares the features of the registered user 10 with the features extracted from the authentication face image. The first face recognition engine 60 then transmits a result (matching result) showing the comparison result between the features of user 10 and the features extracted from the authentication face image to the management device 100. The management device 100 determines that the matching between the pre-registered user 10 and the person associated with the authentication face image (user 10 in this example) was successful if the similarity between the features obtained from the first face recognition engine 60 is higher than a predetermined value. On the other hand, if the similarity is lower than a predetermined value, the management device 100 determines that the matching between the pre-registered user 10 and the person associated with the authentication face image could not be performed and the identity of the person could not be confirmed.

[0035] When the facial recognition performed by the first facial recognition engine 60 is completed, the management device 100 stores the matching result in the authentication log 32. The first facial recognition engine 60 may perform the matching after recognizing the person to be matched as user 10 using an ID card or the like, or it may perform the matching with an unspecified number of users that have been registered in advance and compare them with the authentication facial image. In the latter case, it is expected that the first facial recognition engine 60 will extract user 10 as the match target if the similarity is determined to be higher than that of other targets as a result of the matching.

[0036] Next, the management device 100 transmits the authentication face image to the second provider and causes the second face authentication engine 70 to perform face authentication based on the authentication face image (step S13). When the second face authentication engine 70 completes face authentication, it transmits the matching result to the management device 100. The management device 100 determines whether the matching of user 10 was successful based on the similarity between the feature quantities obtained as a result of the matching by the second face authentication engine 70, and stores the result in the authentication log 32. Similarly, the management device 100 transmits the authentication face image to the third provider and causes the third face authentication engine 80 to perform face authentication based on the authentication face image (step S14). When the third face authentication engine 80 completes face authentication, it transmits the matching result to the management device 100. The management device 100 determines whether the matching of user 10 was successful based on the similarity between the feature quantities obtained as a result of the matching by the third face authentication engine 80, and stores the result in the authentication log 32.

[0037] Then, the management device 100 transmits the authentication result for user 10 to the authentication terminal 40 based on the matching results from multiple facial recognition engines (step S15). In other words, the management device 100 transmits to the authentication terminal 40 the result of determining whether or not it is permissible to provide user 10 with the service requested by user 10.

[0038] At this time, the management device 100 may refer to the external service partnership information 34 and determine the authentication result for user 10. For example, the external service partnership information 34 may include the level of security required by the external service. That is, the management device 100 compares the level of security required by the service provider 200 with the matching results from multiple facial recognition engines and then transmits the authentication result for user 10 to the authentication terminal 40 (step S15).

[0039] In terms of service operation, various levels of security can be set for a service. For example, if the authentication process according to the embodiment is used for daily attendance management, if the required authentication strength is too high and authentication errors (situations where it is difficult to verify the identity of the user attempting authentication) occur frequently, the usability may be low even if the security is high. On the other hand, if the authentication process according to the embodiment is used for payment processing at a financial institution, it is desirable to set the authentication strength to an extremely high level to prevent fraudulent use.

[0040] Therefore, the management device 100 manages information such as how much authentication strength is required for each service. For example, if a service requires low authentication strength, the management device 100 may allow the user 10 to access the service if at least one of the multiple facial recognition engines successfully authenticates the user. Alternatively, if a service requires high authentication strength, the management device 100 may not allow the user 10 to access the service unless all of the multiple facial recognition engines successfully authenticate the user.

[0041] The management device 100 obtains information such as authentication strength from the service provider 200. The management device 100 then sends a determination result to the service provider 200 indicating whether the service is available or not, based on the matching result between the authentication facial image and the user 10 (step S16). Based on the determination result from the management device 100, the service provider 200 provides the predetermined service to the user 10 whose identity has been confirmed (step S17). For example, the service provider 200 notifies the user 10 via the authentication terminal 40 that the user 10's identity has been confirmed, and activates the gate opening / closing device to allow the user 10 to enter. Note that the provision of such services may also be performed by the management device 100, which has been entrusted by the service provider 200.

[0042] As shown in Figure 2, the management device 100 determines whether to permit user 10's request based on the authentication results of each of the multiple facial recognition engines that authenticated the facial image. In other words, by using multiple facial recognition engines, the management device 100 can improve the accuracy of identity authentication and make flexible settings according to the authentication strength for service provision involving authentication. In the above example, the management device 100 obtained the similarity determined by each facial recognition engine and determined whether user 10's authentication was successful by determining the obtained similarity. However, the success or failure of facial recognition may be performed by each facial recognition engine instead of the management device 100. In this case, each facial recognition engine maintains a similarity threshold value. Each facial recognition engine then compares the registered features of user 10 with the features extracted from the authentication facial image, and if the similarity between the compared features is higher than a predetermined value, it determines that user 10's matching was successful and transmits the determination result to the management device 100. In other words, the entity that performs the success or failure of facial recognition may be either the management device 100 or each facial recognition engine.

[0043] Next, the registration of facial images according to the embodiment will be described in detail using Figures 3 and 4. Figure 3 is (1) for illustrating the registration process according to the embodiment.

[0044] The management device 100 provides a predetermined registration page to a user 10 who wishes to register a facial image and accepts the registration request. The screen 300 shown in Figure 3 shows a dedicated page for facial image registration displayed on the user terminal 20.

[0045] On screen 300, user 10 sets an ID and password for registration and proceeds with the facial image registration process. Once user 10 enters the ID and password, screen 300 transitions to screen 302 (step S21).

[0046] Screen 302 includes a preview image 304 to be captured by the camera installed in the user terminal 20. The management device 100 displays a guide 306 in the preview image 304. The guide 306 indicates the desirable area for the face to occupy in the captured image. Since the face recognition process includes a step in which the face recognition engine detects a face from the image, it is desirable that all parts that make up the face are included in the image. For this reason, the management device 100 displays a guide 306, such as an estimate of the area occupied by the face and lines that serve as a guide for the user 10 to bring their face to the center of the image, prompting the user 10 to take an appropriate picture.

[0047] User 10 takes a picture of themselves using, for example, a camera located on the inside (screen side) of the user terminal 20, adjusts their position while viewing the preview image 304 so that their face fits within the guide 306, and then presses the capture button 308. Once User 10 has taken a picture in accordance with this operation, the management device 100 can acquire the face image for registration via the user terminal 20.

[0048] Once shooting is complete, screen 302 transitions to screen 310 (step S22). At this stage, the management device 100 may provide the user 10 with processing means for processing the captured image. Specifically, the management device 100 may provide a trimming tool or the like for removing unnecessary parts of the captured image.

[0049] For example, if user 10 uses the captured image as an identification photo with a specified size, the user can crop the image to the specified size. The cropping tool also allows user 10 to move the captured image. Therefore, if user 10's face is not included within the circle of guide 306, user 10 can shift the entire image to adjust it so that their face is included within the circle. Furthermore, user 10 may enlarge or reduce the image using operations such as pinch-in and pinch-out to adjust it so that their face is appropriately sized within the circle of guide 306. The management device 100 may also perform the cropping process automatically. For example, the management device 100 detects user 10's face through face detection processing and automatically adjusts the position of the face so that the detected face's coordinates fall within the circle of guide 306. The management device 100 also automatically adjusts the size of the detected face so that it fits within the circle of guide 306. Additionally, the management device 100 may remove any extraneous parts (such as the background) other than the detected face. In this way, the management device 100 automatically performs the cropping, so that user 10 can automatically obtain an image suitable for registration simply by taking an image that includes a face. This allows the management device 100 to reduce the burden on user 10 in terms of face registration.

[0050] Furthermore, the management device 100 may provide other processing means. This will be explained with reference to Figure 4. Figure 4 is a diagram (2) illustrating the registration process according to the embodiment.

[0051] Once user 10 has finished cropping, the management device 100 transitions from screen 310 in Figure 3 to screen 320 in Figure 4. On screen 320, user 10 can adjust the overall brightness of the image. For example, user 10 can operate the brightness adjustment bar 322 to adjust the overall brightness to be brighter or darker. Alternatively, if user 10 presses the automatic brightness adjustment button 324, the management device 100 may refer to the current brightness of the image and the brightness reference value required by the face recognition engine, etc., and automatically adjust the brightness to a level that can be registered with the face recognition engine. This allows user 10 to adjust the image to a registerable brightness without complicated procedures.

[0052] Once user 10 has finished adjusting the brightness, the management device 100 transitions screen 320 to screen 326 (step S31). On screen 326, user 10 can select the background of the image. For example, if user 10 wants to remove the background because it is included in the captured image, they can press the background selection button 330 to change the captured image to a processed image 328 with the background changed to a solid white color, etc. Alternatively, if user 10 is going to use the captured image as an identification photo with a specified background, they can change the background to the specified background (such as a solid light blue color).

[0053] Once user 10 has finished adjusting the background, the control device 100 transitions screen 326 to screen 332 (step S32). At this stage, the control device 100 determines the quality of the processed image, which is the captured image after processing is complete.

[0054] As will be explained in detail later, the management device 100 determines for each face recognition engine whether the processed image meets the criteria required by that engine for the registered image. When the management device 100 determines that the processed image meets the registration requirements of all face recognition engines, it displays a determination result 336 indicating that the criteria have been met. After confirming the determination result 336, if the user 10 wishes to register this processed image, they press the save button 334. When the user 10 presses the save button 334, the management device 100 saves this processed image in the storage unit as a registration image to be used for registration.

[0055] If the processed image does not meet the criteria, the management device 100 transitions from screen 326 to screen 338. On screen 338, the management device 100 displays a judgment result 340 indicating that the processed image does not meet the criteria for registration requests of any of the face recognition engines and therefore cannot be registered. As shown in Figure 4, the management device 100 may also display the details of the criteria that the processed image failed to meet based on the quality judgment result in the judgment result 340. This allows the user 10 to obtain information on what points to pay attention to when creating a new processed image if they need to redo the shooting or processing.

[0056] When the management device 100 saves the processed image to the storage unit as a registration image to be used for registration, it transitions from screen 332 to screen 342, displays the face image registration completion display 344, and notifies the user 10 that registration is complete (step S33).

[0057] In this way, the management device 100 determines the quality of the processed image to determine whether or not it can be registered by the face recognition engine. This prevents situations that would inconvenience the user 10, such as when the image is cropped after the quality determination and the determination result changes. Furthermore, with the management device 100, processing such as cropping and quality determination can be performed together on the server-side platform, enabling the acquisition of high-quality face images, which leads to improved accuracy in subsequent feature extraction.

[0058] (1-2. Procedure for management processing according to the embodiment) Next, the procedure of the management process according to the embodiment will be explained using Figures 5 to 7. First, the flow of the registration process will be explained using Figure 5. Figure 5 is a sequence diagram showing the procedure of the registration process according to the embodiment.

[0059] As shown in Figure 5, User 10 applies to register a facial image via a website or the like provided by the management device 100 (Step S40). Subsequently, User 10 takes a photograph of the facial image to be registered. If User 10 has an image taken in advance, they may select that image to be used for registration (Step S41).

[0060] When user 10 takes a picture or selects an image, the face image is transmitted to the management device 100 (step S42). The management device 100 receives the face image transmitted from user 10 (step S43).

[0061] Subsequently, user 10 performs a series of processing operations on the face image using processing means provided by the management device 100. For example, user 10 performs a trimming operation to cut out the face portion (step S44). The management device 100 performs the trimming operation on the face image according to the user's operation (step S45).

[0062] Furthermore, user 10 adjusts the brightness of the face image using processing means provided by the management device 100 (step S46). The management device 100 adjusts the brightness of the face image, such as making it brighter or darker, according to the user's operation (step S47).

[0063] Furthermore, user 10 changes the background of the face image using processing means provided by the management device 100 (step S48). The management device 100 changes the background of the face image according to the user's operation (step S49).

[0064] When user 10 has finished processing the face image and generated the processed image, user 10 sends a quality check request to the management device 100 for the processed image (step S50). When the management device 100 receives the quality check request from user 10, it sends the processed image to the face recognition provider 50 (step S51).

[0065] The facial recognition provider 50 performs a quality check in its facial recognition engine to determine whether the processed image can be used for authentication (step S52). This process is performed for each provider's facial recognition engine. The facial recognition provider 50 transmits the results of the quality check to the management device 100 (step S53).

[0066] The management device 100 refers to the quality check results from multiple facial recognition engines and determines whether there are any problems with all the results (step S54). If it determines that there is a quality problem in any of the facial recognition engines (step S54; No), the management device 100 notifies the user 10 that registration is NG and the reason for it, as shown in screen 338 of Figure 4 (step S55).

[0067] On the other hand, if all facial recognition engines determine that there are no quality issues (step S54; Yes), the management device 100 notifies the user that the registration determination is OK, as shown in screen 332 of Figure 4 (step S56).

[0068] After confirming the notified result, user 10 requests the management device 100 to register the authentication face image, for example by pressing the save button 334 displayed on screen 332 (step S57).

[0069] When the management device 100 receives a registration request from the user 10, it registers the quality-determined face image as an authentication face image (step S58). The management device 100 then sends the registered authentication face image to each face recognition provider 50 (step S59). The face recognition provider 50 loads the authentication face image into the face recognition engine, extracts features, and then associates the features with the user ID and registers them (step S60). After that, the face recognition provider 50 may discard the raw data of the acquired registration face image.

[0070] Next, the procedure of the authentication process according to the embodiment will be described using Figure 6. Figure 6 is a sequence diagram showing the procedure of the facial recognition process according to the embodiment.

[0071] As shown in Figure 6, the authentication terminal 40 photographs the user 10 requesting facial authentication and acquires a facial image (step S70). The authentication terminal 40 transmits the acquired facial image to the management device 100 (step S71).

[0072] When the management device 100 acquires a face image, it transmits the acquired face image to the first face recognition engine 60 (step S72). The first face recognition engine 60 compares the acquired face image with the registered face image and performs face recognition processing (step S73). The management device 100 acquires the comparison result (first comparison result) from the first face recognition engine 60 (step S74).

[0073] Next, the management device 100 transmits the face image acquired in step S72 to the second face recognition engine 70 (step S75). The second face recognition engine 70 compares the acquired face image with the registered face image and performs face recognition processing (step S76). The management device 100 obtains the comparison result (second comparison result) from the second face recognition engine 70 (step S77).

[0074] The management device 100 similarly transmits the facial images to all facial recognition engines in which the authentication facial images are registered and obtains all matching results (step S80). That is, the management device 100 similarly transmits the facial images to the third facial recognition engine 80 (not shown in the figure) and other registered facial recognition engines and obtains matching results.

[0075] Then, the management device 100 determines the matching result (step S81). As described above, the success or failure of authentication by each facial recognition engine is determined based on whether the similarity between the feature quantities transmitted from the facial recognition engines exceeds a predetermined similarity. Details of the determination process that determines whether or not to allow the user 10 to use the service based on multiple authentication determinations will be described later with reference to Figure 7. After such determination, the management device 100 transmits the determination result to the authentication terminal 40 (step S82). The authentication terminal 40 displays the result transmitted from the management device 100 on a screen or the like (step S83).

[0076] Next, the determination procedure of the facial recognition process according to the embodiment will be explained using Figure 7. Figure 7 is a flowchart of the determination procedure of the facial recognition process according to the embodiment.

[0077] As shown in Figure 7, the management device 100 identifies the service that the user intends to use (step S101). For example, the management device 100 identifies the service that the user intends to use based on external service partnership information linked to the authentication terminal 40 and information from the service provider.

[0078] The management device 100 then refers to the criteria set for the service that the user intends to use. For example, based on the authentication strength of the service, the management device 100 refers to criteria such as not allowing service use unless the matching result is OK in all facial recognition engines, or allowing service use if the matching result is OK in any one of the facial recognition engines (step S102).

[0079] The management device 100 then determines whether the matching results obtained from the multiple facial recognition engines meet the service criteria (step S103). If the service criteria are met (step S103; Yes), the management device 100 determines that the facial recognition was successful (step S104). In other words, the management device 100 determines that it will allow the authenticated user to use the service. On the other hand, if the service criteria are not met (step S103; No), the management device 100 determines that the facial recognition was unsuccessful (step S105). In other words, the management device 100 determines that it will not allow the authenticated user to use the service.

[0080] (1-3. Configuration of the control device according to the embodiment) Next, the configuration of the management device 100 that performs the management processing according to the embodiment will be described. Figure 8 is a diagram showing an example of the configuration of the management device 100 according to the embodiment.

[0081] As shown in Figure 8, the management device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The management device 100 may also include an input unit (e.g., a keyboard or mouse) for receiving various operations from an administrator or other person managing the management device 100, and a display unit (e.g., a liquid crystal display) for displaying various information.

[0082] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card) or a network interface controller. The communication unit 110 is connected to the network N (e.g., the Internet) by wire or wireless connection and transmits and receives information with the user terminal 20, authentication terminal 40, facial recognition provider 50, etc. via the network N. For example, the communication unit 110 may transmit and receive information using any communication standard or technology such as Wi-Fi (registered trademark), Bluetooth, SIM (Subscriber Identity Module), or LPWA (Low Power Wide Area).

[0083] The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The storage unit 120 includes a user information storage unit 121, a quality check item storage unit 122, a facial image storage unit 123, an authentication log storage unit 124, and an external service information storage unit 125.

[0084] The following describes each memory unit in order, using Figures 9 to 12. In the examples shown in Figures 9 to 12, the information stored in memory unit 120 is sometimes conceptually represented as "A01," but in reality, each piece of information described later will be stored in memory unit 120.

[0085] The user information storage unit 121 stores information about users who use facial recognition. Figure 9 shows an example of the information stored by the user information storage unit 121. Figure 9 is a diagram showing an example of the user information storage unit 121 according to the embodiment. In the example shown in Figure 9, the user information storage unit 121 has items such as "user ID", "login information", "face photo registration", and "external service information".

[0086] "User ID" indicates identification information that identifies the user. "Login Information" indicates login information required for registering a facial photo, such as a password linked to the User ID. "Facial Photo Registration" indicates whether the user has registered a facial photo or not. "External Service Information" indicates information about external services used by the user.

[0087] The quality check item storage unit 122 stores information regarding quality check items for each facial recognition engine. Figure 10 shows an example of the quality check item storage unit 122 according to the embodiment. In the example shown in Figure 10, the quality check item storage unit 122 has items such as "check item," "first facial recognition engine," "second facial recognition engine," and "third facial recognition engine."

[0088] The quality check items shown in Figure 10 indicate the criteria each facial recognition engine uses to check the quality of a facial image and output a result of registration OK or NG. One example shown in Figure 10 shows that the first facial recognition engine determines the angle of the face in the image, including left / right, up / down, and tilt deviations, and whether these fall within predetermined criteria. In other words, the first facial recognition engine requires that the face in the image be facing directly forward as a registration requirement, and uses the angle of the face as a quality check item. On the other hand, the example in Figure 10 shows that the third facial recognition engine checks only the tilt deviation, regardless of the left / right or up / down direction of the face angle in the image. Thus, each facial recognition engine has different facial angles and characteristics that it excels at recognizing, and this is related to the differences in the recognition results. When the management device 100 receives a registration NG judgment from the first facial recognition engine, it obtains which check items were flagged. The management device 100 can list the acquired information, for example, in the judgment result 340 shown in Figure 4, and notify the user of which items the processed image to be registered was not appropriate.

[0089] Note that the quality check items shown in Figure 10 are just an example, and the control device 100 may store information related to various other quality check items besides those shown in Figure 10.

[0090] The facial image storage unit 123 stores facial image data, including facial images used for facial recognition. Figure 11 shows an example of the facial image storage unit 123 according to the embodiment. In the example shown in Figure 11, the facial image storage unit 123 has items such as "User ID," "Registered Facial Image Data," "For Authentication," and "For ID Photo."

[0091] "User ID" refers to identification information that identifies the user. "Registered facial photo data" refers to facial photo data registered by the user. "For authentication" refers to facial photo data used for facial authentication. "For identification photos" refers to facial photo data that has been cropped or had its background changed for use as an identification photo. In other words, the management device 100 can store the user's facial photo data separately for authentication and for identification photos.

[0092] The authentication log storage unit 124 stores logs of user facial recognition attempts. For example, the authentication log storage unit 124 stores the date and time the user attempted facial recognition, the result of the facial recognition, etc. The authentication log storage unit 124 may also store logs of user service usage. For example, the authentication log storage unit 124 may store user attendance records, etc.

[0093] The external service information storage unit 125 stores information about external services that become available to the user after identity verification such as facial recognition. Figure 12 shows an example of the external service information storage unit 125 according to the embodiment. In the example shown in Figure 12, the external service information storage unit 125 has items such as "service ID," "type," and "judgment criteria."

[0094] The "Service ID" indicates identification information that identifies the service. The "Type" indicates the type of service. The "Decision Criteria" indicates the criteria for whether or not to allow the user to use the service based on the authentication strength related to facial recognition. For example, if the decision criterion is "OR judgment," the user can use the service if the matching result is OK in any one of the multiple facial recognition engines. If the decision criterion is "AND judgment," the user can use the service only if the matching result is OK in all of the multiple facial recognition engines. If the decision criterion is "OK judgment with 2 or more," the user can use the service if the matching result is OK in any two or more of the multiple facial recognition engines. Note that the decision criteria shown in Figure 12 are just examples, and external services may register various decision criteria in the management device 100 based on the desired authentication strength.

[0095] Returning to Figure 8, let's continue the explanation. The control unit 130 is implemented by, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc., which executes a program stored inside the management device 100 using RAM (Random Access Memory) or the like as the working area. The control unit 130 is also a controller and is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0096] As shown in Figure 8, the control unit 130 includes a registration processing unit 130A and an authentication processing unit 130B. The registration processing unit 130A controls the processing related to the registration of face images. The registration processing unit 130A includes a reception unit 131, a provision unit 132, a quality determination unit 133, and a registration unit 134. The authentication processing unit 130B controls the processing related to authentication. The authentication processing unit 130B includes an acquisition unit 135, a transmission unit 136, and a result determination unit 137.

[0097] The reception unit 131 receives facial images from the user, which will serve as the basis for the ground truth data used by each of the multiple facial recognition engines for authentication processing. Note that the ground truth data used by the facial recognition engines for authentication is not the facial image itself, but rather the feature quantities extracted from the facial image.

[0098] For example, the reception unit 131 provides the user with a page (website, etc.) for registering a facial image, and receives the facial image from the user via this page. This facial image is used as a registration image to be registered with the facial recognition engine. If the facial image has been processed by a processing means provided by the provision unit 132 (described later), the reception unit 131 accepts the processed image as the registration image.

[0099] The provisioning unit 132 provides processing means for the user to process the facial image received by the receiving unit 131.

[0100] For example, the providing unit 132 provides processing means for converting at least one of the following: the size of the face image, the area occupied by the face in the face image, the brightness of the face image, and the background of the face image. Specifically, as shown in Figures 3 and 4, the providing unit 132 provides editing tools on a page for registering face images that allow the user to perform operations such as cropping, brightness adjustment, and background change. The types of processing are not limited to those mentioned above, and the providing unit 132 may also provide tools that can perform, for example, teeth whitening or skin tone whitening on the face image. These processing can be performed by the user themselves by operating the tools to adjust the teeth or skin tone to the desired color, or the providing unit 132 may automatically adjust them to the appropriate color.

[0101] Furthermore, the providing unit 132 may provide reference information for processing the image into a processed image suitable for the service used by the user after authentication processing, or through multiple facial recognition engines. The reference information is, for example, a guide 306 displayed on the screen of the user terminal 20 to help the user process the image into a suitable processed image, as shown in Figure 3. The guide 306 is displayed based, for example, on the proportion and position of the faces included in the facial image. The reference information may also include horizontal lines to check the parallelism of the faces and vertical lines to check the center of the image.

[0102] Furthermore, the providing unit 132 does not have to provide all the processing means shown in Figures 3 and 4. For example, if it is clear that the user will not use the processed image for identification photos, the providing unit 132 may skip the background change page, which is mainly used for creating identification photos, and proceed with the registration process.

[0103] Furthermore, if it is difficult for the user to select the brightness and size of the image themselves, the providing unit 132 may perform a process to automatically adjust the brightness and size. That is, the providing unit 132 may, in accordance with the user's request, convert at least one of the following: the size of the face image, the area occupied by the face in the face image, the brightness of the face image, or the background of the face image, and automatically generate a processed image that can be used as correct answer data for the authentication process of multiple face recognition engines. In this case, the providing unit 132 may, for example, refer to the quality check item storage unit 122 to obtain information such as the required brightness of the face image, the size of the image, and the position where the face should be placed in the image. Then, the providing unit 132 automatically processes the face image to conform to these requirements and generates a processed image. At this time, the providing unit 132 may utilize known technologies, such as a process to detect faces contained in the image. As a result, the providing unit 132 can generate a processed image that guarantees the required quality without requiring any effort from the user.

[0104] Furthermore, if the quality determination unit 133, described later, determines that a predetermined quality check item related to the processed image does not meet the quality requirements, the providing unit 132 may convert the information within the processed image related to that quality check item to meet the quality requirements. For example, if the quality determination unit 133 determines that the brightness of the processed image is insufficient, the providing unit 132 may automatically convert the processed image to make it brighter to meet the quality requirements and generate the converted image. In this case, the providing unit 132 may present the converted image to the user once, or it may automatically replace the original processed image with the converted image and proceed with the registration process.

[0105] The quality determination unit 133 determines whether the registration images, such as the face images received by the reception unit 131 and the processed images which are face images processed by the processing means, are of a quality that can be used as correct answer data for the authentication process of multiple face recognition engines.

[0106] Specifically, the quality determination unit 133 determines for each of the multiple facial recognition engines whether the registration image is of sufficient quality to be used as correct data for the authentication process of the multiple facial recognition engines.

[0107] More specifically, the quality determination unit 133 determines whether the registration image meets the criteria for predetermined quality check items set for each of the multiple face recognition engines, and based on the determination result, determines whether the registration image is of a quality that can be used as correct data for the authentication process of the multiple face recognition engines.

[0108] The registration unit 134 registers the registration image if it determines, based on the judgment result from the quality judgment unit 133, that the registration image is of sufficient quality to be used as correct data for authentication processing for all of the multiple face recognition engines.

[0109] If the quality determination unit 133 determines that any of the predetermined quality check items related to the registration image do not meet the quality standards, the registration unit 134 may notify the user which quality check item does not meet the standards. For example, the registration unit 134 notifies the user which quality check item does not meet the standards by displaying the screen 338 and the determination result 340 shown in Figure 4 on the user terminal 20.

[0110] Furthermore, if the registration unit 134 determines, based on the quality determination result by the quality determination unit 133, that the registration image is not of sufficient quality to be used as correct data for authentication processing for any of the multiple face recognition engines, it will notify the user that the registration image cannot be registered as an authentication face image. In other words, if there are multiple face recognition engines specified in the service, the registration unit 134 will only register face images that can be registered by all of those face recognition engines. This ensures security in face recognition processing.

[0111] Furthermore, when registering a registration image, the registration unit 134 may store a second face image linked to the registration image, wherein at least one of the following aspects differs from the registration image: the size of the face image, the area the face occupies in the face image, the brightness of the face image, or the background of the face image. Specifically, when registering a user, the registration unit 134 may store a face image for identification photos together with the authentication face image. This allows the management device 100 to provide the face image for identification photos to the service side as needed, such as for creating an ID card in the service.

[0112] The acquisition unit 135 acquires facial images from the user for use in executing authentication processing on multiple facial recognition engines. For example, the acquisition unit 135 acquires facial images for executing authentication processing via the authentication terminal 40 that has taken a picture of the user.

[0113] The transmitting unit 136 transmits the facial image acquired by the acquisition unit 135 to each of the multiple facial recognition engines.

[0114] Furthermore, the transmitting unit 136 does not necessarily need to transmit the facial image to all facial recognition engines, depending on the characteristics of the acquired facial image and the requirements of the service. Some facial recognition engines may have higher accuracy in matching bright images (e.g., images taken during the day) or dark images (e.g., images taken at night), or engines that have been trained to have higher accuracy in matching specific races. For this reason, if the acquired facial image has a brightness lower or higher than a predetermined standard value, or if the facial image is of a specific race, the transmitting unit 136 may preferentially transmit the facial image to a facial recognition engine with higher accuracy for those conditions. In some cases, the service that the user intends to use after authentication may specify a particular facial recognition engine (e.g., a facial recognition engine that is highly reliable for the service). In such cases, the transmitting unit 136 may select the facial recognition engine to which to transmit the facial image, depending on the type of user request (i.e., the service the user intends to use).

[0115] In other words, the transmitting unit 136 may select which facial recognition engine to transmit the facial image to based on information about the facial image or a user request. For example, the transmitting unit 136 may select which facial recognition engine to transmit the facial image to based on the brightness of the facial image or user characteristic information (such as skin color or race) contained in the facial image.

[0116] The result determination unit 137 determines whether or not to allow the user's request based on the number of facial recognition engines that successfully matched the facial image with the correct data among the multiple facial recognition engines.

[0117] For example, the result determination unit 137 determines whether to permit the user's request based on whether the number of facial recognition engines that successfully matched the facial image with the correct data among multiple facial recognition engines meets the criteria set by the service the user will use after the authentication process. Specifically, the result determination unit 137 refers to the external service information storage unit 125 and, if a matching result is obtained that satisfies the criteria required by the service, it determines that the user's identity has been authenticated and permits the use of the service. If the service does not specify any criteria, the result determination unit 137 may permit the user's request if any of the facial recognition engines successfully match the image (OR determination), or it may permit the user's request if all of the facial recognition engines successfully match the image (AND determination).

[0118] Furthermore, if the face image is transmitted by the transmission unit 136 only to a specific face recognition engine, the result determination unit 137 may determine whether or not to permit the user's request based on the authentication result of the face recognition engine to which the face image was transmitted.

[0119] (2. Modified examples of embodiments) (2-1.Device configuration) In the above embodiment, an example was shown in which the user terminal 20 is used when registering a facial image in the management system 1, and the authentication terminal 40 is used during the authentication process. However, the configuration and roles of the devices within the management system 1 can be changed as appropriate.

[0120] For example, when a user attempts to use a service on a website displayed on the user terminal 20, the user terminal 20 may also act as an authentication terminal 40. That is, the user terminal 20 may transmit a facial image of the user to the management device 100 and have the management device 100 perform facial authentication processing for the user. For example, if the user terminal 20 receives a result from the management device 100 indicating that the facial authentication has been successful, it may perform actions after service use has been permitted, such as making the service the user attempted to access available.

[0121] Furthermore, the above embodiment shows an example in which each facial recognition provider is operated by a different business operator than the management device 100, and each facial recognition provider is operated by a different business operator. However, the management device 100 may not receive facial recognition engines from facial recognition providers, but may operate multiple facial recognition engines itself and execute the processing according to the embodiment in an on-premise environment. In this case, the management device 100 may have multiple facial recognition engines with different characteristics, for example, by generating multiple facial recognition engines using different learning methods. This allows the management device 100 to perform management processing using multiple facial recognition engines without necessarily relying on multiple business operators. Also, the management device 100 does not necessarily have to be a cloud server, and may function as a platform for managing multiple facial recognition engines in a system built on a local network.

[0122] (3. Other Embodiments) The processing according to the above-described embodiment may be carried out in various other forms besides those described above.

[0123] For example, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0124] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0125] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0126] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0127] (4. Effects of the control device related to this disclosure) As described above, the management device (management device 100 in the embodiment) relating to this disclosure includes a reception unit (reception unit 131 in the embodiment), a provision unit (provision unit 132 in the embodiment), and a quality determination unit (quality determination unit 133 in the embodiment). The reception unit receives a facial image from the user, which will be the source of the correct answer data used by each of the multiple facial recognition engines for authentication processing by the multiple facial recognition engines. The provision unit provides processing means for the user to process the facial image received by the reception unit. The quality determination unit determines whether the processed image, which is a facial image processed by the processing means, is of a quality that can be used as correct answer data for authentication processing by the multiple facial recognition engines.

[0128] Thus, the management device relating to this disclosure provides a means for processing facial images received from users and determines the quality of the processed images to determine whether the facial recognition engine can use them for authentication processing. In this way, the management device 100 can perform processing such as cropping and quality determination processing together on the server-side platform, making it possible to obtain high-quality facial images, which leads to improved accuracy in subsequent feature extraction. Furthermore, by checking the quality of the processed images, the management device can prevent the need to retake image captures, thus reducing the burden on the user and enabling accurate registration of facial images.

[0129] Furthermore, the service provider will provide reference information for processing the image into a processed image suitable for the service used by the user, after authentication processing, using multiple facial recognition engines.

[0130] In this way, the management device provides users with reference information indicating face size and other details along with processing methods, thereby standardizing the size and quality of registration images received from users and facilitating smooth registration.

[0131] Furthermore, the providing unit provides processing means for converting at least one of the following: the size of the face image received by the receiving unit, the area occupied by the face in the face image, the brightness of the face image, and the background of the face image.

[0132] In this way, the management device can provide an environment where users can edit images as they wish by providing means for processing images taken by the user.

[0133] Furthermore, the provisioning unit, in accordance with the user's request, transforms at least one of the following: the size of the face image received by the receiving unit, the area occupied by the face in the face image, the brightness of the face image, and the background of the face image, and generates a processed image that can be used as correct answer data for the authentication process of multiple face recognition engines.

[0134] In this way, the management device can automatically process facial images, enabling the generation of processed images that meet the required quality without requiring any effort from the user.

[0135] Furthermore, if the quality determination unit determines that a predetermined quality check item related to the processed image does not meet the quality requirements, the supply unit will convert the information within the processed image related to that predetermined quality check item to meet the quality requirements.

[0136] In this way, the management device can automatically convert images to maintain quality, thereby generating processed images that guarantee the required quality without requiring any effort from the user.

[0137] The management device may also be configured to include an acquisition unit (acquisition unit 135 in this embodiment), a transmission unit (transmission unit 136 in this embodiment), and a result determination unit (result determination unit 137 in this embodiment). The acquisition unit acquires a facial image from the user for multiple facial recognition engines to perform authentication processing. The transmission unit transmits the facial image acquired by the acquisition unit to each of the multiple facial recognition engines. The result determination unit determines whether or not to permit the user's request based on the authentication results of each of the multiple facial recognition engines that authenticated the facial image.

[0138] In this way, the management device performs authentication processing using multiple facial recognition engines. This allows the management device to improve authentication accuracy and enhance security, as well as provide flexible authentication methods, such as setting different judgment criteria for each service.

[0139] Furthermore, the result determination unit determines whether or not to grant the user's request based on the number of facial recognition engines that successfully matched the facial image with the correct data among the multiple facial recognition engines.

[0140] In this way, the management device performs authentication using multiple facial recognition engines, allowing it to leverage the strengths of various facial recognition engines, such as a facial recognition engine that is strong in a particular environment or a facial recognition engine that is strong for a particular race.

[0141] Furthermore, the result determination unit determines whether to permit the user's request based on whether the number of facial recognition engines that successfully matched the facial image with the correct data among the multiple facial recognition engines meets the criteria set by the service used by the user after the authentication process.

[0142] In this way, the management device can determine whether or not to permit a user's request based on the service's criteria. This means that it can perform flexible authentication processing according to the authentication strength and needs of the service, rather than simply granting the user's identity based on a successful match by a single facial recognition engine.

[0143] Furthermore, the transmission unit selects which facial recognition engine to send the facial image to based on information about the facial image or the user's request. The result determination unit determines whether or not to permit the user's request based on the authentication result of the facial recognition engine to which the facial image was sent.

[0144] Thus, the management device can leverage its strength of utilizing multiple facial recognition engines and selectively use them based on the image characteristics that each engine excels at. This allows the management device to perform authentication with greater security in mind, by using only the most accurate facial recognition engines.

[0145] Furthermore, the transmission unit selects which facial recognition engine to send the facial image to based on the brightness of the facial image or the user's characteristic information contained in the facial image.

[0146] In this way, the management device can selectively transmit facial images that each facial recognition engine excels at by selecting a facial recognition engine according to the characteristics of the image.

[0147] The management device may also be configured to include a reception unit, a quality determination unit, and a registration unit (a registration unit 134 in this embodiment). The reception unit receives a registration image from the user, which will serve as the basis for the correct answer data used by each of the multiple facial recognition engines for authentication processing. The quality determination unit determines, for each of the multiple facial recognition engines, whether the registration image is of sufficient quality to be used as the correct answer data for the authentication processing of the multiple facial recognition engines. Based on the determination result by the quality determination unit, the registration unit registers the registration image if it determines that the registration image is of sufficient quality to be used as the correct answer data for the authentication processing of all of the multiple facial recognition engines.

[0148] In this way, the management device registers only facial images that meet the quality requirements of all facial recognition engines used. This allows the management device to smoothly carry out authentication processing using multiple facial recognition engines.

[0149] Furthermore, the quality determination unit determines whether the registration image meets the criteria for predetermined quality check items set for each of the multiple facial recognition engines, and based on the determination result, determines whether the registration image is of sufficient quality to be used as correct data for the authentication process of the multiple facial recognition engines.

[0150] In this way, the management device sets out items to check in quality assessment and performs quality assessment according to these items. As a result, the management device can reliably register facial images of a quality suitable for use by the facial recognition engine.

[0151] Furthermore, if the quality judgment unit determines that any of the predetermined quality check items for the registration image do not meet the quality standards, the registration unit will notify the user of which quality check item does not meet the standards.

[0152] In this way, the management device notifies the user if each item does not meet the quality standards, making it easy for the user to understand how to correct the images for registration.

[0153] Furthermore, if the registration unit determines, based on the quality determination result from the quality determination unit, that the registration image is not of sufficient quality to be used as correct data for authentication processing for any of the multiple face recognition engines, it will notify the user that the registration image cannot be registered as an authentication face image.

[0154] In this way, the management device can notify the user if registration failed with any of the facial recognition engines, thereby prompting the user to properly register with all of them.

[0155] Furthermore, when registering a registration image, the registration unit stores a second face image (for example, an image for identification photos) associated with the registration image, in which at least one of the following aspects differs: the size of the face image, the area occupied by the face image, the brightness of the face image, or the background of the face image.

[0156] Thus, the management device may store a separate image for identification photos in addition to the authentication image. This allows the management device to provide the service provider with a flexible platform, such as providing the facial image for identification photos as needed, for example, when creating an ID card for the service.

[0157] (5. Hardware Configuration) The information devices such as the management device 100 and user terminal 20 according to the embodiments described above are realized by a computer 1000 having a configuration such as that shown in Figure 13. The following explanation will use the management device 100 according to the embodiments as an example. Figure 13 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the management device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0158] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls various parts. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0159] ROM1300 stores boot programs such as the BIOS (Basic Input Output System) executed by CPU1100 when computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0160] HDD1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU1100 and data used by such programs. Specifically, HDD1400 is a recording medium that records a program that executes the management processing related to this disclosure, which is an example of program data 1450.

[0161] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it generates to other devices via the communication interface 1500.

[0162] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0163] For example, when computer 1000 functions as a management device 100 according to the embodiment, the CPU 1100 of computer 1000 realizes functions such as the control unit 130 by executing a management processing program loaded on RAM 1200. The HDD 1400 stores programs that execute the management processing according to this disclosure and data in the storage unit 120. The CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0164] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention. [Explanation of symbols]

[0165] 1 Management System 10 users 20 User Terminals 40 Authentication terminals 50 facial recognition providers 60 First facial recognition engine 70 Second facial recognition engine 80 Third facial recognition engine 100 Management device 110 Communications Department 120 Storage section 121 User Information Storage Unit 122 Quality Check Item Storage Unit 123 Facial Photo Memory Unit 124 Authentication log storage unit 125 External service information storage unit 130 Control Unit 131 Reception Department 132 Provision Department 133 Quality Judgment Department 134 Registration Department 135 Acquisition Department 136 Transmitter 137 Result judgment section 200 service providers

Claims

1. A unit for acquiring facial images from the user in order to perform authentication processing on multiple facial recognition engines, A transmission unit transmits the facial image acquired by the acquisition unit to each of the plurality of facial recognition engines, A result determination unit determines whether or not to permit the user's request based on the authentication results of each of the multiple facial recognition engines that authenticated the aforementioned facial image, Equipped with, The result determination unit, The system determines whether to permit the user's request based on the number of facial recognition engines that successfully match the facial image with the correct data among the multiple facial recognition engines. A control device characterized by the following features.

2. The result determination unit, Based on whether the number of facial recognition engines that successfully match the facial image with the correct data among the plurality of facial recognition engines meets the criteria set by the service used by the user after the authentication process, the system determines whether to permit the user's request. The control device according to feature 1.

3. The aforementioned transmitting unit Based on the information regarding the facial image or the user's request, select which facial recognition engine to send the facial image to. The result determination unit, Based on the authentication result of the facial recognition engine to which the facial image was transmitted, it is determined whether or not to permit the user's request. The control device according to claim 1 or 2.

4. The aforementioned transmitting unit Based on the brightness of the facial image or the user's characteristic information contained in the facial image, the system selects which facial recognition engine to send the facial image to. The control device according to feature 3.

5. Computers Obtain facial images from the user to perform authentication processing on multiple facial recognition engines, The acquired facial image is transmitted to each of the multiple facial recognition engines. Based on the authentication results of each of the multiple facial recognition engines that authenticated the aforementioned facial image, it is determined whether or not to permit the user's request. This includes, and furthermore, The system determines whether to permit the user's request based on the number of facial recognition engines that successfully match the facial image with the correct data among the multiple facial recognition engines. A management method characterized by the following features.

6. Computers, A unit for acquiring facial images from the user in order to perform authentication processing on multiple facial recognition engines, A transmission unit transmits the facial image acquired by the acquisition unit to each of the plurality of facial recognition engines, A result determination unit determines whether or not to permit the user's request based on the authentication results of each of the multiple facial recognition engines that authenticated the aforementioned facial image, A management program characterized by functioning as a management device equipped with the following features: The result determination unit, The system determines whether to permit the user's request based on the number of facial recognition engines that successfully match the facial image with the correct data among the multiple facial recognition engines. Management program.

Images