Method and system for incorporating aged face into face recognition system using artificial intelligence

Abstract

A system for performing face recognition includes a memory for storing a plurality of registered face recognition templates for a plurality of registered users, a camera for capturing a current face image of an individual, and a controller operatively coupled to the memory and the camera. The controller is configured to determine whether a current facial image of the individual matches one of a plurality of registered facial recognition templates. When the current face image of the individual matches one of the plurality of registered face recognition templates, the controller is configured to recognize a registered user of the plurality of registered users that matches the current face image of the individual, and updating a registered facial recognition template for the matched registered user based on the current facial image of the individual.

Description

Technical Field

[0001] The present disclosure relates generally to facial recognition systems, and more particularly to incorporating aging faces into facial recognition systems. Background Art

[0002] Many systems use facial recognition to identify an individual and determine whether the individual is authorized to access a secure system or area again. The secure system or area may be, for example, a secure computer system, a secure banking system, a secure area of ​​a building, and / or any other secure system or area. A facial recognition system may store multiple images of each authorized individual, each image may be taken from a different angle, such as a frontal image, a left profile image, a right profile image, and so on. However, people age. This means that over time, facial recognition systems may become less accurate in correctly identifying people. What is desired is a method and system for incorporating the effects of aging on a person's face. What is desired is a method and system for incorporating an aging face into a facial recognition system using artificial intelligence. Summary of the invention

[0003] The present disclosure generally relates to methods and systems for using facial recognition, and more specifically to incorporating aging faces into facial recognition systems. An example may be found in a method for performing facial recognition. The illustrative method includes storing a plurality of registered facial recognition templates for a plurality of registered users. A current facial image of an individual is captured. A current facial recognition template for the individual is generated based at least in part on the current facial image of the individual. An artificial intelligence (AI) algorithm is executed to determine whether the current facial recognition template for the individual matches one of the plurality of registered facial recognition templates. The artificial intelligence (AI) algorithm is trained to recognize changes in facial appearance due to aging using a plurality of facial images of a training data set, wherein the training data set includes facial images that are not facial images of any of the plurality of registered users, and further training is performed based at least in part on a plurality of registered facial recognition templates for the plurality of registered users. When the current facial recognition template for the individual matches one of the plurality of registered facial recognition templates, a registered user corresponding to the matching registered facial recognition template in the plurality of registered facial recognition templates is identified. Based on one or more differences between the current facial recognition template and the previously registered facial recognition template of the registered user for matching, the registered facial recognition template for matching is updated. Then, an artificial intelligence (AI) algorithm is retrained using the updated registered facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

[0004] Another example may be present in a method for performing facial recognition. The method includes storing multiple registered facial recognition templates for multiple registered users. Capturing a current facial image of an individual, and generating a current facial recognition template for the individual based at least in part on the current facial image of the individual. Executing an artificial intelligence (AI) algorithm to determine whether the current facial recognition template for the individual matches one of the multiple registered facial recognition templates, wherein the artificial intelligence (AI) algorithm is trained to recognize changes in facial appearance due to aging. When the current facial recognition template for the individual matches one of the multiple registered facial recognition templates, identifying a registered user corresponding to the matching registered facial recognition template in the multiple registered facial recognition templates, and updating the registered facial recognition template for the matching registered user based on one or more differences between the current facial recognition template and a previous registered facial recognition template for the matching registered user. Then, retraining the artificial intelligence (AI) algorithm using the updated registered facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

[0005] Another example may exist in a system for performing facial recognition. The system includes: a memory for storing a plurality of registered facial recognition templates for a plurality of registered users, a camera for capturing a current facial image of an individual, and a controller operatively coupled to the memory and the camera. The controller is configured to determine whether the current facial image of the individual matches one of the plurality of registered facial recognition templates. When the current facial image of the individual matches one of the plurality of registered facial recognition templates, the controller is configured to identify a registered user among the plurality of registered users that matches the current facial image of the individual, and to update the registered facial recognition template for the matching registered user based on the current facial image of the individual.

[0006] The above summary is provided to facilitate understanding of some innovative features unique to the present disclosure, but is not intended to be a complete description. A comprehensive understanding of the present disclosure can be obtained by viewing the entire specification, claims, drawings and abstract as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure may be more fully understood by considering the following description of various examples in conjunction with the accompanying drawings, in which:

[0008] Figure 1 is a schematic block diagram illustrating an exemplary system for performing facial recognition;

[0009] Figure 2A and Figure 2B are flow charts that collectively illustrate illustrative methods for performing facial recognition;

[0010] Figure 3A and Figure 3B are flow charts that collectively illustrate an illustrative method for performing facial recognition; and

[0011] Figure 4 is a flow chart illustrating an exemplary method.

[0012] Although the present disclosure is subject to various modifications and alternative forms, the details thereof have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the present disclosure to the specific examples described. On the contrary, it is intended to cover all modifications, equivalents and alternatives that fall within the essence and scope of the present disclosure. DETAILED DESCRIPTION

[0013] The following description should be read with reference to the accompanying drawings, in which similar elements in different drawings are numbered in the same manner. The accompanying drawings are not necessarily drawn to scale, and they depict examples that are not intended to limit the scope of the present disclosure. Although examples of various elements are shown, those skilled in the art will recognize that many of the examples provided have suitable alternatives that can be utilized.

[0014] It is assumed herein that all numbers are modified by the term "about" unless the content clearly dictates otherwise. The recitation of numerical ranges by endpoints includes all numbers subsumed within that range (eg, 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0015] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally employed in its sense including "and / or" unless the content clearly dictates otherwise.

[0016] It should be noted that references to "one embodiment", "some embodiments", "other embodiments", etc. in the specification indicate that the described embodiments may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is contemplated that the feature, structure, or characteristic is described in conjunction with one embodiment, and it is contemplated that the feature, structure, or characteristic may be applied to other embodiments regardless of whether explicitly described, unless otherwise clearly stated.

[0017] Figure 11 is a schematic block diagram showing an exemplary system 10 for performing facial recognition. The exemplary system 10 includes a memory 12 for storing a plurality of registered facial recognition templates 14 for a plurality of registered users. The registered facial recognition templates 14 are labeled 14a, 14b, and 14c, respectively. Although a total of three registered facial recognition templates 14 are shown, it should be understood that there may be more than three registered facial recognition templates 14, especially in the case where the system 10 is used for an access control system 22 installed in a large facility. The system 10 includes a camera 16 for capturing a current facial image of an individual. In some cases, the camera 16 may be installed adjacent to a controlled door or gate, for example, so as to capture their image when a person approaches the door or gate. Although a single camera 16 is shown, in some cases, there may be a plurality of cameras 16, each of which is installed adjacent to a controlled door or access area. In the example shown, the controller 18 is operatively coupled to the memory 12 and to the camera 16. In some cases, controller 18 includes or is configured to execute an artificial intelligence (AI) algorithm 20 .

[0018] The controller 18 is configured to determine whether the current facial image of the individual captured by the camera 16 matches one of the multiple registered facial recognition templates 14. When the current facial image of the individual matches one of the multiple registered facial recognition templates 14, the controller 18 is configured to identify a registered user among the multiple registered users that matches the current facial image of the individual, and update the registered facial recognition template for the matching registered user based on the current facial image of the individual.

[0019] In some cases, the controller 18 is configured to execute the AI ​​algorithm 20 to determine whether the current facial image of the individual matches one of the multiple registered facial recognition templates 14. In some cases, the AI ​​algorithm 20 may be trained to recognize changes in facial appearance due to aging using multiple facial images of a training data set. The training data set may include facial images that are not facial images of any of the multiple registered users. In some cases, the training data set may include a VGGFace data set and / or a VGGFace2 data set. The AI ​​algorithm 20 may also be trained at least in part based on multiple registered facial recognition templates 14 for multiple registered users. When the current facial image of the individual matches one of the multiple registered facial recognition templates 14, the controller 18 may retrain the AI ​​algorithm 20 using the updated registered facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging, including for registered users that match the current facial image of the individual and in some cases for other registered users.

[0020] In some cases, the system 10 may be configured to support an access control system 22 that controls access to a secure area or a restricted area. In some cases, the camera 16 may be mounted adjacent to the access control system 22 so that the camera 16 is positioned to capture facial images of each person attempting or otherwise desiring to access the secure area or the restricted area. The camera 16 may be positioned to capture facial images of each person approaching a door or gate that restricts access to the secure area or the restricted area when locked, but allows access to the secure area or the restricted area by unlocking the door or gate when the approaching individual is identified and has the appropriate access rights to gain access to the secure area or the restricted area. When the current facial image of the individual matches one of the multiple registered facial recognition templates 14, the controller 18 may be configured to determine whether the matched registered user has permission to access the corresponding secure area, and if so, the controller 18 may control the access control system 22 to allow the matched registered user to access the secure area.

[0021] Figure 2A and Figure 2B 2 are flow charts collectively illustrating an exemplary method 24 for performing facial recognition. The exemplary method 24 includes storing a plurality of registered facial recognition templates for a plurality of registered users, as shown at block 26. The facial recognition template for any particular registered user may be based on one facial image of the registered user or a plurality of facial images of the registered users, such as a frontal image, a left profile image, a right profile image, and so forth.

[0022] A current facial image of the individual is captured, as shown at block 28. Based at least in part on the current facial image of the individual, a current facial recognition template for the individual is generated, as shown at block 30. In some cases, generating the current facial recognition template for the individual includes converting the current facial image of the individual into the current facial recognition template for the individual. In some cases, converting the current facial image of the individual into the current facial recognition template may include extracting one or more features from the current facial image of the individual and providing the extracted one or more features to the current facial recognition template. For example, the one or more features may correspond to one or more facial features of the individual (e.g., the distance between the eyes, the depth of the eye sockets, the distance from the forehead to the chin, the shape of the cheekbones, the contour of the lips, ears, and / or chin). These are just examples.

[0023] An artificial intelligence (AI) algorithm is executed to determine whether a current facial recognition template for an individual matches one of a plurality of registered facial recognition templates, as shown at block 32. The AI ​​algorithm is trained to recognize changes in facial appearance due to aging using a plurality of facial images of a training data set, wherein the training data set includes facial images that are not facial images of any of a plurality of registered users, as shown at block 32a. In some cases, the training data set may be a large-scale data set that includes facial images annotated with age, gender, and race. The training data set may be generated internally using the captured images of the person. For example, the training data set may be commercially available. In some cases, the training data set may include a VGGFace data set and / or a VGGFace2 data set. The AI ​​algorithm is also trained based, at least in part, on a plurality of registered facial recognition templates for a plurality of registered users, as shown at block 32b.

[0024] Method 24 Figure 2B Continued above. When the current facial recognition template for an individual matches one of the multiple registered facial recognition templates, method 24 includes several illustrative actions, as shown at box 34. These actions include identifying a registered user corresponding to a matching registered facial recognition template among the multiple registered facial recognition templates, as shown at box 34a. These actions include updating the registered facial recognition template for the matching registered user based on one or more differences between the current facial recognition template and the previously registered facial recognition template for the matching registered user, as shown at box 34b. These actions include retraining an artificial intelligence (AI) algorithm using the updated registered facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging, as shown at box 34c.

[0025] In some cases, the plurality of registered facial recognition templates and the updated registered facial recognition template may each include a timestamp, and two or more of the timestamps may be used to retrain an artificial intelligence (AI) algorithm to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging. For example, the difference between the timestamp of the updated registered facial recognition template and the timestamp of the previous registered facial recognition template may represent a time difference in aging of the registered user, and may be used to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

[0026] In some cases, the method 24 may also include: when the current facial recognition template of the individual matches one of the multiple registered facial recognition templates, determining whether the matched registered user has permission to access the secure area, and if so, controlling the access control system to allow the matched registered user to access the secure area, as shown at block 36. In some cases, when the current facial recognition template of the individual does not match any of the multiple registered facial recognition templates, the method 24 may control the access control system to prevent the individual from accessing the secure area, as shown at block 38.

[0027] Figure 3A and Figure 3B 4 is a flowchart collectively illustrating an exemplary method 40 for performing facial recognition. The method 40 includes storing a plurality of registered facial recognition templates for a plurality of registered users, as shown at block 42. A current facial image of an individual is captured, as shown at block 44. Based at least in part on the current facial image of the individual, a current facial recognition template for the individual is generated, as shown at block 46. In some cases, generating the current facial recognition template for the individual includes converting the current facial image of the individual into the current facial recognition template for the individual. In some cases, converting the current facial image of the individual into the current facial recognition template for the individual may include extracting one or more characteristics of the current facial image and providing the extracted one or more characteristics to the current facial recognition template. For example, the one or more characteristics may correspond to one or more facial features of the individual (e.g., the distance between the eyes, the depth of the eye sockets, the distance from the forehead to the chin, the shape of the cheekbones, the contour of the lips, ears, and / or chin). These are just examples. An artificial intelligence (AI) algorithm is executed to determine whether a current facial recognition template for the individual matches one of a plurality of enrolled facial recognition templates, wherein the artificial intelligence (AI) algorithm is trained to recognize changes in facial appearance due to aging, as shown at box 48.

[0028] Method 40 Figure 3BContinued above. When the current facial recognition template for an individual matches one of the multiple registered facial recognition templates, several exemplary actions are taken, as shown in box 50. These actions include identifying a registered user corresponding to a matching registered facial recognition template among the multiple registered facial recognition templates, as shown in box 50a. These actions include updating the registered facial recognition template for the matched registered user based on one or more differences between the current facial recognition template and the previously registered facial recognition template for the matched registered user, as shown in box 50b. These actions include retraining an artificial intelligence (AI) algorithm using the updated registered facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging, as shown in box 50c. In some cases, the multiple registered facial recognition templates and the updated registered facial recognition template each include a timestamp, and the artificial intelligence (AI) algorithm can be retrained using two or more of the timestamps to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

[0029] In some cases, when the current facial recognition template of the individual matches one of the multiple registered facial recognition templates, the method 40 may include determining whether the matched registered user has permission to access the secure area, and if so, controlling the access control system to allow the matched registered user to access the secure area, as shown at block 52. In some cases, when the current facial recognition template of the individual does not match any of the multiple registered facial recognition templates, the method 40 includes controlling the access control system to prevent the individual from accessing the secure area, as shown at block 54.

[0030] Figure 4 is a flow chart showing an exemplary method 56 for performing facial recognition. As can be seen, the method 56 includes an enrollment phase 58 and a verification phase 60. In the enrollment phase 58, a camera captures a facial image, as shown at box 62. The facial image may include one facial image for each enrollee, or multiple facial images for each enrollee. For example, the multiple facial images of each enrollee may include, for example, a frontal image, a left profile image, a right profile image, an image with glasses, an image with a hat, etc. The facial images are processed, as shown at box 64. This may include generating a registered facial recognition template for each enrolled individual. The results are stored in a database, as shown at box 66.

[0031] In the verification phase 60, the camera captures a facial image, as shown at box 68. The facial image is processed, as shown at box 70. In some cases, the facial image is compared to an enrolled facial recognition template to determine if there is a match, as shown at decision box 72. In some cases, the facial image is processed to generate a current facial recognition template, and the current facial recognition template is compared to the enrolled facial recognition template to determine if there is a match. In any case, if there is a match, the individual is authenticated, as shown at 76, and the AI ​​will update the matched enrolled facial recognition template based on the features extracted from the facial image captured by the camera at box 68, as shown by the dashed line 74. If there is no match, the individual is not authenticated, as shown at box 78.

[0032] Although several exemplary embodiments of the present disclosure have been described in this manner, it will be readily understood by those skilled in the art that other embodiments may be made and used within the scope of the claims appended hereto. However, it should be understood that the present disclosure is in many respects merely illustrative. Details (especially details related to shape, size, arrangement of parts, and exclusion and order of steps) may be changed without exceeding the scope of the present disclosure. Of course, the scope of the present disclosure is defined in the language of the appended claims.

Claims

1. A method for performing facial recognition, the method comprising: storing a plurality of registered facial recognition templates for a plurality of registered users; capturing a current facial image of the individual; generating a current facial recognition template for the individual based at least in part on the current facial image of the individual; executing an artificial intelligence (AI) algorithm to determine whether the current facial recognition template for the individual matches one of the plurality of enrolled facial recognition templates, wherein the artificial intelligence (AI) algorithm: trained to recognize changes in facial appearance due to aging using a plurality of facial images of a training data set, wherein the training data set includes facial images that are not facial images of any of the plurality of enrolled users; training based at least in part on the plurality of registered facial recognition templates for the plurality of registered users; When the current facial recognition template for the individual matches one of the plurality of enrolled facial recognition templates: identifying a registered user corresponding to a matching registered facial recognition template among the plurality of registered facial recognition templates; updating the registered facial recognition template for a matching registered user based on one or more differences between the current facial recognition template and a previously registered facial recognition template for a matching registered user; as well as The artificial intelligence (AI) algorithm is retrained using the updated enrolled facial recognition template to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

2. The method according to claim 1, wherein when the current facial recognition template of the individual matches one of the multiple registered facial recognition templates, it is determined whether the matching registered user has the authority to access the secure area, and if so, the access control system is controlled to allow the matching registered user to access the secure area.

3. The method of claim 2, wherein when the current facial recognition template of the individual does not match any of the plurality of registered facial recognition templates, controlling the access control system to prevent the individual from accessing the secure area.

4. The method of claim 1 , wherein the plurality of registered facial recognition templates and the updated registered facial recognition templates each include a timestamp, wherein two or more of the timestamps are used to retrain the artificial intelligence (AI) algorithm to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

5. The method of claim 4, wherein the difference between the timestamp of the updated registered facial recognition template and the timestamp of the previous registered facial recognition template represents the aging time difference of the registered user and is used to improve the ability of the artificial intelligence (AI) algorithm to recognize changes in facial appearance due to aging.

6. The method of claim 1, wherein generating the current facial recognition template for the individual comprises converting the current facial image of the individual into the current facial recognition template for the individual.

7. The method of claim 6, wherein converting the current facial image of the individual into the current facial recognition template for the individual comprises extracting one or more features from the current facial image and providing the extracted one or more features to the current facial recognition template.

8. The method of claim 7, wherein the one or more characteristics correspond to one or more facial features of the individual.

9. The method of claim 1, wherein the training dataset is a large-scale dataset comprising facial images annotated with age, gender, and ethnicity.

10. A system for performing facial recognition, the system comprising: a memory for storing a plurality of registered facial recognition templates for a plurality of registered users; a camera for capturing a current facial image of the individual; a controller operatively coupled to the memory and the camera, the controller being configured to: determining whether the current facial image of the individual matches one of the plurality of enrolled facial recognition templates; When the current facial recognition template of the individual matches one of the plurality of enrolled facial recognition templates: identifying the registered user of the plurality of registered users that matches the current facial image of the individual; and The registered facial recognition template of the registered user for matching is updated based on the current facial image of the individual.

Images