Face recognition system and method capable of updating registered face template

Abstract

The present invention relates to a face recognition system and method capable of updating registered face templates. The face recognition system according to the present invention comprises: a registered template database (DB) storing registered face templates; an authentication template database for storing authentication face templates; a control unit storing the authentication face templates in the authentication template database according to the user's face authentication, and after a predetermined time, using multiple authentication face templates stored in the authentication template database to calculate the similarity between each of the multiple authentication face templates, using these similarity values ​​to calculate the average similarity value for each of the multiple authentication face templates, and updating the registered face template by selecting the authentication face template with the maximum value in the average similarity value as a new registered face template and storing it in the registered template database, thereby being able to automatically update the registered face template.

Description

Technical Field

[0001] The present invention relates to a face recognition system and method capable of updating a registered face template, and in particular, to a face recognition system and method capable of automatically updating a registered face template using an authenticated face template. Background Art

[0002] Face recognition technology is a technology that automatically detects and analyzes face areas from images that include faces to identify who the corresponding face is. It has been studied for a long time in the field of pattern recognition and computer vision. Based on many research results, face recognition technology has recently been put into practical use in the fields of surveillance systems, entry and exit control, or biometrics.

[0003] As we all know, this face recognition technology, as a relatively mature technology, shows high performance compared to other object recognition technologies. However, it is well known that there are still many problems to be solved in images acquired in real environments due to posture and lighting changes, low resolution / blurring caused by long-distance shooting and human body movement.

[0004] In other words, the face image used for actual authentication may change due to various variables, such as the user's posture, the brightness of the user's surroundings, the contrast of the user's face, etc. As a result, user authentication often fails or the authentication processing time increases.

[0005] In addition, a technology is being used that can automatically update a registered face image based on a face image used by a user for authentication. However, if the face image to be automatically updated is not correctly determined, the number of authentication failures increases. Summary of the invention

[0006] Problems to be solved by the invention

[0007] In order to solve the above problems, the object of the present invention is to provide a face recognition system and method that can use the similarities between multiple authentication face templates to automatically update the registered face template.

[0008] Solutions for solving problems

[0009] In order to achieve the above-mentioned purpose, a face recognition system according to an embodiment of the present invention includes: a registration template database (DB), which stores registration face templates; an authentication template database, which is used to store authentication face templates; and a control unit, which stores the authentication face templates in the authentication template database according to the user's face authentication, and after a predetermined time, uses the multiple authentication face templates stored in the authentication template database to calculate the similarity between each of the multiple authentication face templates, uses these similarity values ​​to calculate the average similarity value for each of the multiple authentication face templates, selects the authentication face template with the maximum value among the average similarity values ​​as the new registration face template, and stores the new registration face template in the registration template database to update the registration face template.

[0010] The control unit may store the template values ​​obtained from the authentication face templates through a neural network. In order to select the new registered face template, the values ​​of the multiple authentication face templates stored in the authentication template database are used to calculate the distance between each of the multiple authentication face templates. These distance values ​​are used to calculate the average distance value for each of the multiple authentication face templates. The face template having the minimum value among the average distance values ​​is selected as the new registered face template.

[0011] The control unit may calculate the distance between each of the plurality of authentication face templates, and select the new registration face template using only the values ​​of the authentication face templates having an average distance value less than a predetermined value.

[0012] The control unit may store the time information of the authentication face template in the time information area of ​​the authentication template database, and use the time information stored in the time information area to select the new registration face template for each predetermined time period.

[0013] The above-mentioned face recognition system may also include a camera unit, which is used to capture images including human faces. The above-mentioned control unit can store the size information of the face image in the image captured by the above-mentioned camera unit in the size information area of ​​the above-mentioned authentication template database, and only use the value of the authentication face template whose size information stored in the above-mentioned size information area is above the predetermined size value to select the above-mentioned new registered face template.

[0014] The size information of the facial image may be at least one of the maximum size of the length and the width of the facial image or the pixels of the facial image.

[0015] The above-mentioned face recognition system may also include: a camera unit, used to capture images including faces; a face detection unit, used to detect face images in the images captured by the above-mentioned camera unit; a feature extraction unit, used to extract feature point information in the above-mentioned face images; a normalization processing unit, which normalizes the face images by using the above-mentioned feature point information to generate an aligned face template; and a face recognition unit, which compares the face template input from the above-mentioned normalization processing unit with the registered face template stored in the above-mentioned registration template database to determine whether the above-mentioned input face template belongs to a registered user.

[0016] In a face recognition method based on a face recognition system according to another embodiment of the present invention, the face recognition system includes a registration template database storing registered face templates and an authentication template database for storing authentication face templates. The face recognition method may include: based on the user's face authentication, storing the authentication face template in the authentication template database; after a predetermined time, using multiple authentication face templates stored in the authentication template database to calculate the similarity between each of the multiple authentication face templates; using these similarity values ​​to calculate the average similarity value for each of the multiple authentication face templates; selecting the authentication face template with the maximum value of the average similarity value as a new registration face template; and storing the new registration face template in the registration template database to update the registration face template.

[0017] In addition, the above-mentioned face recognition method may also include: a step of storing the template value obtained from the above-mentioned authentication face template through a neural network; a step of using the values ​​of multiple authentication face templates stored in the above-mentioned authentication template database to calculate the distance between each of the above-mentioned multiple authentication face templates; a step of using these distance values ​​to calculate the average distance value for each of the above-mentioned multiple authentication face templates; and a step of selecting the face template with the minimum value among the above-mentioned average distance values ​​as the above-mentioned new registered face template.

[0018] Effects of the Invention

[0019] The newly registered face image may be the face image closest to the average face image in the predetermined time period. Therefore, by using the newly registered face image, the speed of authenticating the user's face can be increased, and authentication failures can also be reduced.

[0020] Furthermore, in the present invention, by excluding some facial images from the facial images to be authenticated, changes in facial images with age, seasons, etc. can be further reflected, and authentication failures can also be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a diagram showing a block diagram of a face recognition system according to an embodiment of the present invention.

[0022] Figure 2 It is shown Figure 1 FIG. 4 is a diagram showing an example of a table of an authentication template database.

[0023] Figure 3 It is shown in the form of coordinates Figure 2 A diagram of the template values ​​stored in the Template Value area is shown.

[0024] Figure 4 is a diagram showing a flow chart of a face recognition method according to another embodiment of the present invention. DETAILED DESCRIPTION

[0025] Hereinafter, the preferred embodiments of the face recognition system and method capable of updating the registered face template according to the present invention are described with reference to the accompanying drawings. For reference, when describing the present invention below, the terms of the components of the present invention are named in consideration of the functions of each component, and therefore should not be understood as limiting the technical components of the present invention.

[0026] Figure 1 is a diagram showing a block diagram of a face recognition system according to an embodiment of the present invention, Figure 2 It is shown Figure 1 FIG. 1 is a diagram showing an example of a table of an authentication template database, Figure 3 It is shown in the form of coordinates Figure 2 A diagram of the template values ​​stored in the template value area shown, Figure 4 is a diagram showing a flow chart of a face recognition method according to another embodiment of the present invention.

[0027] like Figure 1 As shown, the face recognition system includes a template storage unit 110, a camera unit 120, a face detection unit 130, a feature extraction unit 140, a normalization processing unit 150, a face recognition unit 160, and a control unit 170. The face recognition system can be specifically divided into a face processing device (not shown) and a management server (not shown) for operation.

[0028] As storage space for storing templates, the template storage unit 110 includes a registration template database 112 and an authentication template database 114. The registration template database 112 may store a plurality of registered face templates stored for registration, normalized feature point information of a plurality of registered face templates, template values ​​of a plurality of registered face templates, and time information when a plurality of registered face templates are stored. The authentication template database 114 may store a plurality of authentication face templates based on user face authentication, normalized feature point information of a plurality of authentication face templates, template values ​​of a plurality of authentication face templates, time information when a plurality of authentication face templates are stored, and size information related to the face detected by the face detection unit 130. Figure 2 An example of a table of the authentication template database 114 is shown in FIG.

[0029] The camera section 120 is installed near the entrance door to capture images including faces of people.

[0030] The face detection unit 130 basically detects a face image present in the image captured by the camera unit 120. The image captured by the camera unit 120 is an image including a face. The face detection unit 130 may use a Haar-like feature algorithm to detect a face image from the image input from the camera unit 120. In this case, facial parts (head, eyes, nose, mouth, and beard) may be detected separately and determined as the final face. The face detection unit 130 may improve the performance of the final face detection by fine-tuning the face image.

[0031] The feature extraction unit 140 extracts feature point information from the face image of the user. The feature extraction unit 140 may extract 68 feature points, ie, landmarks, from the face image using, for example, a pre-learned 3D face geometry model.

[0032] The feature extraction unit 140 finds the feature points of eyebrows, eyes, nose, mouth and their surroundings in the face image and maps them to the face image. Of course, the feature points are not limited to the specific parts listed above. The feature extraction unit 140 can be implemented by a pattern classifier, such as a support vector machine (SVM). The feature extraction unit 140 can identify the left-right tilt of the face based on the position of the two eyes.

[0033] The feature extraction unit 140 may also be configured to extract feature vectors from the landmark information, and use the feature vectors to identify the user's facial expression and the user's current emotional state. In this case, the feature extraction unit 140 may extract feature vectors from the landmark information by using the angle and distance ratio information between the landmarks. In particular, when extracting feature vectors, it is crucial to reduce the amount of calculation by minimizing the dimension of the feature vector. As a result, the speed of identifying the user's facial expression can be increased compared to existing face recognition technologies.

[0034] The feature extraction unit 140 may also use, for example, texture features of the face image, morphological features of the entire face image, morphological features of body parts (eyes, nose, mouth, etc.) to generate a feature vector, and the feature vector expresses the face image. The feature extraction unit 140 may also transform the face image from an RGB image into a brightness / color difference (Y / C) image, and generate a feature vector from the transformed image. The color difference image refers to an image that excludes brightness values ​​and uses only pure color information to express the difference between colors. The feature extraction unit 140 may generate a feature vector using only the color difference image in addition to the brightness image to reduce the influence of lighting.

[0035] The normalization processing unit 150 normalizes the face image by using the feature point information extracted from the face image, thereby generating an aligned face image. The normalization processing unit 150 performs an affine transformation, i.e., a two-dimensional affine transformation or a three-dimensional affine transformation, on the face image by using the feature point information, thereby performing image normalization processing such as face alignment and photometric correction. In this case, the face image can be rotated so that the center of the eye and the center of the mouth are vertical, and the size of the face image can be corrected so that the distance between the center of the eye and the center of the mouth is 40 pixels. Thus, even if the detected face image is a side face rather than a front face, a frontal face image can be generated by transforming the image.

[0036] The facial images aligned in the normalization processing unit 150 can be stored in the template storage unit 110 as facial templates. In this case, the normalized feature point information obtained from the aligned facial images can be stored in the template storage unit 110 to replace the facial template, or can be stored in a separate area together with the facial template. After the normalization processing unit 150 divides the facial image into a plurality of blocks using the normalized feature point information, each block generates a normalized feature vector. The control unit 170 can store the normalized feature vector of each block in the normalized feature point information area of ​​the template storage unit 110. In the present embodiment, the facial template and the normalized feature point information obtained from the aligned facial images are distinguished from each other. However, it should be noted that the normalized feature point information obtained from the aligned facial images is derived from the aligned facial images and is therefore included in the facial template.

[0037] The face recognition unit 160 compares the face template of the image input to the camera unit 120 with the registered face template stored in the registered template database 112 to determine whether the face template of the input image belongs to a registered user. The face recognition unit 160 can use various deep learning techniques, such as deep neural networks (DNN, Deep Neural Network), convolutional neural networks (CNN, Convolutional Neural Network), recurrent neural networks (RNN, Recurrent Neural Network), restricted Boltzmann machines (RBM, Restricted Boltzmann Machine), deep belief neural networks (DBN, Deep Belief Network), deep Q-networks (Deep Q Networks), etc., but is not limited to these. In this embodiment, the face recognition unit 160 can be configured as a convolution neural network composed of multiple layers. At this time, a locally connected convolution layer can be used to effectively extract local features within the face area.

[0038] The face recognition unit 160 can determine the facial similarity by using the feature point information of the face template. At this time, the face recognition unit 160 can determine the similarity of each stored registered face template by using the face template of the input image, and can determine whether the user is a registered user by using each similarity result. For example, it is assumed that four registered face templates are stored in the registered template database 112, and the face similarities are determined to be 97%, 85%, 80% and 75% respectively. The maximum value of the similarity is used to check whether the above user is a registered user. When the maximum value of the similarity is higher than the reference value, the above user can be determined to be a registered user. For example, when the reference value is set to 95%, in the above assumption, the maximum value of the similarity is 97%, and the above maximum value is higher than the above reference value, so the above user can be determined to be a registered user.

[0039] Specifically, the face recognition unit 160 can retrieve the face template processed by the normalization processing unit 150 and the registered face template stored in the registered template database 112, provide data to the convolutional neural network in units of blocks, determine the similarity of each block by calculating the block template value for each block, and add them all up to determine the total similarity. In this case, the normalized feature vector or template value of each block stored in the registered template database 112 can be used to calculate the similarity.

[0040] On the one hand, the face recognition unit 160 can perform face verification in combination with training. Face verification is to verify whether two face templates of the same person are determined to be the same person. In this way, the false recognition rate generated during the face recognition process in the image can be reduced.

[0041] In training, multiple learning models may be used to train facial feature points. The multiple learning models may include at least one of a user's facial learning model, another person's facial learning model, an illumination learning model, and a posture learning model. For example, in training, based on facial feature points detected from learning data of multiple learning models, the learning data may be divided into at least one region to train facial feature points.

[0042] During training, the representative feature vector for each facial feature point can be stored in a buffer in the same number as the facial feature points used for learning data. For example, in the case where the facial features are of the same person, the data of the facial feature points are formed around a specific vector, so the representative feature vector as the center can be stored in a separate buffer to make it the center point of learning.

[0043] In training, if a predetermined number of different images of the same user are input as learning data, output values ​​of facial feature points, facial feature points caused by lighting changes, and facial feature points for posture changes can be obtained.

[0044] In the registration mode, the control unit 170 stores the registration face template generated by the normalization processing unit 150 in the registration template database 112. In addition, the control unit 170 controls the face recognition unit 160 in the training mode and performs deep learning training, that is, the verification face template of the same user generated by the normalization processing unit 150 is compared with the registration face template stored in the registration template database 112, and the value of face verification according to the training can be provided to the convolution neural network.

[0045] In the authentication mode, the control unit 170 controls the face recognition unit 160 to compare the face template generated by the normalization unit 150 with all face templates registered in the registration template database 112 to determine whether the face template of the input image belongs to a registered user.

[0046] If the face template of the input image belongs to a registered user, for example, after opening the door, the control unit 170 stores the face template of the input image and / or the normalized feature point information in the face template area and / or the feature point information area of ​​the authentication template database 114, and stores the template value calculated by the face recognition unit 160 through the convolutional neural network in the template value area, and may also store the storage time in the time information area. In addition, the control unit 170 may store the size information related to the face detected by the face detection unit 130 in the size information area of ​​the authentication template database 114. In this case, the size information may be the maximum size of the length and / or width of the face image or the number of pixels of the face image.

[0047] The control unit 170 may update the registered face template stored in the registered template database 112 after a predetermined period of time, such as one month. The control unit 170 retrieves the template value of the corresponding user stored in the authentication template database 114, such as the template value of ID1. Figure 3 In the form of coordinates Figure 2 The template values ​​stored in the template value area are shown in FIG. 1 , where blue is the template value of the registered face template currently stored in the registered template database 112 , and all the remaining colors are the template values ​​of the face template stored in the authentication template database 114 .

[0048] The control unit 170 obtains the mutual similarity among the template values ​​obtained within a predetermined time period, uses these similarity values ​​to obtain the average similarity value for each of the multiple authentication face templates, and selects the face template with the maximum value among these average similarity values ​​as the new registered face template. Then, the control unit 170 stores all the information related to the face template selected as the registered face template in the registered template database 112 to update the registered face template. In this case, the control unit 170 can keep the information of the previously stored registered face template as it is. Thus, the control unit 170 can use the newly updated face template in the registered template database 112 to process face recognition. If the similarity is low, face recognition can also be processed by using the registered face template originally stored in the registered template database 112.

[0049] The average similarity value means, for example, when there are three authentication face templates, if the similarity between the first authentication face template and the second authentication face template is 82, the similarity between the second authentication face template and the third authentication face template is 90, and the similarity between the first authentication face template and the third authentication face template is 86, then the average similarity of the first authentication face template is (82+86) / 2=84, the average similarity of the second authentication face template is (82+90) / 2=86, and the average similarity of the third authentication face template is (90+86) / 2=88. Therefore, the authentication face template with the largest average similarity value is the third authentication face template.

[0050] Specifically, the control unit 170 uses the template values ​​to calculate the distance between the templates, uses these distance values ​​to calculate the average distance value for each of the multiple authentication face templates, and selects the face template with the minimum value among these average distance values ​​as the new registration face template. In this case, the control unit 170 can pre-exclude face templates with average distance values ​​above a predetermined threshold from the candidates for registration face templates. That is, in the entire distribution of template values ​​of face templates stored in the authentication template database 114, face templates that exceed a predetermined deviation can be excluded from the candidates for registration face templates. Figure 3 As shown, compared with the template value on the right, the two template values ​​on the left are closer to the registered face template value, but are far away from the entire template value stored in the authentication template database 114. Therefore, the face template selected as the new registered face template can further reflect the changes in the face image with age, season, etc.

[0051] The control unit 170 may also use the size information of the face image detected by the face detection unit 130 to exclude face templates whose size information is smaller than a predetermined threshold from the registered face template candidates in advance. Thus, errors caused by the normalization process can be eliminated.

[0052] The control unit 170 can also use the time information of the stored face template to determine the registered face template for each predetermined time period, thereby reflecting the facial changes according to time within a day and the changes in illumination during night and day time periods.

[0053] Figure 4 is a diagram showing a flow chart of a face recognition method according to another embodiment of the present invention.

[0054] In the registration mode, the control unit 170 stores the registration face template generated by the normalization processing unit 150 in the registration template database 112 ( S402 ).

[0055] In the training mode, the control unit 170 controls the face recognition unit 160 to operate in the training mode, so that the face recognition unit 160 performs deep learning training, that is, the verification face template of the same user generated by the normalization processing unit 150 is compared with the registered face template stored in the registered template database 112 (S404).

[0056] In the authentication mode, the control unit 170 controls the face recognition unit 160 to operate in the authentication mode, so that the face recognition unit 160 compares the face template generated by the normalization processing unit 150 with the face template registered in the registered template database 112, thereby determining whether the face template of the input image belongs to the registered user (S406).

[0057] If the face template of the input image belongs to a registered user, the control unit 170 stores the face template of the input image in the face template area of ​​the authentication template database 114 as an authentication face template (S408). In this case, the control unit 170 stores the normalized feature point information in the feature point information area of ​​the authentication template database 114, and stores the template value obtained by the convolutional neural network in the face recognition unit 160 in the template value area, and may also store the storage time in the time information area. In addition, the control unit 170 may store the size information related to the face detected by the face detection unit 130 in the size information area of ​​the authentication template database 114.

[0058] The control section 170 determines whether a predetermined period of time, for example, one month, has passed (S410). When the predetermined period of time has passed, the control section 170 reads a plurality of authentication face templates stored in the authentication template database 114 (S412).

[0059] The control unit 170 calculates the distance between each of the multiple authentication face templates, and only determines the values ​​of the authentication face templates whose average distance value is less than a predetermined value as the registration template candidates (S414). At this time, the control unit 170 can also determine the values ​​of the authentication face templates whose size information stored in the size information area of ​​the authentication template database is greater than the predetermined size value as the registration template candidates.

[0060] The control unit 170 calculates the similarity between each of the registered template candidates determined from the multiple authentication face templates stored in the authentication template database 114, and selects the authentication face template with the largest average similarity value as the new registered face template (S416). In this case, in order to select a new registered face template, the control unit 170 calculates the distance between each of the authentication face template values ​​of the selected registration template candidates from the multiple authentication face template values ​​stored in the authentication template database 114, and selects the face template with the smallest average distance value as the above-mentioned new registered face template.

[0061] The control unit 170 updates the registered face template by storing the selected new registered face template in the registered template database 112 ( S418 ).

[0062] The embodiments of the present invention described above are merely exemplary of the technical ideas of the present invention, and the protection scope of the present invention shall be understood to be subject to the claims. Moreover, a person skilled in the art in the field of the present invention may make various modifications and changes without departing from the essential features of the present invention, and all technical ideas within the scope equivalent to the present invention shall be understood to be included in the scope of the rights of the present invention.

Claims

1. A face recognition system, including a function of updating a registered face template, characterized in that: include: A registration template database storing registration face templates, wherein the registration face templates are used as face templates generated based on face images of registered users and are used as benchmarks for verifying face templates of users attempting face authentication; An authentication template database, used to store multiple authentication face templates, wherein each of the multiple authentication face templates is a face template generated based on a face image of a user attempting face authentication, and in the face authentication, each time when it is determined that the user attempting face authentication is the registered user, the face template used in the face authentication is set as the authentication face template and is stored in the authentication template database each time; a camera unit configured to capture an image including a face of a user attempting the facial authentication; A face detection unit, used to detect a face image in the image taken by the camera unit; A feature extraction unit, used to extract feature point information from the above-mentioned face image; A normalization processing unit, which performs normalization processing on the face image by using the feature point information, thereby generating an aligned face template; The face recognition unit compares the face template of the user attempting the face authentication input from the normalization unit with the registered face template stored in the registered template database and calculates the similarity, and when the calculated similarity is greater than a predetermined reference value, determines that the user attempting the face authentication is the registered user; and The control unit, after a predetermined period of time and multiple face authentications, uses the multiple authentication face templates stored in the above-mentioned authentication template database to calculate the similarity between each of the above-mentioned multiple authentication face templates in all combinations of two of them, averages the relevant similarities of each of the above-mentioned multiple authentication face templates to calculate the average similarity value, selects the authentication face template representing the maximum value in the above-mentioned average similarity value as the new registered face template, and stores the new registered face template in the above-mentioned registered template database to update the above-mentioned registered face template.

2. The face recognition system according to claim 1, characterized in that: The control unit stores the time information of the authentication face template in the time information area of ​​the authentication template database, and selects the new registration face template for each predetermined time period using the time information stored in the time information area.

3. The face recognition system according to claim 1, characterized in that: The control unit stores the size information of the facial image in the image taken by the camera unit in the size information area of ​​the authentication template database, and selects the new registered face template only using the value of the authentication face template whose size information stored in the size information area is greater than a predetermined size value.

4. The face recognition system according to claim 3, characterized in that: The size information of the facial image is at least one of the maximum size of the length and width of the facial image or the number of pixels of the facial image.

5. A face recognition method, comprising a function of updating a registered face template, which is executed by the face recognition system of claim 1 comprising a function of updating a registered face template, characterized in that: The registered face template is used as a face template generated based on the face image of the registered user and is used as a benchmark for verifying the face template of the user attempting face authentication. The above-mentioned face recognition method includes: The step of storing a plurality of authentication face templates in an authentication template database, wherein each of the plurality of authentication face templates is a face template generated based on a face image of a user attempting to perform face authentication, and in the face authentication, each time when it is determined that the user attempting to perform face authentication is the registered user, the face template used in the face authentication is set as the authentication face template and is stored in the authentication template database each time; After a predetermined time has passed and multiple face authentications have been performed, and multiple authentication face templates are stored in the authentication template database, a step of using the multiple authentication face templates stored in the authentication template database to calculate the similarity between all combinations of two of the multiple authentication face templates; For each of the plurality of authentication face templates, averaging the relevant similarities of each of the templates to obtain an average similarity value; The step of selecting the authentication face template representing the maximum value in the above average similarity value as a new registration face template; and The step of storing the new registered face template in the registered template database to update the registered face template.

Images