Device for identity authentication via simultaneous recognition of multiple features within facial region

Abstract

According to the present invention, a device for identity authentication via simultaneous recognition of multiple features within a facial region comprises: a communication unit for receiving an image including a front face of a subject from a user terminal; a detection unit for detecting a plurality of individual sub-regions from a face region in the image received from the communication unit; an extraction unit for, through the plurality of sub-regions, generating first feature data in the form of a vector value, in which a plurality of landmarks are extracted for the entire face region from a single image, and generating second feature data in the form of a vector value, in which a plurality of landmarks are extracted for both eyes and surrounding regions thereof; a connection unit for, when the user terminal requests identity registration, generating third feature data in the form of a vector value by simply concatenating the respective vector values of the first and second feature data individually extracted from the extraction unit; and a storage unit for storing the third feature data.

Description

Identity authentication device through simultaneous recognition of multiple features within the facial area

[0001] The present invention relates to an identity authentication device that simultaneously recognizes multiple features within a facial area of ​​a subject and authenticates the identity of the subject by confirming the degree of similarity with features already registered.

[0002]

[0003] Due to the development of communication technology, data processing technology, and sensor technology, the existing target identification technology, in which verification personnel checked documents about the target or directly checked the target's appearance, has evolved into a technology that identifies the target by authenticating the target's biometric information (fingerprints, voice print, iris, face, finger vein, spleen vein, etc.).

[0004] This target identification technology is applied to various service fields and industrial fields that require target matching confirmation, target authority confirmation, etc. through target biometric authentication, and this biometric authentication technology has been disclosed in Republic of Korea Patent Publication No. 10-2008-0047925 (application date: 2006.11. 27., publication date: 2008.05. 30., hereinafter referred to as “prior art”), and the present invention is presented as a technology related thereto that improves and supplements the prior art.

[0005] However, since the prior art does not have a countermeasure for the case where the newly recognized biometric information is not complete in the process of authenticating biometric information by comparing the previously stored biometric information with the newly recognized biometric information, there was a problem in that when authentication was attempted through facial recognition in a situation where the subject of authentication had to inevitably wear a mask, sunglasses, etc. and part of the eyes or facial area could not be exposed, identity authentication was impossible unless the subject of the authentication request removed the mask or sunglasses, or even if identity authentication was performed, it was difficult to trust the identity authentication result.

[0006]

[0007] The present invention aims to solve the above-described problem by providing an identity authentication device that can accurately authenticate the identity of a person requesting identity authentication through simultaneous recognition of multiple features in the facial area even in situations where wearing a mask or sunglasses, etc. is unavoidable.

[0008]

[0009] According to one embodiment of the present invention, an identity authentication device through simultaneous recognition of multiple features in a facial region comprises: a communication unit that confirms a request for either identity registration or identity authentication from a user terminal and receives a single image including the entire face of a subject from the user terminal according to the request; a detection unit that recognizes a facial region from the single image received from the communication unit and detects a plurality of partial regions for both eyes, a nose, a mouth, and a facial contour from the recognized facial region; a generation unit that generates a first image for the entire facial region and a second image for both eyes and a surrounding area of ​​both eyes from the single image through the plurality of partial regions detected by the detection unit; an extraction unit that generates first feature data in the form of a vector value by extracting a plurality of landmarks for both eyes, a nose, a mouth, and a facial contour from the first image generated from the generation unit using different deep learning-based feature extraction models, and generates second feature data in the form of a vector value by extracting a plurality of landmarks for both eyes and a surrounding area of ​​both eyes from the second image; It is characterized by including: a connection unit that simply concatenates each vector value of the first and second feature data individually extracted from the extraction unit to generate third feature data in the form of a vector value only when it is confirmed that the user terminal has requested identity registration through the communication unit; and a storage unit in which the third feature data generated from the connection unit is stored.

[0010] Here, the detection unit extracts a plurality of feature points from a single image received from the communication unit with an extraction complexity lower than the extraction complexity of extracting a plurality of feature points from the first and second images in the extraction unit, thereby recognizing a face region, and detects a plurality of partial regions for both eyes, a nose, a mouth, and a facial contour through the extracted plurality of feature points.

[0011] And, the detection unit is characterized in that it detects the tilt and rotation of the face region through the relative position difference for the detected multiple partial regions, and corrects the tilt and rotation of the detected face region to create a frontal face image of the subject from a single image.

[0012]

[0013] *In addition, the generation unit is characterized in that it generates first and second images with different resolutions for the entire face area, both eyes, and the surrounding area of ​​both eyes according to the input values ​​of each deep learning-based feature extraction model for the frontal face image of the target corrected by the detection unit.

[0014] In addition, the extraction unit is characterized in that it generates first and second feature data by extracting the number of unit vectors for a plurality of feature points for both eyes, a nose, a mouth, and a facial contour from the first image so that the number of unit vectors for a plurality of feature points for both eyes and a surrounding area of ​​both eyes from the second image is greater than the number of unit vectors for a plurality of feature points for both eyes and a surrounding area of ​​both eyes from the second image.

[0015] Meanwhile, in the case where it is confirmed through the communication unit that the user terminal has requested identity authentication, when a single image including the front face of the received target is received, an authentication unit is further included that performs facial authentication of the target requested from the user terminal by determining whether, among the plurality of third feature data pre-stored in the storage unit, there exists third feature data whose similarity with the first and second feature data generated through the detection unit, generation unit, and extraction unit exceeds a preset threshold value for the single image received from the user terminal requesting identity authentication.

[0016] At this time, the authentication unit is characterized in that it performs identity authentication for a single image including the front face of the subject requested from the user terminal by determining whether there exists third feature data whose sum result of applying a weight according to the reliability of the feature extraction model that extracted the first feature data to a similarity value with the first feature data among a plurality of third feature data stored in the storage unit and applying a weight according to the reliability of the feature extraction model that extracted the second feature data to a similarity value with the second feature data exceeds a preset threshold.

[0017] In addition, the connection unit is characterized in that it obtains a user ID of a subject who has requested identity registration through the user terminal from the communication unit, matches the obtained user ID with third characteristic data generated from a single image including the front face of the subject, and stores the result in the storage unit.

[0018]

[0019] According to the present invention, an identity authentication device through simultaneous recognition of multiple features in a facial region simply connects each feature data in the form of vector values ​​individually extracted from multiple regions through each feature extraction model in a single image acquired from a user terminal and stores it as a single feature data, and at the time of identity authentication, authenticates whether the weighted sum of the similarity value between each feature data in the form of vector values ​​individually extracted from multiple regions through each feature extraction model in a single image acquired from the user terminal and the previously stored simply connected feature data and the reliability value of the feature extraction model that extracted each feature data exceeds a preset threshold. Therefore, even if an image in which the entire face region or part of the eye region is unclear is acquired during the identity authentication process, identity authentication is performed based on the result of summing the similarities for each of the face and the eyes, thereby enabling identity authentication with high accuracy and reliability regardless of a situation in which feature extraction is impossible for a specific region, such as when the identity authentication request subject is wearing a mask.

[0020]

[0021] FIG. 1 illustrates an identity authentication device through simultaneous recognition of multiple features in a facial region according to a preferred embodiment of the present invention.

[0022] FIG. 2 is a visual representation of the operation of each component in an identity registration process through an identity authentication device that simultaneously recognizes multiple features in a facial region according to a preferred embodiment of the present invention.

[0023] FIG. 3 is a visual representation of the operation of each component in an identity authentication process using an identity authentication device that simultaneously recognizes multiple features in a facial region according to a preferred embodiment of the present invention.

[0024]

[0025] Hereinafter, a preferred embodiment of an identity authentication device through simultaneous recognition of multiple features in a facial region of the present invention will be described in detail with reference to the attached drawings.

[0026] The same reference numerals in each drawing represent the same components. In addition, specific structural and functional descriptions of embodiments of the present invention are merely illustrative for the purpose of explaining embodiments according to the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant technology, and are preferably not interpreted in an ideal or excessively formal sense unless explicitly defined herein.

[0027] Before explaining each component of an identity authentication device through simultaneous recognition of multiple features in a facial region according to a preferred embodiment of the present invention with reference to the attached drawings, the identity authentication device through simultaneous recognition of multiple features in a facial region of the present invention is linked to a user terminal such as a smart phone through an application, and when a registration or authentication request is made for the identity of the subject through the user terminal, the identity authentication device through simultaneous recognition of multiple features in a facial region can perform the registration or authentication of the identity from a single image including the face of the subject of the identity registration or identity authentication request provided from the user terminal. The identity authentication device through simultaneous recognition of multiple features in a facial region can be provided in the form of a server that is connected to the user terminal through a communication network and performs identity registration and identity authentication through an application installed in the user terminal. It should be understood that each component of the identity authentication device through simultaneous recognition of multiple features in a facial region, which will be described below, is not an actual component such as an independent device or part installed in a server, but an abstract component provided through the linked operation of a processor, RAM, a storage medium, and an application installed in the storage medium in the server.

[0028]

[0029] FIG. 1 illustrates an identity authentication device through simultaneous recognition of multiple features in a facial region according to a preferred embodiment of the present invention, and FIG. 2 visualizes the operation of each component in an identity registration process through an identity authentication device through simultaneous recognition of multiple features in a facial region according to a preferred embodiment of the present invention.

[0030] Referring to FIGS. 1 and 2, an identity authentication device (100, hereinafter referred to as an “identity authentication device”) through simultaneous recognition of multiple features in a facial region according to a preferred embodiment of the present invention, when an identity registration request is made from a user terminal (200) along with a face-containing image of an identity registration request target, extracts feature data in the form of vector values ​​for each of the entire face region and each of the eye regions from the face-containing image through separate feature extraction models, and stores each feature data by simply concatenating them, and when an identity authentication request is made from the user terminal (200) along with a face-containing image of an identity authentication request target, similarly extracts feature data for the entire face region and each of the eye regions, and performs identity authentication by comparing the similarity with the feature data stored in a simply concatenated form, and may include a communication unit (110), a detection unit (120), a generation unit (130), an extraction unit (140), a connection unit (150), a storage unit (160), and an authentication unit (170).

[0031] The communication unit (110) can confirm a request for either identity registration or identity authentication from the user terminal (200), and receive a single image including the entire face of the subject from the user terminal (200) according to the request. Here, when an identity registration request is made from a user terminal (200) having an application that is linked to an identity authentication device (100) installed through the corresponding application, the communication unit (110) can include an algorithm that can call the detection unit (120), the generation unit (130), the extraction unit (140), and the connection unit (150) described below, but can call the authentication unit (170) described below when an identity authentication request is made from the user terminal (200) through the corresponding application, and can be provided in the form of a typical communication module that is connected to a communication network and communicates with the user terminal (200). In addition, the communication unit (110) may provide the user terminal (200) with the result of identity registration through the connection unit (150) described later and the result of identity authentication through the authentication unit (170).

[0032] The detection unit (120) can recognize a face area from a single image received from the communication unit (110), and detect multiple partial areas for both eyes, nose, mouth, and facial contour from the recognized face area. Here, the detection unit (120) extracts multiple feature points for the face area from a single image received from the communication unit (110) with lower extraction complexity than the first feature extraction model that extracts multiple feature points for the entire face area of ​​the extraction unit (140) described later, and the second feature extraction model that extracts multiple feature points for both eyes and the surrounding area of ​​both eyes, and determines the position and degree of inclination of the face shape within the single image through the relative position difference of the extracted multiple feature points, and according to the determination result, the face area within the single image can be corrected to face the front from the central position, and the position and degree of inclination of the face shape within the single image can also be determined through the absolute position or relative position difference of each landmark corresponding to the eyes, nose, mouth, forehead, and facial outline (= jawline) within the face area.

[0033] The generation unit (130) can generate a first image for the entire face region and a second image for both eyes and the surrounding areas of both eyes from a single image through a plurality of partial regions detected by the detection unit. Here, the generation unit (130) can generate a first image in which the sizes of the x-axis and y-axis are the same for the entire face region so as to have a resolution corresponding to an input value of a deep learning-based first feature extraction model that extracts a plurality of feature points from the entire face region in a frontal face image of the target corrected by the detection unit (120), and can generate a second image in which the sizes of the x-axis and y-axis are the same for the entire face region so as to have a resolution corresponding to an input value of a deep learning-based second feature extraction model that extracts a plurality of feature points from both eyes and the surrounding areas of both eyes in a frontal face image of the target corrected. At this time, since the area formed by the facial region within a single image is larger than the area formed by both eyes and the surrounding areas of both eyes, the first image for the entire facial region has a higher resolution than the second image for both eyes and the surrounding areas of both eyes, and the first image can be normalized to have a resolution such as 112 X 112, 224 X 224, 448 X 448, etc., and the second image can have a resolution such as 32 X 32, 64 X 64, etc. In addition, the second image can include individual images for the right eye and the surrounding area of ​​the right eye, and the left eye and the surrounding area of ​​the left eye, respectively.

[0034] The extraction unit (140) can generate first feature data in the form of vector values ​​by extracting a plurality of landmarks for both eyes, a nose, a mouth, and a facial contour from the first image generated by the generation unit (130) using a first feature extraction model based on deep learning that extracts a plurality of landmarks for the entire facial region, and can generate second feature data in the form of vector values ​​by extracting a plurality of landmarks for both eyes and the surrounding areas of both eyes from the second image using a second feature extraction model based on deep learning that extracts a plurality of landmarks for both eyes and the surrounding areas of both eyes. Here, the extraction unit (140) can generate the first and second feature data by extracting the number of unit vectors for the plurality of landmarks for both eyes, the nose, the mouth, and the facial contour from the first image so that the number of unit vectors for the plurality of landmarks for both eyes and the surrounding areas of both eyes is greater than the number of unit vectors for the plurality of landmarks for both eyes and the surrounding areas of both eyes from the second image, similar to the image generation resolution of the generation unit (130).

[0035]

[0036] The connection unit (150) can generate third feature data in the form of a vector value by simply concatenating each vector value of the first and second feature data individually extracted from the extraction unit (140) only when it is confirmed through the communication unit that the user terminal (200) has requested identity registration. At this time, the connection unit (150) obtains the user ID of the subject who has requested identity registration through the user terminal (200) from the communication unit (110), matches the obtained user ID with the third feature data generated from a single image including the front of the face of the subject, and stores the matched result in the storage unit (160) to be described later.

[0037] The storage unit (160) may be provided in the form of a storage medium in which third characteristic data matched with each user ID is stored from the connection unit (150). At this time, the storage unit (160) may store the third characteristic data corresponding to each user ID in a folder corresponding to each user ID, and when the connection unit (150) obtains the user ID of the subject who has requested identity registration from the communication unit (110), the storage unit (160) creates a unique folder corresponding to the user ID and places the third characteristic data generated from a single image including the front of the face of the subject in the created folder, so that the user ID and the third characteristic data generated from the single image including the front of the face of the subject can be stored in a matched form.

[0038] In other words, the identity authentication device (100) according to a preferred embodiment of the present invention extracts feature data in the form of vector values ​​for the entire facial region and both eyes and the surrounding areas of both eyes from a single image including the facial region of the identity registration request target obtained from a user terminal (200) requesting identity registration, through different feature extraction models, and stores the extracted feature data for the entire facial region, both eyes and the surrounding areas of both eyes in a simply connected form in the storage unit (160).

[0039] Hereinafter, with reference to the attached drawing, the configuration of the authentication unit (170) that performs identity authentication using the third characteristic data stored in the storage unit (160) and a single image received from the user terminal (200) when identity authentication is requested from the user terminal (200) will be described in more detail.

[0040]

[0041] FIG. 3 is a visual representation of the operation of each component in an identity authentication process through an identity authentication device that simultaneously recognizes multiple features in a facial region according to a preferred embodiment of the present invention, and it should be understood that the abbreviated operation of the detection unit (120), generation unit (130), and extraction unit (140) illustrated in FIG. 3 is performed in the same manner as the operation of each component described above with reference to FIG. 2.

[0042]

[0043] Referring to FIGS. 1 to 3, when the authentication unit (170) receives a single image including the front face of the received subject only when it is confirmed that the user terminal (200) has requested identity authentication through the communication unit (110), the authentication unit (170) can perform facial authentication of the subject requested from the user terminal (200) by checking whether, among the plurality of third feature data pre-stored in the storage unit (160), there exists third feature data whose similarity with the first and second feature data generated through the detection unit (120), the generation unit (130), and the extraction unit (140) exceeds a preset threshold for the single image received from the user terminal (200) requesting identity authentication. Here, the authentication unit (170) can obtain the user ID of the user terminal (200) requesting identity authentication through the communication unit (110) and load the third feature data matched with the user ID stored in the storage unit (160). And, the authentication unit (170) can calculate the similarity for each vector value of the first and second feature data extracted through the extraction unit (140) and the third feature data loaded from the storage unit (160), and in the case of calculating the similarity for each vector value, it can be done through any one of the calculation methods of the cosine distance (or cosine similarity, Cosine distance), Euclidean distance, Manhattan distance, Mahalanobis distance, and histogram intersection, and the calculated result value can be expressed as a value between 0 and 1.Thereafter, the authentication unit (180) multiplies the similarity value of the third feature data and the first feature data by the feature extraction reliability value (a value between 0 and 1) of the first feature extraction model from which the first feature data is extracted, and multiplies the similarity value of the third feature data and the second feature data by the feature extraction reliability value (a value between 0 and 1) of the second feature extraction model from which the second feature data is extracted, and if the authentication result value exceeds a preset threshold, a notification message can be transmitted to the user terminal (200) through the communication unit (110) to notify that the identity authentication has been successful. Conversely, if the authentication result value is lower than the preset threshold, a notification message can be transmitted to the user terminal (200) to notify that the identity authentication has failed.

[0044] In other words, when identity authentication is requested from a user terminal (200), the identity authentication device (100) according to a preferred embodiment of the present invention extracts feature data in the form of vector values ​​for the entire face area and both eyes and the surrounding areas of both eyes from a single image including the face area of ​​the identity authentication request target from the user terminal (200) requesting identity authentication through different feature extraction models, and then calculates a similarity value with the simply connected feature data stored in the storage unit (160), and performs identity authentication by determining whether the weighted sum of the reliability values ​​of each feature extraction model that extracted the corresponding feature data for each calculated similarity value exceeds a preset threshold.

[0045] Accordingly, the identity authentication device (100) according to a preferred embodiment of the present invention simultaneously performs feature comparison for not only the entire face area of ​​the identity authentication request target from the user terminal (200) but also both eyes and the surrounding areas of both eyes, so that even if an unclear image is obtained of a part of the entire face area or a part of the entire eye area, the degree of similarity can be determined through the similarity for each of the face and eyes and the reliability value of each feature extraction model, thereby ensuring high accuracy and reliability for identity authentication.

[0046]

[0047] That is, the identity authentication device through simultaneous recognition of multiple features in a facial region according to the present invention simply connects each feature data in the form of vector values ​​individually extracted from multiple regions through each feature extraction model in a single image acquired from a user terminal and stores it as a single feature data, and at the time of identity authentication, authenticates whether the weighted sum of the similarity value between each feature data in the form of vector values ​​individually extracted from multiple regions through each feature extraction model in a single image acquired from the user terminal and the previously stored simply connected feature data and the reliability value of the feature extraction model that extracted each feature data exceeds a preset threshold. Therefore, even if an image in which the entire face region or part of the eye region is unclear is acquired during the identity authentication process, identity authentication is performed based on the result of summing the similarities for each of the face and the eyes, thereby enabling identity authentication with high accuracy and reliability regardless of a situation in which feature extraction is impossible for a specific region, such as when the identity authentication request subject is wearing a mask.

[0048]

[0049] The above-described embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following patent claims.

Claims

1. A communication unit that confirms a request for either identity registration or identity authentication from a user terminal and receives a single image including the front face of the target from the user terminal according to the request; A detection unit that recognizes a face region from a single image received from the above communication unit and detects multiple partial regions for both eyes, a nose, a mouth, and a facial contour from the recognized face region; A generation unit that generates a first image for the entire face area and a second image for both eyes and the surrounding area of ​​both eyes from a single image through a plurality of partial areas detected from the above detection unit; An extraction unit that generates first feature data in the form of a vector value by extracting a plurality of landmarks for both eyes, a nose, a mouth, and a facial contour from a first image generated from the generation unit using different deep learning-based feature extraction models, and generates second feature data in the form of a vector value by extracting a plurality of landmarks for both eyes and the surrounding area of ​​both eyes from a second image; A connection unit that generates third feature data in the form of a vector value by simply concatenating each vector value of the first and second feature data individually extracted from the extraction unit only when it is confirmed that the user terminal has requested identity registration through the communication unit; and An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized by including a storage unit in which third feature data generated from the above-mentioned connection unit is stored.

2. In paragraph 1, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that the detection unit extracts multiple feature points from a single image received from the communication unit with an extraction complexity lower than the extraction complexity of extracting multiple feature points from the first and second images in the extraction unit to recognize a facial region, and performs detection of multiple partial regions for both eyes, a nose, a mouth, and a facial contour through the extracted multiple feature points.

3. In paragraph 2, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that the detection unit detects tilt and rotation of a facial region through a relative position difference for a plurality of detected partial regions, corrects the tilt and rotation of the detected facial region, and generates a frontal facial image of the subject from a single image.

4. In paragraph 3, An identity authentication device through simultaneous recognition of multiple features within a facial region, characterized in that the above generation unit generates first and second images with different resolutions for the entire facial region, both eyes, and the surrounding areas of both eyes according to the input values ​​of each deep learning-based feature extraction model for the frontal facial image of the target corrected by the detection unit.

5. In paragraph 4, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that the extraction unit extracts from the first image a number of unit vectors for a plurality of feature points for both eyes, a nose, a mouth, and a facial contour so that the number is greater than the number of unit vectors for a plurality of feature points for both eyes and the surrounding area of ​​both eyes from the second image, thereby generating first and second feature data.

6. In paragraph 1, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that it further includes an authentication unit that performs facial authentication of a subject requested from the user terminal by determining, when a single image including the front face of the received subject is received, whether, among the plurality of third feature data pre-stored in the storage unit, third feature data exists whose similarity with the first and second feature data generated through the detection unit, generation unit, and extraction unit exceeds a preset threshold value for the single image received from the user terminal requesting identity authentication.

7. In paragraph 6, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that the authentication unit performs identity authentication on a single image including the front of the face of a subject requested from the user terminal by determining whether there exists third feature data whose sum result of applying a weight according to the reliability of a feature extraction model that extracted the first feature data to a similarity value with the first feature data among a plurality of third feature data stored in the storage unit and applying a weight according to the reliability of a feature extraction model that extracted the second feature data to a similarity value with the second feature data exceeds a preset threshold.

8. In paragraph 1, An identity authentication device through simultaneous recognition of multiple features in a facial region, characterized in that the connection unit obtains a user ID of a subject who has requested identity registration through the user terminal from the communication unit, matches the obtained user ID with third feature data generated from a single image including the front face of the subject, and stores the result in the storage unit.

Images