Secure identity verification method using face recognition

Abstract

A secure identity verification method according to the present invention comprises: generating a secure work area when acquisition of a face image and an identification card image from an edge terminal is detected; extracting the face image and the identification card image from encrypted data received from the edge terminal in the generated secure work area; checking the degree of matching between feature data in a face area of the extracted face image and feature data in a face area of a face photograph included in the identification card image; sealing set data including the feature data for the face image and the identification card image confirmed to match and character string data in the identification card with a sealing key and transmitting same to storage; and when identity authentication is requested from a specific edge terminal, receiving the sealed set data corresponding to the face image received from the edge terminal from the storage, unsealing the set data, and comparing feature data extracted from the face image with the unsealed set data to perform identity verification.

Description

Secure identity verification using facial recognition

[0001] The present invention relates to a technology for secure identification through facial recognition.

[0002]

[0003] Due to the development of communication technology, data processing technology, and sensor technology, the existing target identification technology, in which verification personnel checked documents about the target or directly checked the target's appearance, has evolved into a technology that identifies the target by authenticating the target's biometric information (fingerprints, voice print, iris, face, finger vein, spleen vein, etc.).

[0004] This target identification technology is applied to various service fields and industrial fields that require target matching confirmation, target authority confirmation, etc. through target biometric authentication, and this biometric authentication technology has been disclosed in Republic of Korea Patent Publication No. 10-2008-0047925 (application date: 2006.11. 27., publication date: 2008.05. 30., hereinafter referred to as “prior art”), and the present invention is presented as a technology related thereto that improves and supplements the prior art.

[0005] Meanwhile, biometric information is important information that can identify a subject to the extent that it can identify an individual. If such biometric information is leaked to the outside, the theft of the leaked biometric information can result in economic loss to the individual, and furthermore, there is a problem that it can pose a threat to national security.

[0006] However, the conventional technology authenticates biometric information by comparing previously stored biometric information with newly recognized biometric information, and transmits and stores the recognized biometric information image to the network through a designated compression algorithm. However, the security of the previously stored biometric information, which is the most important and standard for biometric information authentication, is not provided, and if the biometric information image is leaked to the outside during the authentication and storage process of biometric information, the biometric information can be stolen without authorization. In addition, since it only confirms whether the target is correct through biometric information recognition of an already known target, and it is impossible to confirm the identity of an unrecognized target, when applied to a system that requires identity confirmation of an unrecognized target, a separate person must directly visually confirm the target and confirm the target's identity with an ID card, etc., which reduces the reliability of identity authentication and causes the target to have to carry an ID card at all times.

[0007]

[0008] The present invention aims to provide a secure identity verification technology through facial recognition that minimizes the possibility of biometric information being stolen due to external leakage and also enables identity verification of an unidentified subject in order to solve the above-described problem.

[0009] In addition, the present invention relates to an authentication system and method utilizing biometric information, and aims to provide a technology for overcoming the limitations of existing biometric information recognition technology and enhancing security.

[0010] In particular, the present invention introduces a compression algorithm for safe protection of stored biometric information and prevention of external leakage in order to resolve security vulnerabilities in the biometric information recognition process, and presents a technology for accurate identification of unrecognized subjects.

[0011]

[0012] A secure identity verification method through facial recognition according to one embodiment of the present invention relates to a secure identity verification method through an authentication server, wherein an application linked to at least one edge terminal for acquiring biometric information is installed, encrypted data including biometric information is received from the edge terminal through the application, and the method performs storage or authentication of the received biometric information, the method comprising: a preparatory step of, when acquisition of a facial image and an ID card image from the edge terminal is detected, generating a hardware-based secure work area accessible only through an application linked to the edge terminal at the time of acquisition detection; a decryption step of loading encrypted data received from the edge terminal through the application linked to the edge terminal into the secure work area created in the preparatory step, and decrypting the loaded encrypted data to extract the facial image and the ID card image; A processing step of extracting feature data including multiple features in a facial region through image processing of a facial image extracted in the above decryption step, extracting feature data including multiple features in a facial region through image processing of a facial photograph included in an ID card image, and extracting string data for text in the ID card image through OCR (Optical Character Recognition); A comparison step of comparing the feature data for the facial image extracted in the above processing step with the feature data for the ID card image to confirm the degree of matching; A storage step of sealing set data including the feature data for the facial image and the feature data and string data for the ID card image with a seal key according to the comparison result of the above comparison step, and transmitting the sealed set data to a storage;And, when the acquisition of a facial image according to an identity authentication request from an edge terminal is confirmed, an authentication step of receiving an encrypted facial image from the edge terminal, receiving a sealed set data corresponding to the facial image from a storage, unsealing it, comparing the feature data extracted from the decrypted facial image with the unsealed set data, and transmitting an identity authentication result according to the degree of matching to the edge terminal is characterized by including;

[0013] Here, the processing step is characterized by including a correction step of removing noise from the facial image and ID card image extracted in the decryption step and setting a data extraction area of ​​the ID card image; an extraction step of extracting a facial photograph and text from the data extraction area of ​​the ID card image set in the correction step; and a generation step of generating feature data including a plurality of features of the facial region from the facial image corrected in the correction step and the facial photograph extracted in the extraction step through image processing.

[0014] In addition, the above application is characterized by sharing the same time information between the edge terminal and the authentication server and generating a TOTP (Time Base One Time Password) code that is valid only for a preset unit of time.

[0015] At this time, the preparation step is characterized by being a step of creating a security work area and, in the created security work area, creating a TOTP code corresponding to the time of acquisition of the face image and ID image of the edge terminal through an application linked to the edge terminal.

[0016] In addition, the decryption step is characterized by being a step of deriving a random key using a value that combines the TOTP code generated by the edge terminal at the time when biometric information is input and the unique identification information of the edge terminal as a seed, encrypting the input biometric information using the random key to generate an encryption key, receiving data encrypted with the biometric information using the generated encryption key from the edge terminal, and loading it into the secure work area generated in the preparation step.

[0017] In addition, the encrypted data received in the above decryption step is characterized in that it is data in which biometric information is encrypted using a symmetric key encryption algorithm through an encryption key generated by the edge terminal, and the header of the encrypted data includes a code string corresponding to the unique identification information of the edge terminal.

[0018] In addition, the decryption step is characterized by being a step of deriving unique identification information of the edge terminal from the header of the encrypted data received and loaded within the secure work area, and generating a decryption key by deriving a random key using as a seed a value obtained by combining the TOTP code generated in the preparation step and the derived identification information.

[0019] Meanwhile, the above preparation step is characterized as a step of creating an enclave area in a Trusted Execution Environment (TEE) in which a central processing unit is configured to execute in an area that encrypts and protects a portion of memory.

[0020] In addition, the storage step is characterized by being a step of sealing biometric information using an encryption key derived from a central processing unit (CPU) in set data including feature data for a facial image and feature data and string data for an ID card image in an enclave area according to the comparison result of the comparison step.

[0021] In addition, the above-described safe identification method using facial recognition is characterized in that it is implemented as a computer-readable recording medium recording a program for executing each step of the safe identification method using facial recognition.

[0022]

[0023] The secure identity verification method through facial recognition according to the present invention is capable of strengthening the security of stored facial images and personal information, which are of the highest importance in facial recognition-based authentication, since the decryption, verification, sealing, and authentication of encrypted facial images and ID images received from edge terminals are performed in a memory area isolated from the outside, and thus it is impossible to confirm the decryption and sealing method of facial images and ID images through an access path other than a designated access, and data related to verified facial images and ID images are stored in a storage after being sealed, so that even if external leakage of data stored in the storage occurs, the sealing can be unsealed only in a device including a central processing unit that performed the sealing of the data.

[0024] In addition, the secure identity verification method through facial recognition according to the present invention performs initial identity verification by comparing an ID image and a facial image, and stores feature data extracted from a facial photo in the facial image and the ID image, so that afterward, only by recognizing the facial image through an edge terminal, the subject can be identified and the identity can be verified through the recognized facial image, the previously stored facial feature data, and the information in the ID card, thereby ensuring a high reliability of identity verification, and since the subject of verification can be identified and the identity can be verified without carrying the ID card, there is an effect of providing high convenience to users of authentication-based services through facial recognition in using the service.

[0025]

[0026] Figure 1 is a flowchart sequentially illustrating each step of a secure identity verification method using facial recognition according to a preferred embodiment of the present invention.

[0027] FIGS. 2 and 3 are flowcharts illustrating a method for safely storing facial information and identification information and a method for safely recognizing facial information in a secure facial recognition authentication system including a single storage unit, in which a method for safely verifying identity through facial recognition according to a preferred embodiment of the present invention is implemented.

[0028] FIGS. 4 and 5 are flowcharts illustrating a method for safely dividing facial information and identification information and a method for safely recognizing facial information in a secure facial recognition authentication system including three repositories, in which a method for safely verifying identity through facial recognition according to a preferred embodiment of the present invention is implemented.

[0029]

[0030] Hereinafter, a preferred embodiment of a secure identity verification method using facial recognition of the present invention will be described in detail with reference to the attached drawings.

[0031] The same reference numerals in each drawing represent the same components. In addition, specific structural and functional descriptions of embodiments of the present invention are merely illustrative for the purpose of explaining embodiments according to the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant technology, and are preferably not interpreted in an ideal or excessively formal sense unless explicitly defined herein.

[0032] Before describing a secure identification method through facial recognition according to a preferred embodiment of the present invention with reference to the attached drawings, the secure identification method through facial recognition according to a preferred embodiment of the present invention is implemented by an authentication server in a secure facial recognition authentication system, which comprises a plurality of edge terminals, each of which has an application installed thereon, which encrypts a facial image or a facial image and an ID card image acquired from a target object through the application and requests storage and authentication of the encrypted data, an authentication server, each of which has an application corresponding to the application installed on each edge terminal, which performs authentication by decrypting the encrypted data received from each edge terminal and comparing it with previously stored data, or sealing the decrypted data so that unsealing is performed only on a designated central processing unit, and a storage in which the sealed data from the authentication server is stored, and each step to be described below is also performed by an operation based on a configuration of hardware and software constituting the authentication server, rather than an independent configuration, and it should be understood that even in the case of the edge terminal, an optical sensor such as a camera is arranged to acquire a facial image through photography or scanning.

[0033]

[0034] FIG. 1 is a flowchart sequentially illustrating each step of a secure identification method through facial recognition according to a preferred embodiment of the present invention, and FIGS. 2 and 3 are flowcharts illustrating a secure identification method through facial recognition according to a preferred embodiment of the present invention, a method for safely storing facial information and identification information in a secure facial recognition authentication system including a single storage, and a facial recognition authentication method.

[0035] Referring to FIGS. 1 to 3, a method for secure identity verification through facial recognition according to a preferred embodiment of the present invention is described in a secure facial recognition authentication system comprising an authentication server (100), an edge terminal (200), and a storage (300). When the authentication server (100) detects acquisition of a facial image by photographing the face of a target object (O) of the edge terminal (200) and acquisition of an ID card image by scanning an ID card (C) of the target object (O), a separate security work area (S) is set, data (e1) of an encrypted facial image and data (e2) of an encrypted ID card image are received from the edge terminal (200), the received encrypted data (e1, e2) are placed in the security work area, and then the data is decrypted on the security work area and whether the identity matches the decrypted facial image and the ID card image is sealed so that the decrypted facial image and the ID card image, which have been confirmed, can be unsealed only through a designated central processing unit, and the sealed data is stored in a secure storage device. After transmitting to the storage (300), when facial recognition authentication is requested from the edge terminal (200) through encrypted data (e1), the authentication server (100) creates a secure work area, loads the encrypted data (e1) on the secure work area to perform decryption, receives and loads sealed data corresponding to the encrypted data (e1) from the storage (300), and compares the facial image for which authentication is requested with the unsealed data, thereby performing authentication for facial recognition, and may include a preparation step (S100), a decryption step (S200), a processing step (S300), a comparison step (S400), a storage step (S500), and an authentication step (S600).

[0036] The preparation step (S100) is a step in which, when acquisition of a facial image of a person (=object, O) from an edge terminal (200) is detected through an application linked to at least one edge terminal (200) that acquires a facial image for entering identity information and an ID card image or a facial image for identity verification, a hardware-based security work area (S) that can be accessed only through an application linked to the edge terminal at the time of acquisition detection is created. Here, the applications installed in the edge terminal (200) and the authentication server (100) are mutually linked, so that the edge terminal (200) and the authentication server (100) share the same time information through the application, and can generate a TOTP (Time Base One Time Password) code that is valid only for a preset unit time, and the edge terminal (200) and the authentication server (100) can generate the same TOTP code in the same time zone. Additionally, the secure work area (S) created in the preparation stage (S100) may be an enclave area in a trusted execution environment (TEE) configured to allow a central processing unit (CPU) to run in an area that encrypts and protects a portion of volatile memory (RAM, Random Access Memory).

[0037] Here, the trusted execution environment technology is a security technology based on the central processing unit, and depending on the type of central processing unit chip, the representative examples include Trust Execution Technology, SGX Software Guard Extensions, Silent Lake supported by Intel-based central processing units, Secure Service Container, Secure Execution supported by IBM-based central processing units, TrustZone supported by ARM-based central processing units, Secure Encrypted Virtualization and the Secure Nested Paging extension supported by AMD-based central processing units, and MultiZone, Keystone Customizable TEE Framework, and Penglai Scalable TEE for RISC-V supported by RICS-V-based central processing units.

[0038] In other words, in the preparation stage (S100), the authentication server (100) detects the acquisition of the face image and ID image or the face image alone of the edge terminal (200) through an application interoperable with the application installed on the edge terminal (200), creates an enclave area accessible only through the application, and generates a TOTP code for the time of acquisition of the face image of the edge terminal (200).

[0039] The decryption step (S200) is a step of receiving data (e1) in which a face image is encrypted and data (e2) in which an ID image is encrypted from an edge terminal (200) through an application, loading the data into an enclave area (S) created in the preparation step (S100), generating a decryption key for decrypting the encrypted data (e1, e2) through the TOTP code and the loaded encrypted data (e1, e2) according to the time of acquisition of the face image and ID image of the edge terminal (200) in the enclave area (S) created in the preparation step (S100), decrypting the encrypted data (e1, e2) through the generated decryption key, and extracting the face image and ID image.

[0040] At this time, the encrypted data (e1, e2) loaded in the decryption step is data (e1, e2) in which the face image and the ID card image are encrypted using a symmetric key encryption algorithm through an encryption key generated by the edge terminal (200), and the header of the encrypted data (e1, e2) includes a code string corresponding to the unique identification information of the edge terminal (200). More specifically, the edge terminal (200) generates a TOTP code at the time of acquiring the face image and the ID card image through an application installed in the edge terminal (200), and encrypts the face image and the ID card image using an encryption key generated through random number key derivation using a value that combines the generated TOTP code and the unique identification information of the edge terminal (200) as a seed, and the unique identification information of the edge terminal (200) (e.g., device ID, user ID, MAC address value, etc.) is entered in the header area of ​​the data in the form of a code string that is directly or converted. It may be prepared data.

[0041] Meanwhile, in the decryption step (S200), after decryption of the encrypted data (e1, e2), additional verification can be performed by checking the file structure and data integrity of the extracted facial image and ID image to confirm whether the facial image and ID image have been decrypted into information in the correct format or whether any modification has occurred at the time of acquisition from the edge terminal (200).

[0042] That is, in the decryption step (S200), an application is installed that is interconnected with the edge terminal (200) and the authentication server (100) while sharing time information so that the same TOTP code for the same time zone is generated in the preparation step (S100), and since the header of the encrypted data (e1, e2) includes a code string corresponding to the unique identification information of the edge terminal (200), the authentication server (100) can extract the facial image and ID image from the encrypted data (e1, e2) loaded in the enclave area (S) by generating a decryption key without a request to receive additional information according to a predefined encryption / decryption key generation method.

[0043] The processing step (S300) is a step of extracting feature data including multiple features in the facial region through image processing of the facial image extracted in the decryption step (S200), extracting feature data including multiple features in the facial region through image processing of the facial photo included in the ID card image, and extracting string data for text in the ID card image through OCR (Optical Character Recognition), and may include a correction step (S310), an extraction step (S320), and a generation step (S330).

[0044] The correction step (S310) is a step for removing noise from the facial image and ID card image extracted in the decryption step (S200) and setting the data extraction area of ​​the ID card image. Referring to FIG. 2, the authentication server (100) stores a form set (Form set) according to the form of each ID card, in which information on the area where the text / facial photo of each ID card is displayed and pattern images for patterns such as holograms and UV inks that are applied differently depending on the type of ID card are matched and stored according to the type of ID card, in response to various types of ID cards. In the correction step (S310), first, the type of ID card is confirmed through OCR on the pre-designated area on the ID card image extracted in the decryption step (S200). For example, in the case of a resident registration card, the words "Resident Registration Card" are displayed in the upper left area of ​​the ID card (C), and in the case of a driver's license, the words "Driver's License" are displayed in the upper right area of ​​the ID card (C). Therefore, in the correction step (S310), prior to OCR reading of the entire area of ​​the ID card image, partial OCR reading is performed on a preset area, thereby making it possible to determine in a short time what type of ID card (C) the ID card image corresponds to. Afterwards, an ID card form corresponding to the ID card (C) is extracted from a pre-stored form set, and the OCR reading target area and the facial and limb areas within the ID card image are confirmed. In addition, in this step, in order to prevent the problem of text or facial photo being unclearly identified on the ID card image due to the pattern being scanned while being reflected, an ID card image including a high-definition facial photo and text can be obtained through correction such as noise removal and image interpolation using a pattern image matched to the previously identified ID card type among the form sets, and the facial image can also be corrected by minimizing noise in the image depending on the shooting environment through similar image processing.

[0045] Meanwhile, in the correction step (S310), it is possible to additionally check whether the identification card (C) has been altered or forged by checking whether the pattern image confirmed from the form set matches the pattern of the identification card image, and at this time, if the identification card image is confirmed to have been altered or forged in the correction step (S310), instead of proceeding to the extraction step (S320) described below, a notification message can be generated and transmitted to the edge terminal (200) and the public institution or certification institution server (400) to notify the fact of use of an altered or forged identification card.

[0046] The extraction step (S320) is a step for extracting facial photos and text from the data extraction area of ​​the ID card image set in the correction step (S310). In this step, the facial photos and text can be separately extracted from the ID card image through image capture and OCR reading of the facial photos and text extraction area for each ID card type confirmed in the previous correction step (S310).

[0047] The generation step (S330) is a step for generating feature data (f1, f2) including multiple features of the facial region from the facial image corrected in the correction step (S310) and the facial photograph extracted in the extraction step (S320) through image processing. In this step, the facial region is distinguished through differences in line components, brightness, and luminance within the facial image and facial photograph, and feature data (f1, f2) is extracted from multiple feature points, which are landmarks, for the positions and shapes of eyes, nose, mouth, eyebrows, cheekbones, chin, etc. on the distinguished facial region from each of the facial image and facial limbs. If the facial region is created in a tilted shape as a result of the landmark extraction, an additional normalization task may be performed to rotate each landmark and arrange it in a form facing the front.

[0048] The above-described processing step (S300) can be performed on the enclave area (S) generated in the preparation step (S100), as in the preceding decryption step (S200), and independently processes the image of the face of the target object (O) and the facial photo of the ID card image, thereby generating individual feature data (f1, f2) from each, so that in the comparison step (S400) described later, through comparison between the individually generated feature data (f1, f2), it is possible to confirm whether the ID card (C) is the ID card of the corresponding object (O).

[0049] The comparison step (S400) is a step for comparing the feature data (f1) for the facial image generated in the generation step (S330) with the feature data (f2) for the ID card image to confirm the degree of matching. This step generates a deformation by rotating the landmarks in each feature data (f1, f2) in the same 3D space or increasing or decreasing them in the x-axis, y-axis, and z-axis directions, and then confirms whether the average value of the matching rate among the deformations in the virtual 3D space for the landmarks in each feature data (f1, f2) is greater than a reference value, thereby confirming whether the ID card (C) is the ID card of the corresponding object (O). If the average value of the matching rate among the deformations in the virtual 3D space for the landmarks in each feature data (f1, f2) is confirmed to be lower than the reference value or lower than the minimum reference value specified in advance, the entire process can be terminated by sending a notification message to the edge terminal (200) notifying that the facial image and the facial photo in the ID card image do not match, without proceeding to the subsequent storage step (S500), and in some cases, the fact can be additionally notified to a public institution or certification authority server (400).

[0050] The storage step (S500) is a step of sealing the set data (s) that groups the feature data (f1) for the facial image, the feature data (f1) for the ID card image, and the string data, with a seal key (Seal Key) and transmitting the sealed biometric information to the storage (300) only when it is confirmed in the comparison step (S400) that the average value of the matching rate among the deformations in the virtual 3D space for the landmarks in each feature data (f1, f2) is greater than or equal to the reference value. The sealing in this step is performed using the seal key, which is an encryption key derived from the central processing unit (CPU) of the authentication server (100), and after performing the sealing in the enclave area (S) so that only the application described above and the device on which the central processing unit is installed can unseal the sealed feature data (s), the sealing is then transmitted to the storage (300).

[0051] In addition, in the storage step (S500), for the purpose of confirming from which edge terminal (200) the sealed set data (s) was received, a data table matching the sealed set data (s), the unique identification information of the edge terminal (200), and the information on the storage target storage (300) of the sealed set data (s) may be additionally created.

[0052] Referring to FIG. 3, the authentication step (S600) may be initiated by, as in the preparation step (S100), when acquisition of a facial image in response to an identity authentication request from the edge terminal (200) is confirmed, the authentication server (100) creates an enclave area (S) and generates a TOTP code for the time of facial image acquisition on the enclave area (S). In addition, the authentication step (S600) receives an encrypted facial image from the edge terminal (200), loads it into the enclave area (S), generates a decryption key through the generated TOTP code and the code string of the data header of the loaded encrypted facial image, and decrypts the encrypted facial image, while requesting the reception of sealed set data (s) corresponding to the encrypted facial image from the storage (300) through the code string of the data header of the encrypted facial image. And, in the authentication step (S600), feature data is extracted and generated from the decrypted facial image, as in the correction step (S310) and the generation step (S330), and the sealed set data (s) corresponding to the facial image received from the storage (300) is loaded into the enclave area (S) to unseal the sealed set data (s). Thereafter, this is a step in which identity authentication is performed for the facial image acquired from the edge terminal (200) by determining whether the average value of the matching rate among the deformations in the virtual three-dimensional space for the landmarks in the feature data in the unsealed set data (s) generated from the facial image is greater than or equal to a reference value.

[0053] At this time, in the authentication step (S600), if the difference between the ID card issuance date and the current date in the string data included in the set data (s) is confirmed to be above a certain level, and if the average value of the matching rate among the deformations in the virtual 3D space for each landmark in the feature data generated from the facial image and the feature data included in the set data (s) is higher than the minimum value but lower than the reference value, the authentication step (S600) may be performed on the facial image acquired from the edge terminal (200) by checking whether the average value of the matching rate among the deformations in the virtual 3D space for the landmarks in the feature data generated from the facial image and the feature data included in the unsealed set data (s) is higher than the corrected correction value through a correction value that is lowered in proportion to the difference between the ID card issuance date and the current date based on the reference value, and the identity authentication result for the above-described facial image is transmitted to the edge terminal (S600).

[0054]

[0055] Meanwhile, the storage step (S500) and the authentication step (S600) in the secure identity verification method through facial recognition via the authentication server (100) described above are steps for performing authentication by comparing the facial image received and loaded from the edge terminal (200) with the set data stored in a sealed form in the storage (300) after sealing and storing the set data including the feature data and the string data generated through the facial image and the ID image acquired from the target object (O) in the secure facial recognition authentication system including a single storage (300), and may also be steps for dividing and storing the set data including the feature data and the string data generated through the facial image and the ID image acquired from the target object (O) in the secure facial recognition authentication system including a plurality of storages according to the number of storages, or performing authentication through the dividedly stored set data. Hereinafter, with reference to the attached drawings, examples of divided storage of set data and facial recognition authentication in the secure facial recognition authentication system including three storages will be described. Let me explain.

[0056]

[0057] FIGS. 4 and 5 are flowcharts illustrating a method for safely dividing facial information and identification information and a method for safely recognizing facial information in a secure facial recognition authentication system including three repositories, in which a method for safely verifying identity through facial recognition according to a preferred embodiment of the present invention is implemented.

[0058] Referring to FIGS. 4 and 5, in the secure facial recognition authentication system including three repositories, the storage step (S500) of the secure identity verification method through facial recognition of the authentication server (100) is, only when it is confirmed in the comparison step (S400) that the average value of the matching rate among the deformations in the landmark 3D space within each feature data (f1, f2) is greater than or equal to the reference value, the feature data (f1) for the facial image, the feature data (f1) for the ID card image, and the set data (s) grouped by the string data are divided into three data areas according to the number of the first, second, and third storage servers (310, 320, 330) connected by a high-speed network, and after the divided set data is replicated according to the number of the storage servers (310, 320, 330), the first divided set data in which data in the first data area is missing, the second divided set data in which data in the second data area is missing, and the data in the third data area are divided into three data areas according to the number of the first, second, and third storage servers (310, 320, 330). This is a step of creating missing third partition set data, sealing each partition set data with a sealing key, individually matching the sealed three partition set data (s1, s2, s3) to each target storage server (310, 320, 330), creating a data table for the matching result, and transmitting each sealed partition set data (s1, s2, s3) to the matched storage server (310, 320, 330) based on a data cable.

[0059] In addition, in the authentication step (S600) of the secure identity verification method through facial recognition of the authentication server (100) in the secure facial recognition authentication system including three repositories, when a facial image is acquired from the edge terminal (200) for the purpose of identity verification, the authentication server (100) detects the acquisition of the facial image of the edge terminal (200) and creates an enclave area (S). Afterwards, when the edge terminal (200) encrypts the facial image acquired for facial recognition authentication, includes unique identification information in the header of the encrypted data (e), and transmits it to the authentication server (100), the authentication server (100) loads the received encrypted data (e1) into the enclave area (S), checks the communication status with the first, second, and third storage servers (310, 320, 330), and requests the transmission of sealed segmented set data (s1, s2, s3) corresponding to the code string included in the header value of the encrypted data (e1) loaded into the enclave area (S) from the first, second, and third storage servers (310, 320, 330), and then transmits the TOTP code generated corresponding to the time of acquiring the facial image of the edge terminal (200) in the enclave area (S) and unique identification information derived from the header value of the encrypted data (e1). A decryption key is generated through the generated decryption key, and a facial image is extracted and verified from the encrypted data (e1) loaded into the enclave area (S) through the generated decryption key, and feature data is extracted and generated through image processing of the extracted facial image.At this time, as shown in FIG. 5, if the sealed divided set data (s1) is not received from the first storage server (310) due to reasons such as a communication failure with the first storage server (310) or damage to the storage area of ​​the first storage server (310), only two sealed divided set data (s2, s3) are received from the second and third storage servers (320, 330), the two sealed divided set data (s2, s3) are loaded onto the enclave area (S), and the sealed divided set data (s2, s3) are unsealed and merged using the sealing key to restore the set data. Afterwards, by checking whether the matching rate among the deformations in the virtual three-dimensional space for each landmark in the feature data included in the set data obtained by unsealing and merging the two sealed split set data (s2, s3) with the landmark in the feature data generated from the facial image extracted from the enclave area (S) is greater than or equal to a reference value, identity authentication is performed for the facial image acquired from the edge terminal (200), and the authentication result is transmitted to the edge terminal (200) through the application, thereby performing secure identity authentication through facial recognition.

[0060]

[0061] That is, since the secure identity verification method through facial recognition according to the present invention decrypts, verifies, seals, and authenticates encrypted facial images and ID images received from an edge terminal in a memory area isolated from the outside, it is impossible to confirm the decryption and sealing method of facial images and ID images through an access path other than a designated access, and data related to facial images and ID images that have been verified are stored in a storage after being sealed, so that even if external leakage of data stored in the storage occurs, the sealing can be unsealed only in a device including a central processing unit that performed the sealing of the data, thereby having the effect of strengthening the security of stored facial images and personal information, which are the most important in facial recognition-based authentication.

[0062] In addition, the secure identity verification method using facial recognition according to the present invention performs initial identity verification by comparing an ID image and a facial image, and stores feature data extracted from a facial photo in the facial image and the ID image, so that afterward, only by recognizing the facial image through an edge terminal, the subject can be identified and the identity can be verified using the recognized facial image, the previously stored facial feature data, and the information in the ID card, thereby ensuring a high reliability of identity verification, and since the subject of verification can be identified and the identity can be verified without carrying the ID card, there is an effect of providing high convenience to users of authentication-based services through facial recognition in using the service.

[0063]

[0064] The above-described embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following patent claims.

Claims

1. A method for secure identity verification through an authentication server in which an application linked to at least one edge terminal that acquires biometric information is installed, encrypted data including biometric information is received from the edge terminal through the application, and storage or authentication of the received biometric information is performed. When acquisition of a facial image or ID image from an edge terminal is detected, a preparatory step is performed to create a hardware-based security work area that can only be accessed through an application linked to the edge terminal at the time of acquisition detection; A decryption step of loading encrypted data received from an edge terminal through an application linked to the edge terminal into the security work area created in the above preparation step, decrypting the loaded encrypted data, and extracting a facial image and an ID card image; A processing step of extracting feature data including multiple features in a facial region through image processing of a facial image extracted in the above decryption step, and extracting feature data including multiple features in a facial region through image processing of a facial photo included in an ID card image, and string data for text in the ID card image through OCR (Optical Character Recognition); A comparison step for comparing the feature data for the facial image extracted in the above processing step with the feature data for the ID card image to confirm the degree of matching; A storage step of sealing the set data including the feature data for the facial image and the feature data and string data for the ID card image with a seal key according to the comparison result of the above comparison step, and transmitting the sealed set data to a storage; and A secure identity verification method through facial recognition, characterized by including an authentication step of, when acquisition of a facial image in response to an identity verification request from an edge terminal is confirmed, receiving an encrypted facial image from the edge terminal, receiving a sealed set of data corresponding to the facial image from a storage, unsealing the sealed set of data, comparing feature data extracted from the decrypted facial image with the unsealed set of data, and transmitting an identity verification result according to the degree of matching to the edge terminal.

2. In paragraph 1, The above processing steps are: A correction step for removing noise from the facial image and ID card image extracted in the above decryption step and setting a data extraction area of ​​the ID card image; An extraction step for extracting a facial photo and text from the data extraction area of ​​the ID card image set in the above correction step; and A method for secure identification through facial recognition, characterized by comprising a generation step of generating feature data including a plurality of features of a facial region from a facial image corrected in the above-mentioned correction step and a facial photograph extracted in the above-mentioned extraction step through image processing.

3. In paragraph 1, The above application is a secure identity verification method using facial recognition, characterized in that it shares the same time information between the edge terminal and the authentication server and generates a TOTP (Time Base One Time Password) code that is valid only for a preset unit of time.

4. In paragraph 3, A method for secure identity verification through facial recognition, characterized in that the above preparation step is a step of creating a security work area and, in the created security work area, creating a TOTP code corresponding to the time of acquiring the face image and ID image of the edge terminal through an application linked to the edge terminal.

5. In paragraph 4, A secure identity verification method through facial recognition, characterized in that the above decryption step is a step of deriving a random key using a value that combines the TOTP code generated by the edge terminal at the time when biometric information is input and the unique identification information of the edge terminal as a seed, encrypting the input biometric information using the random key to generate an encryption key, receiving data encrypted with the biometric information using the generated encryption key from the edge terminal, and loading it into the secure work area generated in the preparation step.

6. In paragraph 5, A secure identity verification method using facial recognition, characterized in that the encrypted data received in the above decryption step is data in which biometric information is encrypted using a symmetric key encryption algorithm using an encryption key generated by the edge terminal, and the header of the encrypted data includes a code string corresponding to the unique identification information of the edge terminal.

7. In paragraph 6, A secure identity verification method using facial recognition, characterized in that the above decryption step is a step of deriving unique identification information of an edge terminal from the header of encrypted data received and loaded within the secure work area, and generating a decryption key through random key derivation using a value obtained by combining the TOTP code generated in the preparation step and the derived identification information as a seed.

8. In paragraph 1, A method for secure identity verification through facial recognition, characterized in that the above preparation step is a step of creating an enclave area in a trusted execution environment (TEE) configured to allow a central processing unit to run in an area that encrypts and protects a portion of memory.

9. In paragraph 8, A secure identity verification method using facial recognition, characterized in that the above storage step is a step of sealing biometric information using an encryption key derived from a central processing unit (CPU) in set data including feature data for a facial image and feature data and string data for an ID card image in an enclave area according to the comparison result of the above comparison step.

10. A computer-readable recording medium recording a program for executing each step of the secure identity verification method using facial recognition according to any one of clauses 1 to 9.

Images