Apparatus and method for performing user authentication on basis of face recognition

Abstract

According to an embodiment of the present invention, a method for performing face recognition-based user authentication, the method being performed by a user terminal, may comprise the steps of: (a) capturing an image through a camera and recognizing a face of a user in the captured image; (b) providing a direction identifier at an random position in the image in consideration of the position of the recognized face in the image; (c) analyzing the captured image received in real time to determine whether a motion of the recognized face matches a command corresponding to the direction identifier; and (d) when it is determined that the motion of the recognized face corresponds to the command, changing the direction identifier, displaying same, and completing authentication, wherein at least one of the direction indicated by the direction identifier and the position at which the direction identifier is provided is randomly provided whenever authentication is performed.

Description

Device and method for performing user authentication based on facial recognition

[0001] The present invention relates to a technology for performing user authentication based on facial recognition technology, and specifically to a system for performing user authentication that can prevent forgery or fraudulent acts while enhancing the convenience of user authentication.

[0002] Today, real-name verification has established itself as an essential procedure in financial transactions. Particularly due to the proliferation of non-face-to-face transactions, the importance of technology that verifies the match between an identification document and the actual user is being further emphasized. For various financial transactions, including remittances, there is a requirement to approve transactions after confirming that the user is the same individual by comparing their identification document with a facial image. The necessity of such authentication procedures is becoming even greater in countries where financial real-name systems are inadequate, and international organizations are also recommending the adoption of such systems.

[0003] Anti-spoofing technologies, introduced in addition to existing identity verification processes, are used to prevent forgery or fraud. A representative method is the "liveness check," which verifies the identity of a real person through actions such as blinking or turning the head. However, this approach can cause inconvenience in terms of user experience or carries the risk of authentication errors or failures due to the liveness check. Therefore, finding a balance between security and convenience remains a major challenge.

[0004] To solve these problems, the present invention proposes an authentication system that is more efficient, minimizes user inconvenience, and maintains high security.

[0005] The present invention aims to solve the problems of the aforementioned prior art and to implement a facial identification authentication system that can prevent acts such as facial forgery or fraud while increasing user convenience.

[0006] As a technical means for achieving the aforementioned technical task, a method for performing facial recognition-based user authentication, performed by a user terminal, a system composed of a PC and a camera, and a dedicated terminal, etc., comprises: (a) capturing an image through a camera and recognizing a user's face in the captured image; (b) providing a direction identifier at an arbitrary location within the image, considering the position of the recognized face within the image; (c) analyzing the captured image received in real time to determine whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) if it is determined that the movement of the recognized face corresponds to a command, changing and displaying the direction identifier and completing authentication, wherein at least one of the direction referred to by the direction identifier and the location where the direction identifier is provided may be provided randomly each time authentication is performed.

[0007] In another embodiment, step (a) identifies the pre-set facial features to determine whether a face exists, and determines whether the face is looking straight ahead based on the respective recognized positions of the facial features, and if it is determined that the face is looking straight ahead, authentication may begin.

[0008] In another embodiment, step (b) may display a direction identifier by overlaying it on a second area corresponding to the direction indicated by the direction identifier among the remaining second areas, excluding the first area where the face is recognized, on the display area of ​​the user terminal.

[0009] In another embodiment, if the area of ​​the second region is not located within a preset threshold range, an identifier can be provided indicating that the distance between the camera and the face should be adjusted.

[0010] In another embodiment, step (c) may include: (c-1) calculating the nth coordinate information of a pre-set facial feature from the nth frame of an image (n is a natural number equal to or greater than 1); (c-2) calculating the n+1th coordinate information of a facial feature from the n+1th frame after the nth frame; (c-3) calculating vector information of a facial feature regarding the direction in which the user's face moves based on the nth coordinate information and the n+1th coordinate information; and (c-4) determining the direction of movement of the face in real time based on the vector information, and determining whether it matches a command by comparing the direction of movement of the face with the direction of the direction identifier.

[0011] In another embodiment, the shape or size of the direction identifier can be adjusted according to a preset method based on the change in the size of the vector information per frame of the image.

[0012] In another embodiment, as the magnitude of the absolute value of the vector information regarding the direction of movement of the face increases, the size of the direction identifier may decrease or the display area of ​​the direction identifier may be erased.

[0013] In another embodiment, step (c-4) compares the direction of movement of the face with the direction of the direction identifier in real time, and if a predetermined amount of time elapses while the difference between the direction of movement of the face and the direction is greater than a preset threshold, it can be determined that authentication has failed.

[0014] In another embodiment, step (d) indicates that authentication is completed according to a preset method when it is determined to correspond to a command, and the preset method may include at least one of displaying a specific identifier, removing a direction identifier, providing a vibration function of the user terminal, or providing a preset sound.

[0015] In another embodiment, the command corresponding to the direction identifier may be a command to move the angle of the face to a range such that the eyes can be recognized within the image captured by the user terminal even if the face is moved and authentication is completed.

[0016] In another embodiment, information regarding the direction of movement of a face that can be compared with the direction of a direction identifier is received, and when the direction of movement of the face and the direction of the direction identifier correspond to each other, if the amount of change in width and the amount of change in position of the eye in the image captured by the user terminal exceed a preset threshold when compared with the width and position of the eye in the image when the user starts authentication with the user terminal, it can be determined that authentication is completed.

[0017] In another embodiment, the device comprises: a communication module; at least one processor; and a memory electrically connected to the processor and storing at least one code for performing a method for performing facial recognition-based user authentication, wherein when the memory is executed through the processor, the processor performs the method, the method comprising: (a) capturing an image through a camera and recognizing a user's face in the captured image; (b) providing a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) analyzing the captured image received in real time to determine whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) if it is determined that the movement of the recognized face corresponds to a command, changing and displaying the direction identifier and completing authentication, wherein at least one of the direction referred to by the direction identifier and the location where the direction identifier is provided may be a device that is provided randomly each time authentication is performed.

[0018] In another embodiment, a method for performing facial recognition-based user authentication performed by a server comprises: (a) receiving an image captured through a camera from a user terminal and recognizing the user's face in the captured image; (b) providing a command to the user terminal to provide a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) receiving a captured image received in real time from the user terminal and determining whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) if it is determined that the movement of the recognized face corresponds to a command, providing a command to the user terminal to change and display the direction identifier and completing the authentication, wherein at least one of the direction referred to by the direction identifier and the location where the direction identifier is provided may be provided randomly each time authentication is performed.

[0019] In another embodiment, the method comprises a communication module; at least one processor; and a memory electrically connected to the processor and storing at least one code for performing a method for performing facial recognition-based user authentication, wherein when the memory is executed through the processor, the processor performs the method, the method comprising: (a) receiving an image captured through a camera from a user terminal and recognizing the user's face in the captured image; (b) providing a command to the user terminal to provide a direction identifier at any location within the image, taking into account the position of the recognized face within the image; (c) receiving a captured image received in real time from the user terminal and determining whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) if it is determined that the movement of the recognized face corresponds to the command, providing a command to the user terminal to change and display the direction identifier and completing the authentication, wherein at least one of the direction referred to by the direction identifier and the location where the direction identifier is provided may be a server provided randomly each time authentication is performed.

[0020] The present invention enables the implementation of a facial identification authentication system equipped with a liveness check function to prevent spoofing associated with conventional facial recognition authentication technology.

[0021] Specifically, it becomes possible to implement facial recognition authentication technology that prevents spoofing by identifying the direction of movement of a user's face in real time and determining whether the identified person actually exists by presenting the direction of movement of the face randomly.

[0022] FIGS. 1a to 1c are drawings showing the configuration of a system for performing facial recognition-based user authentication according to an embodiment of the present invention.

[0023] FIG. 2 is a diagram showing the configuration of a user terminal according to an embodiment of the present invention.

[0024] FIG. 3 is a diagram showing the configuration of a server according to one embodiment of the present invention.

[0025] FIG. 4 is a flowchart illustrating a facial recognition-based user authentication process performed by a user terminal according to an embodiment of the present invention.

[0026] FIG. 5 is a flowchart illustrating a facial recognition-based user authentication process performed by a server according to an embodiment of the present invention.

[0027] FIG. 6 is a flowchart illustrating the process of identifying the position of a user's face from an image and providing a direction identifier accordingly, according to an embodiment of the present invention.

[0028] Figure 7 is an example diagram illustrating a user authentication process that is generally performed.

[0029] FIGS. 8a to 8d are example drawings showing whether recognition is possible according to the direction of a user's face, according to an embodiment of the present invention.

[0030] FIG. 9 is a diagram showing an example of an interface for facial recognition-based user authentication according to one embodiment of the present invention.

[0031] FIG. 10 is an example diagram illustrating a facial recognition-based user authentication process in an image following an arrow presented in a random direction according to an embodiment of the present invention.

[0032] Embodiments of the present invention are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0033] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0034] The following examples are detailed descriptions to aid in understanding the present invention and are not intended to limit the scope of the present invention. Accordingly, inventions within the same scope that perform the same function as the present invention will also fall within the scope of the present invention.

[0035] FIGS. 1a to 1c are drawings showing the configuration of a system for performing facial recognition-based user authentication according to an embodiment of the present invention.

[0036] First, referring to FIG. 1a, a system (10) for performing facial recognition-based user authentication may be composed only of a user terminal (100), and may additionally include a server (300) as an optional embodiment. When the system (10) includes a server (300), although not shown in the drawing, each device may be interconnected via a wired or wireless connection through a communication network.

[0037] Additionally, as an additional embodiment, FIG. 1b is a configuration of a system (10) that can be implemented when a facial recognition-based user authentication program is installed on-premise or embedded within the user terminal (200), and FIG. 1c is an example of a configuration of a system (10) that operates as a combination (100) of a PC or laptop and a camera. At this time, although the configuration of the system (10) may differ from FIG. 1a to FIG. 1c, the process of performing facial recognition is the same, so FIG. 1a will be used as an example to explain the following.

[0038] The present invention relates to a technology for performing user authentication by recognizing the user's face through a terminal. Specifically, face recognition-based user authentication is a biometric technology that verifies identity using the face; it performs authentication by determining whether the individuals are the same person based on the degree of match between the facial appearance stored in a database and the facial appearance measured by sensors such as cameras.

[0039] Generally, the face recognition process can follow three steps: face detection via sensors, face extraction and preprocessing from the detected data, and face recognition. Similar to object recognition in computer vision, the face detection process primarily utilizes Convolutional Neural Networks (CNNs). Additionally, the face extraction and preprocessing process involves extracting important landmarks for recognizing facial features, such as eyes, nose, and mouth, from the extracted face images. Based on these extracted landmarks, the angle or brightness of the face images is changed to transform them into a uniform form, after which face recognition is performed.

[0040] Before recognizing faces using deep learning-based synthetic neural networks as another method of face recognition, researchers performed face recognition by extracting engineered features and calculating similarities between them. Representative methods include extracting eigenfaces using Principal Component Analysis (PCA) and comparing them, and extracting and comparing Fisherfaces using Linear Discriminant Analysis (LDA).

[0041] Principal Component Analysis used for eigenface extraction can be a technique that transforms high-dimensional data (e.g., image data) into a low-dimensional feature space and features linear axes that explain the principal variance of the data. The method for calculating eigenfaces from input images using this is as follows [Equation 1]. That is, to perform the calculation of the average face, all training images Average face about It can be calculated as follows.

[0042] [Mathematical Formula 1]:

[0043] In this case, N can be the number of images in the training data.

[0044] In addition, the method for calculating centered image data is that the centered data is obtained by subtracting the average face from each face image. It can be calculated through the following [Mathematical Formula 2].

[0045] [Mathematical Formula 2]:

[0046] In addition, the covariance matrix C can be calculated through the following [Equation 3].

[0047] [Mathematical Formula 3]:

[0048] Also, the eigenvalues ​​of the covariance matrix and eigenvectors It can be calculated through the following [Mathematical Formula 4].

[0049] [Mathematical Formula 4]:

[0050] In this case, the eigenvectors represent the principal axes (Eingenfaces) of the face feature space, and the face image can be projected by projecting the input face image x into the eigenface space.

[0051] LDA, used for Fisher face extraction, is a technique that finds a linear transformation that maximizes class separation by simultaneously considering between-class and within-class variance. In general, Fisher faces can provide more robust performance than eigenfaces against lighting changes or differences in facial expressions. Therefore, the process of calculating Fisher faces from input images using this method can be as follows.

[0052] First, the variance matrix within the class ( The process of producing ) is data belonging to class c For this, the within-class variance matrix can be calculated as shown in [Equation 5] below.

[0053] [Mathematical Formula 5]:

[0054] Here, is the mean vector of class c, and C is the number of classes.

[0055] Also, the within-class variance matrix ( The process of calculating ) is the overall average vector For this, the variance matrix between classes can be calculated as shown in [Equation 6] below.

[0056] [Mathematical Formula 6]:

[0057] Here, can be the number of data in class C.

[0058] In addition, the linear discriminant analysis to calculate the LDA optimization goal can be [Equation 7], which finds the direction vector w that maximizes the following ratio.

[0059] [Mathematical Formula 7]:

[0060] Here, the generalized eigenvalue problem can calculate the optimal direction vector w through [Equation 8].

[0061] [Mathematical Formula 8]:

[0062] Here, can correspond to an eigenvalue.

[0063] Finally, the process of projecting the face image allows the face image to be projected into the LDA transformation space through the following [Equation 9].

[0064] [Mathematical Formula 9]:

[0065] Here, W can be a matrix composed of optimal direction vectors.

[0066] Meanwhile, as spoofing—which deceives facial recognition technology by using images of a person's face printed on paper or similar materials—is prevalent, spoofing can be prevented through a liveness check. For example, as shown in FIG. 7 (a) to (d), when a user (200) attempts facial authentication by operating a user terminal (100), the user terminal (100) can guide the user (200) through an image on what action (e.g., turning the head to the side) should be performed. Accordingly, the user (200) performs the action of turning the head while looking at the camera of the user terminal (100), and the user terminal (100) recognizes the user's (200) face (210) from this action and completes the authentication.

[0067] However, when performing real-face verification technology in the conventional way, the user had to turn their head (200a) completely, as shown in the examples of FIGS. 8a and 8b, in order to achieve high security.

[0068] Generally, to increase security, it is required to turn the face up, down, left, and right a lot, but in this case, it completely goes out of the field of view (200b) of the user looking at the smartphone (or mobile device) or camera, so as shown in Fig. 8c, recognition fails because it completely goes out of the range where authentication is performed on the camera or mobile screen, which can cause very bad effects and inconvenience to the user experience.

[0069] However, when using the method proposed by the present invention, the real face verification technique is performed only within the field of view (200b) as shown in FIG. 8d, and security can be ensured because the head (200a) only needs to be moved once in a random direction. This has the advantage of maintaining security in terms of the number of times the head (200a) is moved and the ability to verify within the field of view (200b), compared to existing methods that require the movement of the head (200a) in four or three directions.

[0070] Meanwhile, there is no significant difference in security between the method of verifying the real face by moving the face (210) in 2 to 4 directions as in conventional technology and the method of verifying the real face only once in a random direction as in the present invention. Specifically, there are various strategies for authenticating the identity of a user (200), such as the KYC procedure, as follows.

[0071] For example, there may be six strategies such as a Something you know strategy that checks what the user (200) knows (e.g., password, PIN number, security question, etc.), a Something you have strategy that checks what the user (200) has (e.g., security card such as OTP, etc.), a Something you are strategy that checks who the user (200) is (e.g., fingerprint, face, iris recognition, etc.), a Something you do strategy that checks the user (200)'s actions (e.g., entering a CAPTCHA, swiping gesture, mouse movement, etc.), and a Context awareness strategy that recognizes dynamic situations such as the user's (200) actions or environment (e.g., device information where the user attempts authentication, time when the user requests authentication, time when the user enters a CAPTCHA, etc.).

[0072] At this time, among the various strategies mentioned above, the process of authenticating the face in 2 to 4 predetermined directions can be considered as a combination of the Something You Are strategy and the Something You Do strategy. On the other hand, since the present invention is a method of moving the face (210) in one direction randomly determined for each authentication attempt, it can be considered as a combination of three strategies: the Something You Are strategy, the Something You Do strategy, and the Context Awareness strategy, thereby enabling high security.

[0073] In addition, the present invention may be able to ensure that there is no significant problem with security even if it does not move beyond the viewing angle (200b) based on the following grounds.

[0074] Specifically, it may be determined that the method implemented in the present invention may have security issues because there are few actions required of the user (200); however, as explained above, since an authentication strategy is added in a way that requires random actions from the user (200), it can have higher security than the conventional technology.

[0075] In addition, even if the face (210) is moved to a degree that does not deviate from the field of view (200b), authentication is completed only when the face (210) remains fixed in that direction for a certain period of time, so the data of the user (200) can be verified on the time axis, and effective verification is possible even with relatively less movement than the existing method.

[0076] Accordingly, the present invention may propose a facial recognition authentication function using a random direction identifier (110) so that a user (200) can easily use the real face verification technology.

[0077] According to one embodiment of the present invention, a user terminal (100) captures the face of a user (200) in real time through a camera module (160), and a direction identifier (110) for indicating where the face (210) of the user (200) should move and the image of the user (200) captured in real time can be provided on a display module (150) of the user terminal (100).

[0078] Additionally, when the face (210) of the user (200) completes movement along the direction identifier (110), the user terminal (100) recognizes this in real time, changes and displays the direction identifier (110), and completes authentication.

[0079] For example, the facial recognition authentication technology of the present invention can be operated through an interface as shown in FIG. 9. Specifically, the image of a user (200) being captured in real time through a camera module (160) can be displayed on the display module (150) of the user terminal (100). At this time, the display area of ​​the user terminal (100) may be composed of a first area (130) where the face (210) is recognized and a second area (120) excluding the first area (130). In addition, the direction identifier (110) may be overlaid and displayed in one area of ​​the second area (120) that corresponds to the direction indicated by the direction identifier (110). That is, the direction identifier (110) may be displayed in the upper right corner of the second area (120) on the display so that the user (200) turns their head toward the upper right direction.

[0080] At this time, the direction identifier (110) may be displayed in the shape of an image symbol, such as an arrow, that allows the user (200) to identify the direction, as shown in the drawing. The direction identifier (110) in the shape of an arrow in the drawing is one embodiment, and various symbols or characters that allow the user (200) to recognize the direction may be used, so the scope of the present invention is not limited.

[0081] Additionally, the user terminal (100) refers to a communication terminal capable of using terminal applications in a wired or wireless communication environment. Although the user terminal (100) is depicted as a smartphone, which is a type of portable terminal, in FIG. 1, the concept of the present invention is not limited thereto and can be adopted without limitation for any terminal capable of carrying terminal applications as described above.

[0082] In addition, as previously described, the present invention may be configured as a system in which a camera is linked with a computing device, such as a smartphone, PC, or laptop, which is mounted on-premise or embedded.

[0083] To explain this in more detail, the user terminal (100) may include any form of a handheld computing device (e.g., PDA, email client, etc.), a mobile phone, or any form of another type of computing or communication platform, but the present invention is not limited thereto.

[0084] At this time, the facial recognition-based user authentication application used by the user terminal (100) may be an application embedded in the user terminal (100) or an application downloaded from an application distribution server and installed on the user terminal (100).

[0085] As an optional embodiment, a server (300) may be further included on the system (10).

[0086] When a server is provided in the system (10), the server (300) receives an image of the user (200) taken from the user terminal (100) and provides a direction identifier (110) to the user terminal (100) to induce the user (200) to move their face (210) for authentication. That is, the user terminal (100) can only perform the role of taking a picture of the user (200) and transmitting it to the server (300), or displaying an interface received from the server (300).

[0087] Accordingly, the role performed by the server (300) is the same as the process of authentication described above, except for the transmission and reception of data required for the authentication process with the user terminal (100) through the communication network, so redundant explanations are omitted.

[0088] Meanwhile, the communication network performs the role of connecting the user terminal (100) and the server (300). That is, the communication network refers to a communication network that provides a connection path so that the user terminal (100) can transmit and receive data after connecting to the server (300). The communication network may include wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communication, but the scope of the present invention is not limited thereto.

[0089] FIG. 2 is a diagram showing the configuration of a user terminal (100) according to one embodiment of the present invention.

[0090] FIG. 3 is a diagram showing the configuration of a server (300) according to one embodiment of the present invention.

[0091] First, referring to FIG. 2, the user terminal (100) may include a communication module (310), memory (320), processor (330), database (340), display module (350) and camera module (360).

[0092] Additionally, referring to FIG. 3, the server (300) may include a communication module (410) of the server, a memory (420) of the server, a processor (430) of the server, and a database (440) of the server.

[0093] At this time, since the communication module (310), memory (320), processor (330), and database (340) of the user terminal (100) each perform the same role as the communication module (340), memory (420), processor (430), and database (440) of the server (300), redundant descriptions will be omitted.

[0094] In detail, the communication module (310) provides a communication interface necessary to provide transmission and reception signals between the user terminal (100) and the server (300) in the form of packet data in conjunction with a communication network. Furthermore, the communication module (310) can perform the role of receiving a data request from the server (300) and transmitting data as a response thereto.

[0095] Here, the communication module (310) may be a device including hardware and software necessary to transmit and receive signals, such as control signals or data signals, through a wired or wireless connection with another network device.

[0096] The memory (320) has a program written on it to provide a facial recognition-based user authentication method. It also performs the function of temporarily or permanently storing data processed by the processor (330). Here, the memory (320) may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto.

[0097] The processor (330) is a type of central processing unit that controls the entire process for providing a facial recognition-based user authentication method. Each step performed by the processor (330) will be described later with reference to FIG. 4 (also, each step performed by the server (300) will be described later with reference to FIG. 5).

[0098] Here, the processor (330) may include any type of device capable of processing data, such as a processor. Here, 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed, for example, by code or instructions included in a program. Examples of such data processing devices embedded in hardware may include a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

[0099] The database (340) may store data regarding the face (210) of a user (200) for performing facial recognition-based user authentication. Specifically, an algorithm for the processor (330) to identify the face (210) and data regarding the same may be stored.

[0100] Although not shown in FIGS. 2 and 3, some of the data may be stored in a database (not shown) that is physically or conceptually separated from the database (340).

[0101] The display module (350) is a module for outputting visual information of the user terminal (100), and can display the image of the user (200) captured through the camera module (360) described later and a direction identifier (110) for inducing authentication. The display module (350) can be implemented using various panel technologies such as CRT, LCD, and OLED, and as an optional embodiment, a panel with built-in touch input / output functions may be used.

[0102] The camera module (360) is a module for capturing images and can generate video by capturing a user (200) performing authentication in real time. Accordingly, the minimum specifications of the camera module (360) must satisfy image quality capable of capturing the user (200) in real time while distinguishing major body parts such as the eyes of the face (210).

[0103] FIG. 4 is a flowchart illustrating a facial recognition-based user authentication process performed by a user terminal according to an embodiment of the present invention.

[0104] Referring to FIG. 4, the user terminal (100) captures an image through the camera module (160) and can recognize the face (210) of the user (200) in the captured image (S110).

[0105] At this time, the user terminal (100) can determine whether the face (210) exists in the captured image by identifying the pre-set facial features of the face (210). For example, by recognizing the eyes, nose, and mouth in the image through a pre-set facial recognition algorithm (or artificial intelligence), the user terminal (100) determines whether the face (210) exists in the image.

[0106] Additionally, the user terminal (100) determines whether the face (210) is looking straight ahead based on the location where each facial feature is recognized, and if it is determined that the face (210) is looking straight ahead, it can start authentication. That is, the user terminal (100) can start authentication based on the front view so that it can accurately recognize the user.

[0107] Next, the user terminal (100) can provide a direction identifier (110) at any location within the image, taking into account the location within the image of the recognized face (210) (S120).

[0108] At this time, the user terminal (100) may randomly provide at least one of the direction designated by the direction identifier (110) and the location where the direction identifier (110) is provided, as shown in (a) of FIG. 10, whenever authentication is performed. By providing the direction identifier (110) in a random direction, it is possible to prevent spoofing using a previously captured video of a face turning.

[0109] In addition, as previously explained, the user terminal (100) may display the direction identifier (110) by overlaying it on a portion of the second area (120) corresponding to the direction indicated by the direction identifier (110), excluding the first area (130) where the face (210) is recognized on the display area. This is to allow the user (200) to intuitively and comfortably check the direction identifier (110) as they turn their head.

[0110] Additionally, as an optional embodiment, if the area of ​​the second region (120) is not located within a preset threshold range, an identifier indicating that the distance between the camera module (160) and the face (210) should be adjusted may be additionally provided. For example, if the distance between the face (210) and the camera is too far, the accuracy of the user terminal (100) in recognizing the face (210) may be reduced, so a request to narrow the distance between the cameras may be made through a separate identifier. Conversely, if the distance between the face (210) and the camera is too close, the camera module (160) may not be able to properly focus the image, or the second region (120) within the image may become too large, making it impossible to display the direction identifier (110), so a request to widen the distance between the cameras may be made through a separate identifier.

[0111] Next, the user terminal (100) can analyze the captured video received in real time to determine whether the movement of the recognized face (210) matches a command corresponding to the direction identifier (110) (S130).

[0112] As shown in images (b) to (d) of FIG. 10, a user (200) who takes a picture of a face (210) through a camera moves their face (210) in the direction indicated by the direction identifier (110), and the user terminal (100) can perform authentication by recognizing the direction of movement of the face (210).

[0113] At this time, the movement of the face (210) is calculated by identifying the position change of the facial features of the face (210) for each frame of the video, and it can be determined whether the movement of the face (210) matches the direction identifier (110). The specific identification process will be explained in detail through Fig. 6, which will be described later.

[0114] In this case, as an optional embodiment, as illustrated in FIG. 10 (a) to (c), the direction, size, shape, etc. of the direction identifier (110) may be changed (i.e., the size is reduced as in direction identifier (110a) to direction identifier (110c)) as the user (200) turns the face (210), thereby indicating that the user (200) is moving their head in the correct direction.

[0115] Finally, if the user terminal (100) determines that the movement of the recognized face (210) corresponds to a command, it can change and display the direction identifier (110) and complete the authentication (S140).

[0116] Subsequently, when the user terminal (100) determines that the user (200)’s facial movement corresponds to a command, it may notify the user (200) that authentication is complete according to a pre-set method. At this time, the pre-set method may include at least one of the following: displaying a specific identifier (a direction identifier (110d) represented as a dot as in (d) of FIG. 10), removing the direction identifier (110), providing a vibration function of the user terminal (100), or providing a pre-set sound. However, the method of notifying that the facial movement corresponds to a command may be applied in various ways depending on the specifications of the user terminal (100) or the settings of the user (200).

[0117] Meanwhile, the command corresponding to the direction identifier (110) presented to the user (200) by the user terminal (100) for authentication can command the angle of the face (210) to be moved within a range that allows the eyes to be recognized in the image captured by the user terminal (100), even if the user (200) moves the face (210) and the authentication is completed. At this time, the user terminal (100) receives information regarding the direction of movement of the face (210) that can be compared with the direction of the direction identifier (110), and when the direction of movement of the face (210) and the direction of the direction identifier (110) correspond to each other, if the amount of change in width and the amount of change in position of the eyes in the image captured by the user terminal (100) exceed a preset threshold compared to the width and position of the eyes in the image when the user (200) starts authentication with the user terminal (100), it can be determined that authentication is completed.

[0118] These restrictions allow the user (200) to determine how far the face (210) must be turned for authentication, and the user terminal (100) can resolve the uncertainty of authentication that may occur due to the omission of specific facial features during the authentication process.

[0119] FIG. 5 is a flowchart illustrating a facial recognition-based user authentication process performed by a server (300) according to an embodiment of the present invention.

[0120] Steps (S210) to (S240) illustrated in FIG. 5 are performed by the server (300), and the same technical process as steps (S110) to (S140) can be performed.

[0121] First, the server (300) receives an image captured through a camera from the user terminal (100) and can recognize the face (210) of the user (200) in the captured image (S210).

[0122] Next, the server (300) can provide the direction identifier (110) at any location within the image, taking into account the location within the image of the recognized face (210) (S220).

[0123] Next, the server (300) can analyze the captured video received in real time to determine whether the movement of the recognized face (210) matches a command corresponding to the direction identifier (110) (S230).

[0124] Finally, if the server (300) determines that the movement of the recognized face (210) corresponds to a command, it can provide a command to the user terminal (100) to change and display the direction identifier (110) and complete the authentication (S240).

[0125] FIG. 6 is a flowchart illustrating the process of identifying the face position of a user (200) from an image and providing a direction identifier (110) accordingly, according to an embodiment of the present invention.

[0126] Referring to FIG. 6, the user terminal (100) can calculate the nth coordinate information of the pre-set facial features of the face (210) from the nth frame of the image (S131).

[0127] At this time, the facial features may correspond to a pair of eyes, nose, mouth, and ears. Additionally, to calculate the location of each facial feature, the user terminal (100) may set coordinate values ​​for the location of each region of the image in advance, and set the coordinate values ​​of the location where the facial features are located as the n-th coordinate information of the facial features.

[0128] Next, the user terminal (100) can calculate the n+1 coordinate information of the facial features from the n+1 frame after the n-th frame (S132).

[0129] In this case, n can mean a natural number equal to or greater than 1. That is, the n+1th frame can be the next frame consecutive to the nth frame.

[0130] Next, the user terminal (100) can calculate vector information of facial features for the direction in which the user (200)'s face (210) moves based on the n-th coordinate information and the n+1-th coordinate information (S133).

[0131] For example, if the coordinates of the nose in the first frame are (1, 1, 1) and the coordinates of the nose in the second frame are (3, 3, 1), the vector information of the nose can be calculated as (+2, +2, 0). This is a simple example of calculating vector information of facial features, and as the coordinate information becomes more complex, the method of calculating vector information including position and direction information regarding the movement of facial features can be advanced. In other words, the method of determining the direction of movement of the face through the coordinate information of facial features does not limit the scope of the present invention.

[0132] Meanwhile, the user terminal (100) can adjust the shape or size of the direction identifier (110) according to a preset method based on the change in the size of the vector information per frame of the image. For example, by referring to the direction identifiers (110a) to (110c) respectively illustrated in FIG. 10, it can be seen that the length of the arrow becomes shorter as the face (210) of the user (200) turns toward the direction identifier (110). The user (200) can recognize that they are moving their face (210) correctly by checking the size of the direction identifier (110). That is, as the size of the absolute value of the vector information for the direction of movement of the face (210) increases, the size of the direction identifier (110) may decrease or the display area of ​​the direction identifier (110) may be erased.

[0133] In this case, as an additional embodiment, when the direction identifier (110) is configured in the form of an arrow, the user terminal (100) can adjust the length or thickness (width) of the line segments constituting the arrow according to the movement of the face (210). For example, the length of the line segments can be adjusted according to the degree to which the head turns left and right, and the thickness of the line segments can be adjusted according to the height of the head.

[0134] Finally, the user terminal (100) can determine the direction of movement of the face (210) in real time based on vector information and determine whether it matches the command by comparing the direction of movement of the face (210) with the direction of the direction identifier (110) (S134).

[0135] In this optional embodiment, the user terminal (100) can compare the direction of movement of the face (210) and the direction of the direction identifier (110) in real time. However, if a predetermined amount of time elapses while the difference between the direction of movement of the face (210) and the direction facing the direction identifier (110) is greater than a preset threshold, the user terminal (100) may determine that authentication has failed. For example, if the direction identifier (110) points to the left direction of the head but the direction in which the face (210) moves is to the right, the user terminal (100) recognizes that the face (210) is moving in the wrong direction, but a predetermined grace period may be provided because the user (200) can move again in the direction pointed to by the direction identifier (110). However, if the user (200) does not move in the direction indicated by the direction identifier (110) even after a predetermined grace period, the user terminal (100) determines that the user (200) is not performing authentication and may terminate authentication or provide a warning message.

[0136] In this case, as an optional embodiment, if the face (210) does not move along the direction identifier (110), the user terminal (100) may fix the size of the direction identifier (110) or change it to an identifier corresponding to a warning and display it on the display.

[0137] In an additional embodiment, the direction indicated by the direction identifier (110) may vary depending on specific conditions. For example, if the facial features are moved and correspond to specific coordinate information that has been pre-set, the user terminal (100) may set and present a new direction identifier (110), and if the user (200) moves the face (210) further according to the newly created direction identifier (110), authentication may be completed.

[0138] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0139] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

Claims

1. A method for performing facial recognition-based user authentication performed by a user terminal, (a) A step of capturing an image through a camera and recognizing the user's face in the captured image; (b) a step of providing a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) a step of analyzing the captured image received in real time to determine whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) If it is determined that the movement of the recognized face corresponds to the command, the step of changing and displaying the direction identifier and completing authentication is included. At least one of the direction designated by the above direction identifier and the location where the direction identifier is provided is provided randomly each time authentication is performed, A method for performing facial recognition-based user authentication.

2. In Paragraph 1, The above step (a) is, Identifying the preset facial features of the face to determine whether the face exists, and determining whether the face is looking straight ahead based on the respective recognized positions of the facial features. If it is determined that the above face is looking straight ahead, the above authentication is initiated. A method for performing facial recognition-based user authentication.

3. In Paragraph 1, The above step (b) is, Displaying the direction identifier by overlaying it on a portion of the second area corresponding to the direction signified by the direction identifier among the remaining portions of the second area, excluding the first area where the face is recognized, on the display area of ​​the user terminal. A method for performing facial recognition-based user authentication.

4. In Paragraph 3, If the area of ​​the second region is not located within a preset threshold range, providing an identifier meaning to adjust the distance between the camera and the face. A method for performing facial recognition-based user authentication.

5. In Paragraph 1, The above (c) step is, (c-1) A step of calculating the n-th coordinate information of a pre-set facial feature of the face from the n-th frame of the image; (n is a natural number greater than or equal to 1) (c-2) A step of calculating the n+1 coordinate information of the facial features from the n+1 frame after the n frame; (c-3) A step of calculating vector information of the facial features regarding the direction in which the user's face moves based on the n-th coordinate information and the n+1-th coordinate information; and (c-4) A step comprising determining the direction of movement of the face in real time based on the vector information, and determining whether the direction of movement of the face matches the command by comparing the direction of movement of the face with the direction of the direction identifier, A method for performing facial recognition-based user authentication.

6. In Paragraph 5, Adjusting the shape or size of the direction identifier according to a preset method based on the change in size of the vector information per frame of the above image, A method for performing facial recognition-based user authentication.

7. In Paragraph 6, As the magnitude of the absolute value of the vector information regarding the direction of movement of the face increases, the size of the direction identifier decreases or the display area of ​​the direction identifier is erased. A method for performing facial recognition-based user authentication.

8. In Paragraph 5, The above (c-4) step is, Comparing the direction of movement of the face and the direction of the direction identifier in real time, and determining that the authentication has failed if a predetermined amount of time elapses while the difference between the direction of movement of the face and the direction is greater than or equal to a preset threshold. A method for performing facial recognition-based user authentication.

9. In Paragraph 1, The above (d) step If it is determined that it corresponds to the above command, it indicates that the above authentication is completed according to the preset method, and The above-mentioned preset method includes at least one of the following: displaying a specific identifier, removing the direction identifier, providing a vibration function of the user terminal, or providing a preset sound. A method for performing facial recognition-based user authentication.

10. In Paragraph 1, The command corresponding to the above direction identifier is, A command to move the angle of the face within a range that allows the eyes to be recognized in the image captured by the user terminal, even if the above-mentioned face is moved to a state where authentication is completed. A method for performing facial recognition-based user authentication.

11. In Paragraph 10, Information regarding the direction of movement of the face that can be compared with the direction of the direction identifier is received, and when the direction of movement of the face and the direction of the direction identifier correspond to each other, if the amount of change in width and the amount of change in position of the eye in the image captured by the user terminal exceed a preset threshold when compared with the width and position of the eye in the image when the user starts authentication with the user terminal, it is determined that authentication is completed. A method for performing facial recognition-based user authentication.

12. Communication module; At least one processor; and It includes a memory electrically connected to the above processor and storing at least one code for performing a method of performing facial recognition-based user authentication, When the above memory is executed through the above processor, the processor performs the above method, The above method is, (a) A step of capturing an image through a camera and recognizing the user's face in the captured image; (b) a step of providing a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) a step of analyzing a captured video received in real time to determine whether the movement of a recognized face matches a command corresponding to the direction identifier; and (d) If it is determined that the movement of the recognized face corresponds to the command, the step of changing and displaying the direction identifier and completing authentication is included. At least one of the direction designated by the above direction identifier and the location where the direction identifier is provided is provided randomly each time authentication is performed, A device that performs facial recognition-based user authentication.

13. A method for performing facial recognition-based user authentication performed by a server, (a) receiving an image captured through a camera from a user terminal and recognizing the user's face in the captured image; (b) a step of providing a command to a user terminal to provide a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) receiving a captured image received in real time from the user terminal and determining whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) If it is determined that the movement of the recognized face corresponds to the command, the step of providing a command to the user terminal to change and display the direction identifier and completing authentication is included. At least one of the direction designated by the above direction identifier and the location where the direction identifier is provided is provided randomly each time authentication is performed, A method for performing facial recognition-based user authentication.

14. Communication module; At least one processor; and It includes a memory electrically connected to the above processor and storing at least one code for performing a method of performing facial recognition-based user authentication, When the above memory is executed through the above processor, the processor performs the above method, The above method is, (a) receiving an image captured through a camera from a user terminal and recognizing the user's face in the captured image; (b) a step of providing a command to a user terminal to provide a direction identifier at an arbitrary location within the image, taking into account the position of the recognized face within the image; (c) receiving a captured image received in real time from the user terminal and determining whether the movement of the recognized face matches a command corresponding to the direction identifier; and (d) If it is determined that the movement of the recognized face corresponds to the command, the step of providing a command to the user terminal to change and display the direction identifier and completing authentication is included. At least one of the direction designated by the above direction identifier and the location where the direction identifier is provided is provided randomly each time authentication is performed, A server that performs facial recognition-based user authentication.

Images