Facial recognition system and method for controlling the facial recognition system

Abstract

To provide a method and system for authenticating a user.SOLUTION: A control method for a face recognition system according to the present invention includes the steps of acquiring an image using a camera provided in a terminal that executes a face recognition function, and recognizing a face from the image based on a predetermined face recognition algorithm, confirming from the image whether a recognized face satisfies a predetermined face condition, and based on a confirmation result, requesting a predetermined server for user authentication on the recognized face to perform user authentication on the recognized face, and performing control according to the user authentication result of the recognized face based on the user authentication result of the recognized face received from the server.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system for recognizing a user's face for user authentication and a method for controlling the same. [Background technology]

[0002] 2. Description of the Related Art With the development of technology, the use of terminals has increased, and in particular, terminals are being used as a means of authenticating users in various situations.

[0003] For example, there is a need to authenticate a true user in various situations, such as user verification at an entry restriction device (e.g., an entry restriction gate or a speed gate) installed in a restricted area, verification of attendees at a conference, verification of a patient's visit at a hospital, user verification for payment at a cafe, etc. In this way, by utilizing a terminal device in a situation where user authentication is required, unnecessary labor is reduced, and further, the time required for user authentication is shortened.

[0004] There are various methods for user authentication using a terminal device, such as a method in which the user directly inputs identification information (e.g., telephone number, email address, etc.) or biometric information (e.g., fingerprint information, etc.) to identify himself / herself into the terminal device.

[0005] However, in recent years, much research has been conducted into methods for quickly and accurately recognizing a user's face so that user authentication can be performed without the user having to directly input information into a terminal device. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention provides a face recognition system and a method for controlling the same, which can more quickly and accurately recognize the face of a user who is to be authenticated.

[0007] The present invention also provides a face recognition system and a method for controlling the same that grasp a user's intention as to whether or not to perform user authentication.

[0008] Furthermore, the present invention provides a face recognition system and a method for controlling the same that completes user authentication for a user who truly intends to perform the user authentication. [Means for solving the problem]

[0009] In order to solve the above problem, a control method for a face recognition system according to the present invention may include the steps of acquiring an image using a camera provided in a terminal device that executes a face recognition function, recognizing a face from the image based on a predetermined face recognition algorithm, confirming whether the face recognized from the image satisfies predetermined face conditions, requesting user authentication for the recognized face from a predetermined server based on a result of the confirmation in order to perform user authentication for the recognized face, and performing control according to the result of user authentication for the recognized face based on the result of user authentication for the recognized face received from the server.

[0010] In addition, the face recognition system of the present invention may include a camera unit that acquires an image, a storage unit that stores the acquired image, and a control unit that recognizes a face from the image based on a predetermined face recognition algorithm and confirms whether the face recognized from the image satisfies predetermined face conditions, wherein the control unit may request user authentication of the recognized face to a predetermined server based on a result of the confirmation in order to perform user authentication of the recognized face, and may perform control according to the result of the user authentication of the recognized face based on the result of the user authentication of the recognized face received from the server.

[0011] Furthermore, the face recognition system of the present invention includes a camera unit that acquires an image, a storage unit that stores the acquired image, a control unit that recognizes a face from the image based on a predetermined face recognition algorithm and checks whether the face recognized from the image satisfies predetermined face conditions, and a server that performs user authentication for the recognized face if the recognized face satisfies the predetermined face conditions, and the control unit may perform control according to the result of the user authentication for the recognized face based on the result of the user authentication for the recognized face performed by the server.

[0012] Furthermore, the program of the present invention, which is executed by one or more processes in an electronic device and stored in a computer-readable recording medium, may include commands to perform the following steps: acquiring an image using a camera provided in a terminal device that executes a face recognition function; recognizing a face from the image based on a predetermined face recognition algorithm; confirming whether the face recognized from the image satisfies predetermined face conditions; requesting user authentication for the recognized face from a predetermined server based on the result of the confirmation in order to perform user authentication for the recognized face; and performing control according to the result of user authentication for the recognized face based on the result of user authentication for the recognized face received from the server. Effect of the Invention

[0013] As described above, the face recognition system and control method thereof according to the present invention can acquire an image using a camera provided in a terminal that executes a face recognition function, and recognize a face from the image based on a predetermined face recognition algorithm. In this way, the present invention can accurately and quickly recognize a user's face when the user is located near the terminal without direct input from the user. Therefore, the user can have the terminal recognize his or her face more conveniently and intuitively by being located near the terminal.

[0014] Furthermore, the face recognition system and control method according to the present invention can determine whether a face recognized from an image is suitable for user authentication. That is, the present invention can check whether a face recognized from an image satisfies a predetermined face condition. Therefore, according to the present invention, if a face that is not suitable for user authentication is recognized, user authentication is not performed, thereby improving the efficiency of data processing.

[0015] Furthermore, the face recognition system and control method according to the present invention can perform user authentication only for users who satisfy certain face conditions. Therefore, according to the present invention, even if there are multiple users near a terminal device, the face of the user who is to be authenticated can be accurately recognized and authentication of the user can be performed.

[0016] Furthermore, the face recognition system and control method thereof according to the present invention can request a predetermined server to perform user authentication for a recognized face based on whether or not the face recognized from a video satisfies predetermined face conditions. In this way, the face recognition system and control method thereof according to the present invention can perform user authentication for the face of a user who truly desires user authentication. Therefore, according to the present invention, the accuracy and reliability of the user authentication results can be improved by reducing the inconvenience of performing user authentication for a user who does not intend to perform user authentication or for another user who is close to the user performing user authentication. [Brief description of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram for explaining a face recognition system according to the present invention; [Diagram 2] 1 is a flowchart for explaining a face recognition system according to the present invention. [Diagram 3] 4 is a flowchart illustrating a control method of the face recognition system according to the present invention. [Figure 4] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Diagram 5]1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 6] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 7A] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 7B] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 8] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 9] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 10] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 11] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 12] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; [Figure 13] 1 is a conceptual diagram for explaining a face recognition system and a control method thereof according to the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the attached drawings. The same or similar components are denoted by the same or similar reference numerals regardless of the drawing number, and the description thereof will be omitted. The suffixes "module" and "part" of components used in the following description are given or mixed to facilitate the preparation of the specification, and do not have any significance or usefulness in themselves. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of related publicly known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. Note that the attached drawings are merely for the purpose of making the embodiments disclosed in this specification easily understandable, and the technical ideas disclosed in this specification are not limited by the attached drawings, and it should be understood that any modifications, equivalents, or alternatives included in the ideas and technical scope of the present invention are included in the present invention.

[0019] Terms including ordinal numbers such as first, second, etc. are used to describe various components, but the components are not limited to the terms. The terms are used only to distinguish one component from another.

[0020] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, and that there may be additional components in between. In contrast, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no additional components in between.

[0021] Singular expressions include plural expressions unless otherwise specified.

[0022] As used herein, terms such as "comprise" and "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof set forth in the specification, but should not be interpreted as precluding the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0023] The present invention provides a face recognition system and a control method for the face recognition system, which can accurately and quickly recognize a user's face without direct input from the user and recognize a user's face that satisfies a predetermined face condition in order to grasp the user's true intention regarding whether to perform user authentication. Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram for explaining the face recognition system according to the present invention, and FIG. 2 is a block diagram for explaining the face recognition system according to the present invention. FIG. 3 is a flowchart for explaining the control method of the face recognition system according to the present invention. Also, FIGS. 4, 5, 6, 7A, 7B, 8, 9, 10, 11, 12, and 13 are conceptual diagrams for explaining the face recognition system and the control method thereof according to the present invention.

[0024] 1, in the present invention, user authentication is performed by cooperation between a face recognition system 100 and a server 200. In the present invention, the server 200 may be located in the same place as the face recognition system 100 or in a different place. The server 200 is configured to process user authentication for a face recognized by the face recognition system 100.

[0025] It should be noted that the server 200 may be realized as a part of the configuration of the face recognition system 100. In that case, the face recognition system 100 executes functions including the role of the server 200.

[0026] 1, the face recognition system 100 and the server 200 may be realized as separate components. For convenience of explanation, the present invention will be explained by separating the face recognition system 100 and the server 200.

[0027] Needless to say, regardless of whether the facial recognition system 100 and the server 200 are physically separated, the functions and controls performed in the facial recognition system 100 and the server 200 described in this invention may be described as being performed in a "facial recognition system."

[0028] In the present invention, the user's face is recognized, user authentication is performed, and control can be performed so that an event or function related to user authentication (hereinafter, referred to as a target function) is performed depending on the result of the user authentication.

[0029] Here, there are many types of "target functions", and they refer to functions that require user authentication, such as a specific space entry restriction function, a payment function, an attendance confirmation function, a hospital visit confirmation function, etc. For example, the specific space entry restriction function is performed by controlling the operation of the entry restriction gate (or entry restriction device, speed gate) 20 shown in Fig. 1 so that it is in either an open state or a closed state.

[0030] Thus, in the present invention, the face recognition system 100 has various hardware configurations according to the functions to be achieved by face recognition and user authentication (for example, entry restriction, attendance confirmation, payment approval, etc.). In the present invention, as a representative example, user authentication at an entry restriction gate 20 as shown in Fig. 1 will be described. However, it goes without saying that the face recognition and user authentication method described below can be applied to face recognition systems having various functions.

[0031] 1, a face recognition system 100 according to the present invention (hereinafter, referred to as "system 100") is configured to include at least one of a sensor unit 110, a storage unit 120, a communication unit 130, a display unit 140, an audio output unit 150, and a control unit 160. Here, the system 100 according to the present invention is not limited to the above components, and may further include components that perform the same or similar functions as those described in this specification.

[0032] Furthermore, the face recognition system 100 may be linked to the entry restriction gate 20, or may include the entry restriction gate 20 as a component. Needless to say, the face recognition system 100 may be a part of the configuration of the entry restriction gate 20. The sensor unit 110 detects (or photographs) a user (or an object) using a configuration of a detection module (e.g., a camera 111, a thermal imaging camera, a proximity sensor, an infrared sensor, etc.) provided in the system 100.

[0033] In the present invention, "detect" is interpreted in various ways depending on the detection module. As an example, when the camera 111 is used, "detect" is understood as taking an image. As another example, when a proximity sensor is used, "detect" is understood as detecting the presence of a user (or an object). Hereinafter, "detect" is used interchangeably with "taking (or acquiring) an image."

[0034] The sensor unit 110 captures an image (for example, at least one of an RGB image frame and a thermal image frame) in the form of at least one of a video and an image.

[0035] The sensor unit 110 captures images at predetermined time intervals or based on a user request.

[0036] The storage unit 120 is configured to store data related to face recognition and / or user authentication according to the present invention using a memory or a cloud server provided (or inserted) in the system 100. For example, the storage unit 120 stores information detected by the sensor unit 110, images acquired, or information acquired therefrom.

[0037] In addition, the communication unit 130 transmits and receives data to and from the server 200, an external server (not shown), or a cloud server (not shown), using the configuration of a communication module (e.g., a mobile communication module, a short-range communication module, a wireless Internet module, a location information module, a broadcast receiving module, etc.) provided in the system 100.

[0038] Further, the display unit 140 is configured to output visual information and, under the control of the control unit 160, is configured to output information relating to at least one of face recognition and user authentication.

[0039] For example, the display unit 140 is controlled to output information regarding the success (or failure) of user authentication based on the authentication result information regarding the user to be authenticated received from the server 200, or to output guidance information that guides the user to the location so that the sensor unit 110 can photograph the user.

[0040] For example, the display unit 140 is configured to output the authentication result or guidance information regarding execution of the target function (for example, "Welcome USER1!", see 810 in FIG. 8) based on the authentication result information regarding the user to be authenticated.

[0041] Furthermore, the audio output section 150 audibly outputs information relating to the present invention.

[0042] For example, under the control of the control unit 160, the audio output unit 150 outputs the authentication result or guidance information regarding the execution of a target function by voice, based on the authentication result information regarding the user to be authenticated.

[0043] Furthermore, the control unit 160 utilizes at least one of the CPU and the cloud server provided in the system 100 to control each component and to perform overall control related to the present invention.

[0044] The control unit 160 has control authority over at least one of the components of the system 100 for carrying out the present invention.

[0045] The control unit 160 recognizes the user's face based on the information detected by the sensor unit 110 and the information received by the communication unit 130, and performs a process for user authentication.

[0046] The control unit 160 includes a face recognition unit 161, which recognizes faces from the video captured by the sensor unit 110 based on an algorithm having a face recognition function. The face recognition unit 161 also performs a series of controls related to the recognized faces, which will be described later. In the present invention, for the sake of convenience of explanation, the configurations of the face recognition unit 161 and the control unit 160 are not differentiated, but are unified in the explanation of the control unit 160.

[0047] Meanwhile, the system 100 and the server 200 according to the present invention operate in conjunction with each other to perform user authentication for a target function and control the execution of the target function based on the result of the user authentication for the target function.

[0048] The server 200 performs user authentication based on a request for user authentication from the system 100 and transmits the result of the user authentication to the system 100 .

[0049] Meanwhile, in the present invention, when the function executed as a result of user authentication is the operation control of the entry restriction gate 20, the door of the entry restriction gate 20 is controlled to open or not open based on the result of user authentication for the user to be authenticated. The entry restriction gate 20 is configured to be controlled based on a control command transmitted from at least one of the server 200 and the system 100.

[0050] 1, the server 200 according to the present invention is configured to include at least one of a communication unit 210, a storage unit 220, and a control unit 230. Here, the server 200 according to the present invention is not limited to the above components, and may further include components having the same or similar functions as those described in this specification.

[0051] The communication unit 210 is configured to transmit and receive data to and from at least one of the system 100, the restricted access gate 20, an external server (not shown), or a cloud server (not shown).

[0052] For example, the communication unit 210 receives from the system 100 video information including the face of a user to be authenticated or face information related to a user to be authenticated, and transmits to the system 100 a result of user authentication for the user to be authenticated.

[0053] The storage unit 220 is configured to store various information related to user authentication.

[0054] The storage unit 220 includes at least one of a user DB 221 including reference facial feature information related to pre-registered faces, and a terminal DB 222 including information related to the system 100 .

[0055] Here, the reference face feature information includes at least one of a face image registered in advance and feature information (for example, feature vector information) relating to a face registered in advance.

[0056] User information on each person (or person) is stored in the user DB 221. That is, the user information corresponding to each person is matched and stored in the user DB.

[0057] Here, the user information includes at least one of each person's name, date of birth, address, telephone number, email address, account information, employee number, ID, biometric information (fingerprint information, iris information, etc.), reference facial feature information, and information regarding the schedule.

[0058] The reference face image included in the reference face feature information is a face image of a specific person, and may be one or more. In the present invention, the user DB is also called "information about a user," "user identity information," or "user identification information."

[0059] Meanwhile, the terminal DB 222 stores information about the system 100 that performs the user authentication process.

[0060] Here, the information on the terminal includes any one of identification information (serial number) of each system 100 (or the access control gate 20), location, and information on a target function linked to user authentication.

[0061] On the other hand, the control unit 230 uses the reference facial feature information stored in the user DB 221 to perform user authentication for the user to be authenticated.

[0062] Here, user authentication is performed by user authentication unit 231 of control unit 230. In other words, user authentication unit 231 is understood to fulfill part of the role of control unit 230. In the present invention, control unit 230 and user authentication unit 231 are not distinguished but are used interchangeably.

[0063] The basic process of performing user authentication in the present invention will be described below based on the configurations of the system 100 and the server 200 described above.

[0064] 2, in the system 100 according to the present invention, an image is captured using a sensor unit (e.g., a camera) 110, and the captured image is subjected to data processing (S210). The image may be captured at a predetermined time interval.

[0065] In the sensor unit 110, an image frame (or video, for example, an RGB image frame) and a thermal image frame (or video) are captured (S211, S212).

[0066] In this manner, an image (referred to as an "image" in the explanation of the present invention) that is to be subjected to face recognition is obtained by capturing an image using the sensor unit 110.

[0067] Next, the system 100 performs a process of recognizing a face from an image, ie, a video (S220).

[0068] More specifically, the control unit 160 recognizes the user's face from the video (S221), and determines whether the recognized user's face satisfies a predetermined face condition (S222).

[0069] Here, the predetermined face condition is related to whether the recognized face has a condition suitable for user authentication, and the predetermined face condition is set in various ways. For example, the predetermined face condition is related to at least one of various conditions such as i) the position where the recognized face is included in the image, ii) the size of the area occupied by the recognized face in the image, iii) the angle of the recognized face in the image (e.g., whether the user's face is facing forward or not), iv) whether the entire user's face is recognized or not, and v) whether the user is wearing a mask or not.

[0070] When the control unit 160 recognizes a face that satisfies a predetermined facial condition from the image captured by the camera 111, the control unit 160 performs a process for performing user authentication for the face that satisfies the predetermined facial condition using the image.

[0071] The control unit 160 specifies, as the authentication target face, a face that satisfies a predetermined face condition among the faces recognized from the video. For example, the control unit 160 specifies, as the authentication target face, the face 2000a that is not cut off and occupies the widest area among the faces 2000a, 2000b, 2000c, 2000d, and 2000e of multiple users in the video shown in (b) of FIG.

[0072] In this manner, when a face to be authenticated is identified from the image, the control unit 160 further performs spoofing check and temperature check (S230).

[0073] The control unit 160 requests user authentication for an image that has passed the spoofing check and temperature check.

[0074] The control unit 160 transmits a request for user authentication for the authentication target user to the server 200 (S250, see FIG. 2).

[0075] The control unit 160 transmits to the server 200 information about the face of the user to be authenticated (such as the user's facial image and facial feature information) together with a request for user authentication.

[0076] The server 200 determines the degree of similarity between information about the face of the user to be authenticated that is included in the video and reference facial feature information stored in the user DB 221 of the server 200 (S240).

[0077] As an example of a method of determining the degree of similarity, the server 200 compares the face of the user to be authenticated that is included in the video with a reference face image stored in the user DB 221 to perform user authentication.

[0078] Here, comparing the reference face image with the face of the user to be authenticated contained in the video means comparing feature point information corresponding to the reference face image with feature point information on the user's face recognized from the video. That is, the user DB contains the user's reference face image and feature point information defined by facial features such as the eyes, nose, and mouth contained in the reference face image.

[0079] As another example of a similarity determination method, the server 200 performs user authentication by comparing the similarity between a feature vector for the face of the user to be authenticated extracted from a face image and a feature vector included in the reference face feature information. Here, feature vectors for various reference faces are stored and exist in the user DB. The server 200 searches (or extracts) a feature vector that is most similar to the feature vector for the face of the user to be authenticated from the user DB, and if the similarity between the searched feature vector and the feature vector for the face of the user to be authenticated is equal to or greater than a predetermined value (threshold), the server 200 processes the user as having been successfully authenticated.

[0080] Furthermore, the server 200 provides (or transmits) the result of the user authentication to the system 100, and the system 100 provides the result of the user authentication to the user based on the authentication result received from the server 200 (S260). For example, the authentication result is provided to the user from an output unit (e.g., a display unit, an audio output unit (speaker), etc.) provided in the system 100. As one example, when user authentication of the face of the user to be authenticated is successful, the control unit 160 controls the display unit 140 so that the authentication result or guidance information regarding the execution of the target function (e.g., "Welcome USER1!", see 810 in FIG. 8) is output to the display unit 140.

[0081] Furthermore, the control unit 160 performs various controls based on the result of user authentication for the user to be authenticated received from the server 200. As an example, when the system 100 is linked to an entry restriction device or is included in the entry restriction device, the control unit 160 controls the entry restriction gate 20 when the user authentication for the user to be authenticated is successful so that the successfully authenticated user can enter the specific space. The entry restriction gate 20 maintains the closed state as a default operating state, and is controlled to be in the open state based on the control by the control unit 160 or a control signal received from the server 200.

[0082] Furthermore, in the present invention, the system 100 and the entrance control gate 20, the payment terminal, etc. to which the result of the user authentication is applied are operated independently of each other. In this case, the server 200 transmits the result of the user authentication not only to the system 100 but also to the entrance control gate 20 and the payment terminal (S270).

[0083] In this case, the entrance restriction gate 20 and the payment terminal are configured to control their operations based on the result of user authentication received from the server 200 .

[0084] As an example, when the result of user authentication is utilized to control the operation of the entry restriction gate 20, the server 200 transmits a control signal to the entry restriction gate 20 to open the gate (or door) of the entry restriction gate 20 based on the result of user authentication.

[0085] Hereinafter, a method for quickly and accurately recognizing a user's face using each component included in the system 100 and the server 200 will be specifically described.

[0086] In the method for controlling a face recognition system according to the present invention, a process of acquiring an image using a camera 111 provided in a system 100 that performs a face recognition function is performed (S310, see FIG. 3).

[0087] 4(a), many users are located in the specific space 1. For example, in the specific space 1, there are users 1000a, 1000b, and 1000c who are trying to pass through the entrance restriction gate 20, a user 1000d who is stopped at a specific position, and a user 1000e who is freely moving in the specific space 1.

[0088] For ease of explanation, hereinafter, based on the entrance restriction gate 20 or the system 100, a user 1000a standing in the first row will be referred to as a first user 1000a, a user 1000b standing in the second row will be referred to as a second user 1000b, and a user 1000c standing in the third row will be referred to as a third user 1000c. In addition, a user 1000d that is stopped at a specific position will be referred to as a fourth user 1000d, and a moving user 1000e will be referred to as a fifth user 1000e.

[0089] Meanwhile, the system 100 or the camera 111 provided in the system 100 is arranged so as to capture an image of the specific space 1 in order to execute a function to be performed by user authentication (for example, a door operation function of the entry restriction gate 20). The camera 111 is configured to be arranged at an appropriate position so as to recognize the face of the user.

[0090] For example, as shown in (a) of FIG. 4, the camera 111 is positioned around the entry restriction gate 20 or in a certain area of ​​the entry restriction gate 20 so as to capture a specific space in which a user attempting to pass through the entry restriction gate 20 is located.

[0091] As shown in (b) of Fig. 4, the image 400 captured by the camera 111 includes information about a space corresponding to the field of view of the camera 111. As shown in (a) of Fig. 4, when a user is located within the field of view of the camera 111, the image 400 includes graphic objects 2000a, 2000b, 2000c, 2000d, and 2000e corresponding to the user, as shown in (b) of Fig. 4. As shown in the figure, the graphic objects 2000a, 2000b, 2000c, 2000d, and 2000e include images corresponding to the user's face.

[0092] When multiple users are positioned around camera 111 as shown in (a) of FIG. 4, (b) of FIG. 4 includes graphic objects (or facial images) 2000a, 2000b, 2000c, 2000d, and 2000e corresponding to the faces of each of the multiple users.

[0093] In the present invention, in order to avoid confusion of terms, a graphic object corresponding to a user's face is called a "face" or a "face image." For example, a graphic object corresponding to the face of a first user 1000a is called a first face 2000a or a first face image 2000a, and a graphic object corresponding to the face of a second user 1000b is called a second face 2000b or a second face image 2000b. In addition, in the drawings of the present invention, for convenience of explanation, only the faces of the users in the video 400 are highlighted.

[0094] Note that the above-mentioned step S310 corresponds to step S210 described in FIG. 2, and step S310 includes the detailed process described in step S210.

[0095] Next, in the present invention, a process of recognizing a face from the image is performed based on a predetermined face recognition algorithm (S320, see FIG. 3).

[0096] The control unit 160 recognizes a face from the image 400 acquired in (b) of FIG. 4 based on a face recognition algorithm. The face recognition algorithm is performed by various image object recognition methods that recognize objects from an image. As an example, the algorithm for recognizing a face from the image 400 is at least one method of recognizing a user's face from an image using an algorithm trained for face recognition (e.g., YOLO algorithm, CNN algorithm).

[0097] 5A, the control unit 160 assigns bounding boxes 3000a, 3000b, 3000c, 3000d, and 3000e to the recognized faces 2000a, 2000b, 2000c, 2000d, and 2000e, respectively. The bounding boxes 3000a, 3000b, 3000c, 3000d, and 3000e are assigned to the faces 2000a, 2000b, 2000c, 2000d, and 2000e included in the video.

[0098] The control unit 160 assigns different identification information to each recognized face. It will be understood that the identification information assigned to each recognized face is similar to the identification information related to the bounding boxes 3000a, 3000b, 3000c, 3000d, and 3000e.

[0099] Here, the identification information is ID (Identification), and for example, the control unit 160 assigns different IDs (for example, "USER1") to each of the recognized user faces 2000a, 2000b, 2000c, 2000d, and 2000e.

[0100] Note that control unit 160 assigns a different ID to each of multiple faces included in an image, and when the same face is included in different images, it assigns the same ID to the same face included in the different images.

[0101] For example, in FIG. 7A, the same ID (for example, "USER1") is assigned to the same face 2000a included in the images 730, 720, 710, and 700 captured at different times tp3, tp2, tp1, and t0.

[0102] When the control unit 160 recognizes a face from the video, the control unit 160 extracts information about each recognized face. For convenience of explanation, the extracted information is referred to as "face information."

[0103] The control unit 160 extracts face information for each of the recognized faces, and the extracted face information 510 is matched to the ID assigned to the recognized face and stored as shown in Fig. 5(b). The control unit 160 controls so that the face information 510 extracted from the image 400 is matched to the ID assigned to the face recognized from the image and stored in the storage unit 120.

[0104] As shown in (b) of Fig. 5, "face information 510" includes various information extracted in relation to a face included in a video. For example, face information 510 includes at least one of i) position information (pixel coordinate information) 511 of a face (bounding box) in the video, ii) face size information (corresponding to the size of the area occupied by the face in the video, or the size (or width) of the area corresponding to the bounding box) 512, iii) face angle information (information on whether the face faces a reference direction (e.g., camera direction), and iv) face feature information (e.g., vector information).

[0105] Here, facial feature information is information about the features of each face, and refers to feature data or feature vectors related thereto that are features for each part of the face whose features change, such as the eyes, eyebrows, nose, mouth, chin, etc. Each person has different features depending on their facial appearance, so users with different facial appearances have different facial feature information.

[0106] Such face information exists for each video in which the user's face is recognized.

[0107] 5(b), in the video (or frame) captured by the camera 111, there is identification information 520 for each video, and the video identification information 520 and face information 510 extracted from the video are matched with each other. As shown in the figure, the video identification information and face information are stored in the storage unit in the form of a table.

[0108] The video identification information 520 includes at least one of the following: i) a sequence number or ID for classifying the video (e.g., FRAME no: #1025); ii) shooting date and time information of the video (or shooting date and time information of the frame, e.g., 2021-11-14-13:00:05); and iii) information about the terminal device that shot the video (terminal identification information, terminal location information (e.g., LOBBY A)).

[0109] As shown in (b) of FIG. 5, in the storage unit 120, face information 510 of each face recognized from an image, an ID assigned to the face recognized from the image, and identification information 520 of the image in which the face was recognized are matched with each other and stored.

[0110] Furthermore, the control unit 160 controls the storage unit 120 so that the image captured by the camera 111, face information 510 extracted from the image, identification information of the image, etc. are stored in the storage unit 120 only for a predetermined period (a predetermined information storage period, for example, 10 minutes), thereby enabling efficient use of the storage unit 120.

[0111] For example, the control unit 160 controls the storage unit 120 so that the above-mentioned information is deleted from the storage unit 120 after a predetermined time (predetermined information storage time) has elapsed.

[0112] On the other hand, the image acquired from the camera may include the face of one user, and as shown in (a) of FIG. 5, the image 400 may include multiple faces 2000a, 2000b, 2000c, 2000d, and 2000e. In this case, some of the users corresponding to the multiple faces are users who do not intend to be authenticated. Also, when the image includes multiple faces, even if a specific user really intends to be authenticated, authentication may not be performed due to the faces of other users.

[0113] Moreover, even if there is no user in the space attempting user authentication, user authentication may be performed for at least one of the faces 2000a, 2000b, 2000c, 2000d, and 2000e included in the video 400.

[0114] Therefore, in order to perform user authentication by face recognition in a space where multiple users can freely enter, accurate user authentication of the specific user who is the intended user to be authenticated is essential.

[0115] Therefore, in the present invention, user authentication is performed on at least any face that satisfies a predetermined face condition among the faces recognized from the video 400.

[0116] More specifically, in the present invention, a process is performed to check whether a face recognized from an image satisfies a predetermined face condition (S330, see FIG. 3).

[0117] Here, the predetermined face condition is related to whether or not the recognized face has a condition suitable for user authentication, and the predetermined face condition is set in various ways. For example, the predetermined face condition is related to at least one of various conditions such as i) the position where the recognized face is included in the image, ii) the size of the area occupied by the recognized face in the image, iii) the angle of the recognized face in the image (e.g., whether or not the user's face is facing forward), iv) whether or not the entire face of the user is recognized, and v) whether or not the user is wearing a mask. The control unit 160 controls to perform user authentication for the user corresponding to the face by using at least one of the faces included in the image that satisfies the predetermined face condition.

[0118] When the control unit 160 recognizes multiple faces from the video, it checks whether each of the recognized multiple faces satisfies a predetermined face condition, and specifies the faces that satisfy the predetermined face condition from among the multiple faces.

[0119] For example, as shown in (a) of FIG. 5, when the control unit 160 recognizes multiple faces from the image 400, it determines whether or not each of the recognized first face 2000a, second face 2000b, third face 2000c, fourth face 2000d, and fifth face 2000e satisfies predetermined face conditions.

[0120] Only when the image 400 includes a face that satisfies a predetermined face condition, the control unit 160 performs user authentication for the user corresponding to the face included in the image 400. That is, even if the control unit 400 recognizes a face from the image, if the image does not include a face that satisfies a predetermined face condition, the control unit 400 does not perform a process of transmitting a user authentication request to a server for user authentication of the user corresponding to the face included in the image.

[0121] On the other hand, the size of the area occupied by a recognized face will be described as an example of a predetermined face condition. The control unit 160 judges whether the size (for example, the size or area of ​​the face area (or bounding box area)) occupied by each of the multiple faces recognized from the video is equal to or larger than a predetermined size. If a face that is equal to or larger than the predetermined size is present, the control unit 160 specifies the user corresponding to the face as a face to be subject to user authentication.

[0122] For example, as shown in (a) of FIG. 6, the control unit 160 identifies the first face 2000a, which is included in a face area having a predetermined size or larger, as the authentication target face to be subject to user authentication, among the first face 2000a and the fourth face 2000d recognized from the image 610.

[0123] Furthermore, when there are multiple face areas in the video that are equal to or larger than a predetermined size, control unit 160 specifies the face included in the largest face area among the multiple face areas as the authentication target face that is to be subject to user authentication.

[0124] For example, in the image 610 shown in (a) of FIG. 6, even if both the first face 2000a and the fourth face 2000d are larger than a predetermined size, the control unit 160 identifies the first face 2000a contained in the largest facial area as the face to be authenticated that is the subject of user authentication.

[0125] As another example of the predetermined face condition, the position where the recognized face is included in the video will be described. The predetermined face condition relates to whether the position where the recognized face is included in the video is within a predetermined radius from a predetermined position (for example, the center 611) of the video, as shown in (a) of FIG.

[0126] For example, as shown in (a) of FIG. 6, the control unit 160 identifies, among the first face 2000a and the fourth face 2000d recognized from the image 610, the first face 2000a located within a predetermined radius from a predetermined position in the image 610 (e.g., the center 611 of the image) as the authentication target face to be subject to user authentication.

[0127] As yet another example, the predetermined face condition is related to whether or not the face recognized from the video is the entire face. As shown in FIG. 6B, if the first face 2000a recognized from the video 620 does not correspond to the entire face of the user, but corresponds to a part of the face, the control unit 160 does not specify the first face 2000a as a face to be recognized as a target face for user authentication. In this case, if the fourth face 2000d does not have the above-mentioned predetermined size, the control unit 160 determines that the video 630 does not include a face to be recognized as a target face for user authentication.

[0128] However, if the user attempting authentication is too close, only a portion of that user's face will be recognized, and the faces of other users behind them will be determined to be the faces to be authenticated if they are an appropriate size. Therefore, if only a portion of the face of the user considered to be foremost (for example, the user with the largest face) is exposed, it is determined that there is no face to be subject to user authentication, and a guide to move the user back a little, or a guide to move the user so that their entire face is on the screen is output.

[0129] As yet another example, the predetermined face condition is related to whether the angle of the face recognized in the video is within a predetermined angle range based on a predetermined direction, which is related to whether the face faces a reference direction (e.g., the camera direction), i.e., whether the user faces directly in front of the camera.

[0130] For example, as shown in (c) of FIG. 6, if the angle of the fifth face 2000e recognized from the image 630 (e.g., rotated 90 degrees toward -y from the x direction, -90 degrees) does not fall within the range from the first angle (e.g., -20 degrees) to the second angle (e.g., +20 degrees) from the front direction (x) 641 of the camera 111, the control unit 160 does not identify the fifth face 2000e as a face to be subject to user authentication.

[0131] That is, only when the angle of a face recognized in an image satisfies a predetermined condition based on a predetermined direction, control unit 160 determines that the face satisfies a predetermined face condition.

[0132] In this way, in the present invention, in order to quickly and accurately recognize the user's face, faces that satisfy predetermined facial conditions are confirmed in the image captured by the camera 111.

[0133] As described above, when there are multiple predetermined facial conditions, the control unit 160 identifies a face contained in the video as a face to be subject to user authentication only if the face in the video satisfies all of the multiple facial conditions.

[0134] Also, when there are multiple faces in the video that satisfy the predetermined face condition, the control unit 160 specifies the largest face in the video as the face to be subject to user authentication. Alternatively, in some cases, when there are multiple faces in the video that satisfy the predetermined face condition, the control unit 160 specifies all of the multiple faces as the faces to be subject to user authentication.

[0135] Next, in the present invention, a process of requesting user authentication is performed based on the result of checking whether the face recognized from the image satisfies a predetermined face condition (S340).

[0136] When a face recognized from the video satisfies a predetermined face condition, the control unit 160 requests the server 200 to perform user authentication for the user corresponding to the face. In this case, the request for user authentication is transmitted from the system 100 to the server 200.

[0137] In contrast, if a face recognized from a video does not satisfy a predetermined facial condition, the control unit 160 does not transmit a request for user authentication to the server 200. In other words, if the result of checking whether or not a face recognized from a video satisfies the predetermined facial condition indicates that there is no face that satisfies the predetermined facial condition among the multiple faces, the control unit 160 does not transmit a request for user authentication based on the video to the server 200.

[0138] In this way, in the present invention, by controlling the user authentication process to be performed for faces that satisfy certain facial conditions, it is possible to prevent user authentication from being performed for faces of other users that are not the face of the user attempting user authentication, thereby improving the accuracy of user authentication. Also, by not repeating calculations for authentication unnecessarily, calculation resources are reduced and quick processing is possible.

[0139] In the present invention, the case where the user authentication process is performed in the user authentication server 200 has been described, but it goes without saying that the user authentication process may also be performed in the system 100 .

[0140] When a face recognized from the video satisfies a predetermined face condition, the control unit 160 transmits information about the recognized face to the server 200 so that the server 200 performs user authentication for the recognized face.

[0141] At this time, information about the user to be authenticated is transmitted to the server 200 together with a request for user authentication.

[0142] The information about the user to be authenticated is information about a face contained in the video, and includes at least one of: i) the video including the recognized face, ii) identification information (e.g., the ID of the face) regarding faces recognized from the video that meet specified facial conditions, and iii) facial information regarding faces recognized from the video that meet specified facial conditions.

[0143] Here, as described above, the face information includes at least one of the following: i) position information (pixel coordinate information) 511 regarding the face (bounding box) in the image; ii) face size information (corresponding to the size of the area the face occupies in the image, or the size (or width) of the area corresponding to the bounding box) 512; iii) face angle information (information regarding whether the face is facing a reference direction (e.g., camera direction); and iv) face feature information.

[0144] Needless to say, the control unit 160 may transmit only face information related to a specific user to be authenticated among the faces recognized from the video to the server 200. For example, the control unit 160 transmits facial feature information corresponding to the specific face to be authenticated to the server 200, instead of transmitting the video itself to the server 200. In this case, the server 200 performs user authentication based on the facial feature information received from the system 100.

[0145] On the other hand, in the system 100, when multiple faces are recognized from an image, information for identifying a specific face to be authenticated among the multiple faces may not be transmitted to the server 200. In this case, the server 200 performs user authentication for at least one of the multiple faces on its own. The server 200 performs user authentication for the face that occupies the largest area in the image among the multiple recognized faces. In this case as well, the server 200 performs user authentication only for the faces among the multiple faces that satisfy the above-mentioned predetermined face conditions.

[0146] On the other hand, in the server 200, based on a request for user authentication from the system 100, user authentication is performed for the face recognized from the video.

[0147] As described above, the control unit 230 included in the server 200 determines the similarity between information about the face of the user to be authenticated that is included in the video and the reference facial feature information stored in the user DB 221 of the server 200.

[0148] As an example of a method of determining the degree of similarity, the control unit 230 compares the face of the user to be authenticated that is included in the video with a reference face image stored in the user DB 221 to perform user authentication.

[0149] Here, comparing the reference face image with the face of the user to be authenticated contained in the video means comparing feature point information on the reference face image with feature point information on the face of the user recognized from the video. That is, the user DB has feature point information that defines the reference face image of the user according to facial features such as the eyes, nose, and mouth contained in the reference face image.

[0150] As another example of a similarity determination method, the control unit 230 included in the server 200 performs user authentication by comparing the similarity between a feature vector for the face of the user to be authenticated extracted from a face image and a feature vector included in the reference face feature information. Here, feature vectors for various reference faces are stored and exist in the user DB. The control unit 230 included in the server 200 searches (or extracts) a feature vector that is most similar to the feature vector for the face of the user to be authenticated from the user DB, and if the similarity between the searched feature vector and the feature vector for the face of the user to be authenticated is equal to or greater than a predetermined value (threshold), the control unit 230 processes the user as having been successfully authenticated.

[0151] When a person corresponding to the authentication target face is searched for in the user DB 221, the control unit 230 included in the server 200 completes user authentication for the authentication target face. In the present invention, the completion of user authentication for the authentication target face is also referred to as “successful user authentication.”

[0152] In contrast, if the degree of similarity between the information on the face of the user to be authenticated and the reference facial feature information stored in the user DB is low, the control unit 230 included in the server 200 cannot search for an accurate person corresponding to the face to be authenticated from the user DB. That is, the control unit 230 included in the server 200 cannot complete user authentication for the face to be authenticated. In the present invention, the inability to search for an accurate person corresponding to the face to be authenticated from the user DB 221 is also referred to as a "user authentication failure."

[0153] In addition, in the present invention, the user authentication is processed as successful only when the user corresponding to the authentication target face has authority for the event or function that is the target of the user authentication. For example, if the event or function that is the target of the user authentication is the control of a speed gate for permitting entry into a specific space, the server 200 processes the user authentication as successful only when the authentication target user has authority to enter the specific space.

[0154] When a person who matches a reference face image corresponding to the face to be authenticated is searched for in the user DB 221, the control unit 230 included in the server 200 extracts from the user DB 221 user information that matches the searched person.

[0155] Information relating to the authority of the user to be authenticated (or authority information) exists and is included in the user information of the user to be authenticated stored in the user DB 221.

[0156] Here, the authority for an event or function that is the subject of user authentication is interpreted in various ways. For example, the authority for an event or function that is the subject of user authentication is information on whether or not the authenticated user has the authority to access any function, device, or place, such as i) whether or not the user corresponding to the face to be authenticated (hereinafter, the authenticated user) has the authority to be provided with the event or function that is the subject of user authentication, ii) whether or not the authenticated user has the authority to access the event or function that is the subject of user authentication, iii) whether or not the authenticated user has the authority to enter a specific place, and iv) whether or not the authenticated user has the authority to control a specific electronic device.

[0157] For example, if the event or function subject to user authentication is a function of controlling the entry restriction gate 20 to an open state allowing the authenticated user to enter a specific location, as shown in (a) of Figure 4, the control unit 230 included in the server 200 determines whether the authenticated user has the authority to enter a specific location (e.g., a building) based on the user information of the authenticated user.

[0158] The control unit 230 included in the server 200 determines, based on the user information extracted from the user DB 221, whether or not the user corresponding to the face to be authenticated has authority for the event or function that is the subject of user authentication.

[0159] If the user corresponding to the face to be authenticated has authority for the event or function that is the subject of user authentication, the control unit 230 included in the server 200 completes the user authentication and determines that the user authentication has been successful.

[0160] In contrast, even if a person matching the reference face image corresponding to the face to be authenticated is searched for, if the control unit 230 included in the server 200 determines that the user corresponding to the face to be authenticated does not have authority for the event or function that is the subject of user authentication, the control unit 230 determines that the user authentication has failed.

[0161] On the other hand, the server 200 transmits the result of user authentication for the authentication target face to the system 100. Here, if the user authentication is successful, the server 200 transmits to the system 100 at least one of the user information (e.g., name, work department, etc.) corresponding to the authentication target face and a control command related to the event or function that is the target of the user authentication.

[0162] In addition, if user authentication is successful, the server 200 may transmit user information (e.g., name, department, etc.) corresponding to the face to be authenticated to the system 100, and transmit a control command related to the event or function that is the subject of user authentication to a device related to the event or function that is the subject of user authentication (e.g., the entry restriction gate 20).

[0163] Meanwhile, in the present invention, a process of controlling according to the result of user authentication for the authentication target face received from the server 200 is performed.

[0164] The control unit 160 of the system 100 performs different control depending on the result of the user authentication.

[0165] More specifically, the control unit 160 controls at least one of the display unit 140 and the audio output unit 150 so that different information is provided to the user to be authenticated, based on whether user authentication for the face to be authenticated has succeeded or failed.

[0166] When the user authentication for the face to be authenticated is successful, the control unit 160 provides information corresponding to the successful user authentication to the user to be authenticated via at least one of the display unit 140 and the audio output unit 150. Here, the control unit 160 provides customized information for the user to be authenticated by using the user information corresponding to the face to be authenticated.

[0167] 8(a) and 8(b), when user authentication for the first face 2000a and the second face 2000b is successful, the control unit 160 controls the display unit 140 so that information corresponding to the successful user authentication (e.g., "Welcome USER1!") 810, 820 is visually output on the display unit 140 together with the names of the users corresponding to the first face 2000a and the second face 2000b. The control unit 160 also controls the audio output unit 150 so that the information corresponding to the successful user authentication is audibly output by the audio output unit 150.

[0168] 8(c), when user authentication for the third face 2000c fails, the control unit 160 controls the display unit 140 so that information corresponding to the user authentication failure (e.g., "Oops! Please register as a user") 830 is visually output on the display unit 140. In addition, the control unit 160 controls the audio output unit 150 so that information corresponding to the user authentication success is audibly output by the audio output unit 150.

[0169] Furthermore, the control unit 160 performs control related to an event or function so that the event or function that is the subject of user authentication is executed based on the success or failure of user authentication for the face to be authenticated.

[0170] 4(a), the event or function that is the subject of user authentication may be one that controls the entry restriction gate 20 to a state in which the entry of the user to be authenticated to a specific location is possible or impossible. In this case, the control unit 160 controls the entry restriction gate 20 to be in either an open state or a closed state according to the result of the user authentication received from the server 200.

[0171] More specifically, if user authentication of the face to be authenticated is successful, the control unit 160 controls the entry restriction gate 20 to be in an open state for a predetermined time so that a user corresponding to the face to be authenticated can pass through the entry restriction gate 20. In addition, the control unit 160 controls the entry restriction gate 20 to be in a closed state after the predetermined time has elapsed so that the entry restriction function is executed.

[0172] On the other hand, if the user authentication for the face to be authenticated fails, the control unit 160 controls the entrance restriction gate 20 to be in a closed state so that the user corresponding to the face to be authenticated cannot pass through the entrance restriction gate 20.

[0173] Meanwhile, the entry restriction gate 20 controls the open state or the closed state based on a control command received from the server 200. The server 200 transmits the result of the user authentication not only to the system 100 but also to the entry restriction gate 20. The entry restriction gate 20 controls the open state based on the result of the user authentication received from the server 200.

[0174] In this way, in the present invention, by performing user authentication on faces that satisfy predetermined facial conditions, data processing required for user authentication can be efficiently managed.

[0175] On the other hand, in the present invention, before performing user authentication on an authentication target face that satisfies a predetermined face condition, a user's intention to authenticate, that is, whether or not the user corresponding to the authentication target face really wants to be authenticated, is grasped, and user authentication is performed only when the user has the intention to be authenticated. A method for grasping the user's intention to be authenticated will be specifically described below.

[0176] In the present invention, when a face recognized from an image satisfies a predetermined facial condition, it is determined whether or not the user of the face that satisfies the predetermined facial condition intends to perform user authentication, and user authentication is selectively performed based on the result of the determination.

[0177] When the control unit 160 determines that a predetermined face condition is satisfied and that the user corresponding to the authentication target face has the intention to be authenticated, it requests the server 200 to perform user authentication for the authentication target face.

[0178] In contrast, in the present invention, if a face recognized from an image does not satisfy predetermined facial conditions, or if it satisfies the predetermined facial conditions but it is determined that the user corresponding to the face has no intention of user authentication, then the server 200 is not requested to perform user authentication for the recognized face.

[0179] When the control unit 160 determines that the face is not a target face for authentication or that the user corresponding to the target face for authentication has no intention of user authentication (for example, in the case of an entry-restricted gate, the user is a passerby who has no intention of passing through the gate, or a user who has just come out of the entry-restricted gate), the control unit 160 controls so as to restrict a request for user authentication for the target face for authentication to the user authentication server 200.

[0180] The determination of user authentication of the user corresponding to the face to be authenticated is made based on information stored in the storage unit 120 regarding the user to be authenticated.

[0181] Here, the information stored in the storage unit 120 regarding the user to be authenticated includes at least one of an image including the face of the user to be authenticated and face information of the user to be authenticated included in the image.

[0182] As shown in FIG. 7A, the control unit 130 determines the intention of the user to be authenticated by using facial information of the user to be authenticated extracted from an image (e.g., a first image) 700 captured at the current time t0 and at least one of images 710, 720, and 730 captured at a time point prior to the current time t0 (past times, tp1 to tp3, etc.).

[0183] The control unit 160 checks whether or not face information matching the specific ID (USER1) of the user to be authenticated, extracted from the second image 710, third image 720, and fourth image 730 acquired at time points tp1, tp2, and tp3 prior to time point t0 at which the first image 700 in which the face 2000a to be authenticated was recognized, is present in the storage unit 120.

[0184] More specifically, the control unit 160 checks whether face information matching the same ID as the ID assigned to the face to be authenticated in the first image 700 exists in the storage unit 120 .

[0185] It will be understood that the face information matched to the specific ID is the face information relating to the face 2000a to which the specific ID has been assigned.

[0186] As described above, face information is various information related to a specific face (or face image) extracted from a video. For example, face information includes at least one of i) position information related to a face (face image), ii) size information related to a face (size (or width) of an area corresponding to a face image or a bounding box), iii) angle information related to a face, and iv) face feature information.

[0187] If there is facial information in the first image 700 that matches the ID assigned to the face to be authenticated 2000a, the control unit 160 calculates the amount of change in size and position of the face to be authenticated (or bounding box) based on the facial information of the face to be authenticated 2000a extracted from the first image 700 and the facial information of the face to be authenticated 2000a extracted from the images 710, 720, and 730 taken at a time before the first image 700.

[0188] Here, it will be understood that the amount of change in the size and position of the face to be authenticated means the degree of change in the size of the area corresponding to the face to be authenticated and the coordinates corresponding to the face to be authenticated in the multiple images 700, 710, 720, and 730 in which the face to be authenticated is recognized.

[0189] As an example, in FIG. 7A, if the size of the face to be authenticated 2000a in the first image 700 acquired at the current time t0 becomes relatively larger than the size of the face to be authenticated 2000a in the second image 710 acquired at the first past time tp1 (if the area width increases), the size difference (or area difference) of the face to be authenticated 2000a in each image is the amount of change in the size of the face to be authenticated.

[0190] 7B, the difference between the position (pixel coordinates) of the authentication target face 2000a in the first image 700 captured at the current time t0 and the position (pixel coordinates) of the authentication target face 2000a in the second image 710 captured at the first past time tp1 is the change in the position of the authentication target face. In addition, the speed at which the user approaches the system and the speed at which the user moves are also calculated using the difference between the image capture times and the change in size.

[0191] In this way, the control unit 160 calculates at least one of the amount of change in size and position of the face to be authenticated based on the face information, and determines the user authentication intention of the user corresponding to the face to be authenticated based on the calculated amount of change.

[0192] When the control unit 160 determines, based on the calculated amount of change, that the user corresponding to the face to be authenticated has approached the system 100 or the entry restriction gate 20, it determines that the user corresponding to the face to be authenticated has the intention of user authentication.

[0193] Specifically, when the size of the face to be authenticated increases over time, the control unit 160 determines that the user corresponding to the face to be authenticated is approaching the system 100 or the camera 111 with the intention of user authentication.

[0194] For example, as shown in (a) of Fig. 4, assume that multiple users 1000a, 1000b, and 1000c are lined up in front of the system 100 for user authentication. The users who intend to be authenticated move sequentially from a position relatively far from the system 100 or the entrance control gate 20 to a position relatively close to the system 100 or the camera 111.

[0195] Furthermore, as shown in FIG. 7A, when the control unit 160 determines that the size of the authentication target face 2000a has increased over time in the fourth image 730, the third image 720, the second image 710, and the first image 700 based on the amount of change in face size calculated from face information regarding the authentication target face 2000a at the current and past points in time, it determines that the user corresponding to the authentication target face 2000a has approached the system 100 or the camera 111 with the intention of user authentication.

[0196] On the other hand, when the control unit 160 determines, based on the calculated amount of change in position, that the user's face corresponding to the face to be authenticated has moved to a specific area (e.g., an area within a predetermined radius from the central point, see reference numeral 710 in FIG. 7B), it determines that the user corresponding to the face to be authenticated has the intention of user authentication.

[0197] Specifically, when the position of the face to be authenticated moves over time to a specific area (e.g., the central area of ​​the image), the control unit 160 determines that the user corresponding to the face to be authenticated has approached the system 100 or the entry restriction gate 20 with the intention of user authentication.

[0198] For example, a user with the intention of user authentication changes his / her posture and face position so that his / her face is located in a specific area of ​​the image (eg, within a predetermined radius from the center point 740 of the image) as shown in FIG. 7B.

[0199] In this case, as shown in FIG. 7B, when the control unit 160 determines that the authentication target face 2000a has moved to a specific point 740 over time, such as the fourth image 730, the third image 720, the second image 710, and the first image 700, based on the amount of change in the face position calculated from the facial information regarding the authentication target face 2000a at the current and past points in time, it determines that the user corresponding to the authentication target face 2000a has placed his or her face in the system 100 or the entry restriction gate 20 with the intention of user authentication.

[0200] In this way, in the present invention, facial information of the face to be authenticated extracted from at least one other image acquired at a time prior to the acquisition of the target image is used to determine whether or not the user corresponding to the face to be authenticated has the intention of user authentication.

[0201] Therefore, in the present invention, even if a face that satisfies certain facial conditions is recognized, by performing user authentication only on the face of a user who intends to be authenticated, it is possible to eliminate erroneous recognition in which user authentication is performed on the face of a user who does not intend to be authenticated, thereby improving the accuracy and reliability of the user authentication means.

[0202] On the other hand, even if user authentication is successful, user authentication may be processed multiple times for the same user.

[0203] For example, as shown in (a) of Figure 9, if a specific user (USER7) successfully passes user authentication and the entry restriction gate 20 is controlled to an open state, but the specific user (USER7) remains standing there without passing through the entry restriction gate 20, the camera 111 again captures an image 910 including the face 900 of the successfully authenticated user, as shown in (b) of Figure 9.

[0204] In this case, the server 200 performs user authentication again for the user (USER7) having the specific ID based on the information received from the system 100, and transmits the result of the user authentication to the system 100.

[0205] In this case, the control unit 160 controls the entry restriction gate 20 according to the result of user authentication for the face having the specific ID, and when the control unit 160 subsequently receives an additional result of user authentication for the face having the specific ID from the server 200, the control according to the additionally received result of user authentication is restricted.

[0206] After successful user authentication for a face having a specific ID (USER7), if user authentication for the same user is successful again within a predetermined time, as shown in (b) of Figure 9, the control unit 160 controls so that a function corresponding to successful user authentication is not executed even if user authentication for the successfully authenticated face 900 is successful again.

[0207] For example, as shown in (a) of FIG. 9, even if re-authentication of a face 910 having a specific ID (USER7) is successful based on an image 910 acquired at a second time point t1 within a predetermined time from a first time point t0, the control unit 160 controls the display unit 140 so that information corresponding to successful user authentication (e.g., “Welcome USER7!”) 920 is not output to the display unit 140, as shown in (b) of FIG. 9.

[0208] Furthermore, when a predetermined time has elapsed and user authentication for successful authentication face 910 has been successful again, control unit 160 performs control so that the target function corresponding to successful user authentication is executed again.

[0209] For example, assume that the successfully authenticated user must pass through the entrance restriction gate 20 again after going outside the entrance restriction gate 20. As shown in (a) and (c) of FIG. 9, when re-authentication of a face 910 having a specific ID (USER7) based on an image 920 acquired at a third time point t2, which is a predetermined time after the first time point t0, the control unit 160 controls the entrance restriction gate 20 to be opened again by outputting information 930 corresponding to the successful user authentication (e.g., "Welcome USER7! This is your second authentication.") to the display unit 140. Here, it goes without saying that the control of the entrance restriction gate 20 may be directly performed by the server 200 as described above.

[0210] On the other hand, in the present invention, when multiple users perform user authentication sequentially, the target functions executed based on the results of user authentication for the multiple users are controlled so as to be executed smoothly, without interruption, and without hitch.

[0211] In this way, in the present invention, the time term required for repeating user authentication and target functions for a plurality of users can be shortened, and the inconvenience felt by users due to user authentication can be eliminated.

[0212] Hereinafter, an example of the present invention will be described in which user authentication is performed so that the entrance restriction gate 20 is in an open state in order for a plurality of users to enter a specific space in sequence.

[0213] 10(a), when the distance a between the first user 1000a and the entry restriction gate 20 is very short, it takes longer for the first user 1000a to perform user authentication and for the entry restriction gate 20 to become open than it takes for the first user 1000a to move to the front of the entry restriction gate 20. In this case, the first user 1000a has to stop first to attempt user authentication and then wait secondarily until the entry restriction gate 20 becomes open, which is troublesome.

[0214] In the present invention, when user authentication for a user is completed, a predetermined face condition is determined based on a reference distance that must be maintained between the user and the system (terminal device, camera, or entry restriction gate 20) so that the user can immediately pass through the entry restriction gate 20 without waiting until the entry restriction gate 20 is opened. Here, the predetermined face condition includes a condition regarding the size of the user's face included in the image. The condition regarding the size of the area that the face occupies in the image is set in consideration of the user authentication processing time in the server 200, the control time of the entry restriction gate 20 according to the result of the user authentication, and the user's movement time.

[0215] When the size of a face included in the image is larger than a predetermined face size condition, the control unit 130 determines that the size of the face included in the image satisfies the predetermined face size condition. The predetermined face size condition is related to the distance between the user and the entry restriction gate 20, and the distance means a distance that allows enough time to perform all of the user authentication and the control of the entry restriction gate 20 to be opened before the user reaches the entry restriction gate 20.

[0216] In this manner, in the present invention, when a user is authenticated to enter a specific space, a minimum reference distance required of the user is set so that the user can enter the specific space without waiting for user authentication.

[0217] A method for allowing multiple users to pass through the restricted access gate 20 more conveniently and smoothly will now be described.

[0218] In the present invention, when user authentication of the faces of multiple users is successful in succession, the entry restriction gate 20 is controlled so that the open state of the entry restriction gate 20 is maintained until all of the multiple users have passed through, thereby reducing the operating time and unnecessary energy consumed by the entry restriction gate 20 repeatedly switching between the open and closed states.

[0219] As shown in FIG. 11, the control unit 160 recognizes multiple faces 2000a, 2000b, 2000c from images 1110, 1120, 1130 captured by the camera 111, and if user authentication performed by the server 200 results in successful user authentication for each of the multiple faces 2000a, 2000b, 2000c, the control unit 160 controls the entry restriction gate 20 to remain open until all of the multiple users corresponding to each of the multiple faces have passed through (from time t0 to time t2).

[0220] It will be understood that the multiple faces 2000a, 2000b, and 2000c recognized from the video are faces that satisfy the above-mentioned predetermined face conditions.

[0221] Here, the users corresponding to the multiple faces 2000a, 2000b, and 2000c are users who have the intention of user authentication, and the control unit 130 determines the intention of each of the multiple users based on the images captured at the current time and the past time, as described above.

[0222] In this way, in the present invention, when user authentication is successful for each of the multiple faces 2000a, 2000b, 2000c recognized from the video, the entry restriction gate 20 is controlled to remain open until all of the multiple users corresponding to each of the multiple faces have passed through (from time t0 to time t2).

[0223] Therefore, in the present invention, the time required for the entry restriction gate 20 to alternate between open and closed states each time a user passes through the entry restriction gate 20 is shortened, and the user can immediately pass through the entry restriction gate 20 once user authentication is complete.

[0224] Furthermore, in the present invention, the energy required for the entrance restriction gate 20 to repeatedly open and close can be reduced.

[0225] 12(a), a plurality of different entry restriction gates 20a, 20b may exist in the specific space 1. The system 100 of the present invention performs user authentication at each of the plurality of entry restriction gates 20a, 20b individually using images captured by different cameras 111a, 111b relating to the plurality of entry restriction gates 20a, 20b.

[0226] For example, the system 100 according to the present invention performs user authentication at the first entry restricted gate 20a using an image 1210 captured by the first camera 111a, and performs user authentication at the second entry restricted gate 20b using an image 1220 captured by the second camera 111b.

[0227] Meanwhile, a plurality of users who wish to pass through each of the first and second entrance restriction gates 20a and 20b are located around each of the first and second entrance restriction gates 20a and 20b.

[0228] A first user 1000a, a second user 1000b, and a third user 1000c who are attempting to pass through the first entry restriction gate 20a are lined up in this order in front of the first entry restriction gate 20a or the first camera 111a.

[0229] Further, a fourth user 1000d and a fifth user 1000e who are attempting to pass through the second entry restriction gate 20b are lined up in that order around the second entry restriction gate 20b or the second camera 111b.

[0230] In the present invention, control is performed so that user authentication for a specific user recognized from both video 1210 captured by first camera 111a and video 1220 captured by second camera 111b is not performed redundantly.

[0231] More specifically, when the user authentication server 200 requests a first user authentication for a face to be authenticated having a specific ID in order to enter a first entry restriction gate 20a among a plurality of entry restriction gates, and then requests a second user authentication for a face to be authenticated having a specific ID in order to enter a second entry restriction gate 20b within a predetermined time, the user authentication server 200 controls so as not to perform user authentication in response to the request for the second user authentication.

[0232] As shown in FIG. 12(a), in order to pass through the first entry restriction gate 20a, a first user 1000a attempts user authentication by a first camera 111a associated with the first entry restriction gate 20a.

[0233] In this case, as shown in (b) and (c) of Figures 12, first face 2000a of first user 100a is included in both image 1210 captured by first camera 111a and image 1220 captured by second camera 111b.

[0234] If the first face 2000a recognized from the image 1210 captured by the first camera 111a and the image 1220 captured by the second camera 111b both satisfy predetermined face conditions, the control unit 160 makes a redundant request for user authentication to the user authentication server 200.

[0235] That is, the server 200 receives from the system 100 a request for first user authentication for the first face 2000a recognized from the image 1210 captured by the first camera 111a, and a request for second user authentication for the first face 2000a recognized from the image 1220 captured by the second camera 111b.

[0236] As shown in FIG. 13, when a request for first user authentication and a request for second user authentication for the same first face 2000a are made within a specified time period in order to enter different entry restriction gates 20a, 20b, the server 200 processes the request for first user authentication that was requested first, and does not thereafter perform user authentication in response to the request for second user authentication.

[0237] In addition, the server 200 either does not transmit the result of the user authentication in response to the request for the second user authentication to the system 100, or transmits information notifying the system 100 that the user authentication has not been performed (or that it overlaps with the request for the first user authentication).

[0238] When the control unit 160 receives information from the server 200 based on a request for second user authentication regarding the second entry restriction gate 20b, notifying that the result of user authentication has not been received or that user authentication has not been performed (or that the request overlaps with the request for first user authentication), the control unit 160 performs control so as not to perform control according to the result of user authentication.

[0239] As an example, as shown in FIG. 13, when the server 200 performs a first user authentication for a first face 2000a recognized from an image 1210 captured by the first camera 111a at time t0, the system 100 controls the operation of the first entry restriction gate 20a for the first camera 111a, and controls the first display unit 140a for the first camera 111a to output the result of the user authentication or guidance information regarding the execution of a target function.

[0240] 13, when a request for second user authentication for a first face 2000a recognized from an image 1220 captured by the second camera 111b at time t1 is received, the server 200 does not perform user authentication in response to the request for second user authentication, because a request for first user authentication for the same first face 2000a has been received within a predetermined time. Here, the user authentication for the first user is processed as a failure.

[0241] In this way, in the present invention, even if the same face is recognized from images captured by different cameras 111a, 111b for multiple entry restriction gates 20a, 20b, user authentication for the duplicated users is not processed repeatedly, and it is possible to accurately process only the operation of the entry restriction gate where the user is located.

[0242] As described above, the face recognition system and control method thereof according to the present invention can acquire an image using a camera provided in a terminal that executes a face recognition function, and recognize a face from the image based on a predetermined face recognition algorithm. In this way, the present invention can accurately and quickly recognize a user's face based on the user's location near the terminal without direct input from the user. Therefore, the user can transmit his or her face to the terminal more conveniently and intuitively by being near the terminal.

[0243] Furthermore, the face recognition system and control method according to the present invention can determine whether a face recognized from an image is suitable for user authentication. In other words, in the present invention, by checking whether a face recognized from an image satisfies a predetermined face condition, if a face that is not suitable for user authentication is recognized, user authentication is not performed, thereby improving the efficiency of data processing.

[0244] Furthermore, the face recognition system and control method according to the present invention can perform user authentication only on users who satisfy certain facial conditions, and therefore can accurately recognize the face of a user attempting user authentication even when there are multiple users around a terminal device, thereby authenticating the user.

[0245] Furthermore, the face recognition system and control method thereof according to the present invention can request user authentication of a recognized face from a predetermined server based on whether or not the face recognized from a video satisfies predetermined face conditions. In this way, the face recognition system and control method thereof according to the present invention can reduce errors in performing user authentication on users who do not attempt user authentication or other users around a user attempting user authentication, thereby improving the accuracy and reliability of user authentication results, by performing user authentication on the face of a user who truly desires user authentication.

[0246] The computer-readable medium includes any kind of recording device in which data read by a computer system is stored. Examples of the computer-readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0247] The computer-readable medium also includes a storage location, such as a server or cloud storage location that can be accessed by a terminal via communication. In this case, a computer can download the program according to the present invention from the server or cloud storage location via wired or wireless communication.

[0248] Furthermore, in the present invention, the above-mentioned computer is a terminal equipped with a processor, that is, a CPU (Central Processing Unit), and the type of the processor may be any. Furthermore, the configurations and functions of the storage unit 120, the control unit 160, and the like of the present disclosure may be embodied and executed by, for example, a computer memory or a processor.

[0249] However, the above detailed description should not be interpreted as limiting in any respect, but should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the claims, and all modifications within the equivalent scope of the present invention are included in the present invention. [Explanation of symbols]

[0250] 1 Specific space 20 Entry Restriction Gate 20a 1st Restricted Entry Gate 20b 2nd access restricted gate 100 Facial Recognition System 111 Camera 120 Preservation Department 160 Control section 200 Servers 220 Preservation Department 400,610,620,630,910,1110,1120,1130,1210,1220 Video 510 Face Information 511 Location information 512 Size Information

Claims

1. A method for controlling a face recognition system that performs control based on a face recognition result, comprising: acquiring an image using a camera provided in a terminal that performs face recognition; a step of recognizing a face from the video based on a predetermined face recognition algorithm, in which a specific ID for identifying the recognized face is assigned to the recognized face, and facial information including at least one of position information, size information, and angle information related to the recognized face to which the specific ID is assigned is extracted from the video, and the extracted facial information is matched to the specific ID assigned to the recognized face and stored in a storage unit; confirming whether the face recognized from the video satisfies a predetermined face condition related to at least one of a position in the video where the recognized face is included, a size of an area in the video where the recognized face occupies, and an angle of the recognized face in the video; determining a user authentication intention of a user to which the specific ID corresponding to the recognized face is assigned, using face information matched with the specific ID extracted from at least one other image acquired at a time before the image was acquired and stored in the storage unit, and face information matched with the specific ID extracted from the image; a step of requesting a predetermined server to perform user authentication for the recognized face based on the recognized face satisfying the predetermined face condition; and performing control according to a result of user authentication for the recognized face based on the result of user authentication for the recognized face received from the server, The step of determining user authentication intent comprises: Calculating a change amount of a size and a position of a face corresponding to the specific ID from the image and the other image based on face information matched to the specific ID extracted from the image and face information matched to the specific ID extracted from the other image; A method for controlling a face recognition system, comprising determining a user authentication intention of the user to which the specific ID is assigned based on the amount of change.

2. In the step of requesting user authentication, If the recognized face satisfies the predetermined face condition and if it is determined in the step of determining whether the user has the intention of user authentication that the user has the intention of user authentication, a request is made to the server for user authentication of the recognized face; A control method for a face recognition system as described in claim 1, characterized in that even if the recognized face satisfies the specified face conditions, if it is determined in the step of determining the user authentication intention that the user does not have the intention of user authentication, the server is not requested to perform user authentication for the recognized face.

3. In the step of confirming, When a plurality of faces are recognized from the image, it is confirmed whether each of the recognized faces satisfies the predetermined face condition; Identifying a face that satisfies the predetermined facial condition from among the plurality of faces; In the step of requesting user authentication, 2. The method for controlling a face recognition system according to claim 1, wherein a user authentication request is made to the predetermined server for the identified face.

4. The identified face is a face included in a face area having a predetermined size or more among face areas occupied by each of the plurality of faces in the video, When there are a plurality of face areas in the image that are equal to or larger than the predetermined size, 4. The method for controlling a face recognition system according to claim 3, wherein the user authentication is performed for a face included in a largest face area among a plurality of face areas.

5. If the confirmation result indicates that no face satisfies the predetermined face condition among the plurality of faces, 4. The method for controlling a face recognition system according to claim 3, further comprising the step of not making a request for user authentication based on the image to the server.

6. In the step of performing control according to a result of the user authentication, Controlling an access restriction gate having either an open state or a closed state; The access restriction gate is When user authentication for the recognized face is successful, the entrance restriction gate is controlled to have an open state for a predetermined time so that a user corresponding to the recognized face can pass through the entrance restriction gate; 2. The method of claim 1, further comprising controlling the face recognition system to have a closed state so that an entry restriction function is executed after the predetermined time has elapsed.

7. Recognizing a plurality of faces from the video; When the user authentication performed by the server is successful for each of the plurality of faces, The access restriction gate is 7. The method according to claim 6, further comprising controlling the face recognition system to maintain an open state until all of a plurality of users corresponding to the plurality of faces have passed through.

8. In a state in which a plurality of different entry restriction gates are present in a specific space to restrict entry into the specific space, When a first user authentication is requested to the server for the recognized face having the specific ID in order to enter a first entry restriction gate among the plurality of entry restriction gates, and then, within a predetermined time, a second user authentication is requested to the recognized face having the specific ID in order to enter a second entry restriction gate among the plurality of entry restriction gates, 7. The method according to claim 6, wherein the server does not perform user authentication in response to the request for second user authentication.

9. In the step of controlling, Controlling the entrance restriction gate according to a result of user authentication for a face having the specific ID; The control method for a face recognition system according to claim 6, characterized in that, when an additional user authentication result for the face having the specific ID is subsequently received from the server, control is performed in a restricted manner at the entry restriction gate in accordance with the additionally received user authentication result.

10. A camera unit for acquiring images; A storage unit for storing the acquired image; a control unit that recognizes a face from the image based on a predetermined face recognition algorithm, and checks whether the face recognized from the image satisfies predetermined face conditions related to at least one of a position in the image where the recognized face is included, a size of an area in the image where the recognized face occupies, and an angle of the recognized face in the image; The control unit is assigning a specific ID to the recognized face for identifying the recognized face; extracting face information including at least one of position information, size information, and angle information regarding the recognized face to which the specific ID is assigned from the video; The extracted face information is matched to the specific ID assigned to the recognized face and stored in a storage unit; using face information matched to the specific ID extracted from at least one other video acquired prior to the acquisition of the video, which is stored in the storage unit, and face information matched to the specific ID extracted from the video, to determine a user authentication intention of a user to whom the specific ID corresponding to the recognized face is assigned; calculating an amount of change in size and position of a face corresponding to the specific ID from the video and the other video, based on the face information matched to the specific ID extracted from the video and the face information matched to the specific ID extracted from the other video; and determining a user authentication intention of the user to whom the specific ID is assigned, based on the amount of change; requesting a predetermined server to perform user authentication for the recognized face based on the recognized face satisfying the predetermined face condition; A face recognition system comprising: a server that performs control according to a result of user authentication for the recognized face based on the result of user authentication for the recognized face received from the server.

11. A camera unit for acquiring images; A storage unit for storing the acquired image; a control unit that recognizes a face from the video based on a predetermined face recognition algorithm, and checks whether the face recognized from the video satisfies predetermined face conditions related to at least one of a position in the video where the recognized face is included, a size of an area in the video where the recognized face occupies, and an angle of the recognized face in the video; a server that performs user authentication for the recognized face if the recognized face satisfies the predetermined face condition; The control unit is assigning a specific ID to the recognized face for identifying the recognized face; extracting face information including at least one of position information, size information, and angle information regarding the recognized face to which the specific ID is assigned from the video; The extracted face information is matched to the specific ID assigned to the recognized face and stored in a storage unit; using face information matched to the specific ID extracted from at least one other video acquired prior to the acquisition of the video, which is stored in the storage unit, and face information matched to the specific ID extracted from the video, to determine a user authentication intention of a user to whom the specific ID corresponding to the recognized face is assigned; calculating an amount of change in size and position of a face corresponding to the specific ID from the video and the other video, based on the face information matched to the specific ID extracted from the video and the face information matched to the specific ID extracted from the other video; and determining a user authentication intention of the user to whom the specific ID is assigned, based on the amount of change; A face recognition system characterized in that, based on a result of user authentication for the recognized face performed by the server, control is performed in accordance with the result of user authentication for the recognized face.

12. acquiring an image using a camera provided in a terminal that performs face recognition; a step of recognizing a face from the video based on a predetermined face recognition algorithm, in which a specific ID for identifying the recognized face is assigned to the recognized face, and facial information including at least one of position information, size information, and angle information related to the recognized face to which the specific ID is assigned is extracted from the video, and the extracted facial information is matched to the specific ID assigned to the recognized face and stored in a storage unit; confirming whether the face recognized from the video satisfies a predetermined face condition related to at least one of a position in the video where the recognized face is included, a size of an area in the video where the recognized face occupies, and an angle of the recognized face in the video; determining a user authentication intention of a user to which the specific ID corresponding to the recognized face is assigned, using face information matched with the specific ID extracted from at least one other image acquired at a time before the image was acquired and stored in the storage unit, and face information matched with the specific ID extracted from the image; a step of requesting a predetermined server to perform user authentication for the recognized face based on the recognized face satisfying the predetermined face condition; and performing control according to a result of user authentication for the recognized face based on the result of user authentication for the recognized face received from the server, The step of determining user authentication intent comprises: Calculating a change amount of a size and a position of a face corresponding to the specific ID from the image and the other image based on face information matched to the specific ID extracted from the image and face information matched to the specific ID extracted from the other image; A program that determines a user authentication intention of the user to which the specific ID is assigned based on the amount of change.

Images