Electronic device, and method for correcting similarity of feature vector of object included in image of electronic device

Abstract

In an electronic device and an operation method of the electronic device, according to one embodiment, the electronic device comprises: a memory for storing at least one computer program including instructions; and at least one processor, wherein, when individually or collectively executed by the at least one processor, the instructions can instruct the electronic device to: acquire, in response to a user authentication request, an image including a face; check, from the image including the face, a first feature vector based on a plurality of pieces of feature information of the face; check a first similarity between the first feature vector and a second feature vector, which is calculated on the basis of the plurality of pieces of feature information of the face included in the image pre-authenticated as a user of the electronic device; check a first difference vector indicating the difference between the first feature vector and a reference feature vector, which is predetermined and based on the plurality of pieces of feature information of the face of the user of the electronic device, and a second difference vector indicating the difference between the reference feature vector and the second feature vector; and correct the checked first similarity if a first characteristic value of the second difference vector is greater than the predetermined set value.

Description

Method for correcting the similarity of feature vectors of objects included in an image of an electronic device and an electronic device

[0001] The present disclosure relates to an electronic device and a method of operating the electronic device, and to a method for the electronic device to correct the similarity of feature vectors of objects included in an image.

[0002] An electronic device may perform user authentication using an image containing a face. User authentication may refer to an operation of verifying whether the face included in the image is a user registered with the electronic device. To perform user authentication, the electronic device may use a feature vector corresponding to the user's face. The feature vector may include multiple feature information representing features identifiable in the face. The feature information may be expressed numerically. The electronic device may perform user authentication by comparing the feature vector corresponding to the user's face with a first feature vector extracted from the face included in the image input during user authentication.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the foregoing is to be claimed as prior art related to this document, nor is it to be used to determine prior art.

[0004] When an electronic device performs user authentication in a situation where there is substantially the same variation of the state of the face (e.g., brightness, pose, or accessories (e.g., glasses, hat)), it may determine the authentication result differently for each user. The variation of the state of the face may refer to the difference compared to a reference state (e.g., brightness: 0dB, pose: neutral, accessories: none). The electronic device may perform user authentication by extracting a first feature vector from a face included in an image that reflects the variation of the state of the face when performing user authentication. Since the degree of difference in the first feature vector due to the variation of the state of the face varies depending on the user, the user authentication result due to the variation of the state of the face may vary depending on the user.

[0005] The electronic device can lower the threshold to increase the true acceptance rate (TAR) despite differences in feature vectors resulting from variations in the facial state. However, if the electronic device lowers the threshold, a problem may arise where the false acceptance rate (FAR) increases.

[0006] The technical problems to be solved in this document are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this invention belongs from the description below.

[0007] An electronic device according to one embodiment may include a memory that stores at least one computer program including instructions. An electronic device according to one embodiment may include at least one processor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may acquire an image including a face in response to a user authentication request. When the instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first feature vector based on a plurality of feature information of the face from the image including the face. When the instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first similarity between a second feature vector calculated based on a plurality of feature information of the face included in an image of a user of the electronic device who has already been authenticated as a user of the electronic device and the first feature vector. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first difference vector representing the difference between a reference feature vector and a first feature vector based on a plurality of feature information of the user's face of the electronic device that is predetermined, and a second difference vector representing the difference between the reference feature vector and the second feature vector. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may correct the identified first similarity if the first characteristic value of the second difference vector is greater than a predetermined set value.

[0008] A method of operation of an electronic device according to one embodiment may include an operation of acquiring an image including a face in response to a user authentication request. The method of operation of the electronic device may include an operation of identifying a first feature vector based on a plurality of feature information of the face from the image including the face. The method of operation of the electronic device may include an operation of identifying a first similarity between a second feature vector calculated based on a plurality of feature information of the face included in an image of a user of the electronic device who has already been authenticated as such and the first feature vector. The method of operation of the electronic device may include an operation of identifying a first difference vector representing the difference between a reference feature vector based on a plurality of feature information of the face of a user of the electronic device that is predetermined and the first feature vector, and a second difference vector representing the difference between the reference feature vector and the second feature vector. The method of operation of the electronic device may include an operation of correcting the identified first similarity when the first characteristic value of the second difference vector is greater than a predetermined set value.

[0009] A computer-readable recording medium according to one embodiment may store instructions that cause the electronic device to perform when executed by a processor of the electronic device. The instructions may cause the electronic device to acquire an image including a face in response to a user authentication request. The instructions may cause the electronic device to identify a first feature vector based on a plurality of feature information of the face from the image including the face. The instructions may cause the electronic device to identify a first similarity between a second feature vector calculated based on a plurality of feature information of the face included in an image of a user of the electronic device who has already been authenticated as such and the first feature vector. The instructions may cause the electronic device to identify a first difference vector representing the difference between a reference feature vector based on a predetermined plurality of feature information of the face of the user of the electronic device and the first feature vector, and a second difference vector representing the difference between the reference feature vector and the second feature vector. The instructions may cause the electronic device to correct the identified first similarity if the first feature value of the second difference vector is greater than a predetermined set value.

[0010] The electronic device can determine whether to correct the first similarity of a second feature vector based on a reference feature vector. The electronic device can perform similarity correction only on the feature vector among the second feature vectors for which it is determined to correct the first similarity based on the reference feature vector. By not performing similarity correction on the first similarity of all second feature vectors and instead performing similarity correction selectively, the electronic device can prevent the problem of FAR increasing due to similarity correction.

[0011] The electronic device can increase TAR even if the difference between the first feature vector and the reference feature vector due to changes in the face condition varies depending on the user, by determining the similarity correction value to be proportional to the degree of difference between the second feature vector and the reference feature vector.

[0012] The electronic device can prevent the problem of FAR increasing by determining the similarity correction value based on the similarity between the second characteristic value of the second feature vector and the second characteristic value of the first feature vector, and by determining the similarity correction value to be proportional to the first characteristic value of the second feature vector.

[0013] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0014] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0015] FIG. 1 is a block diagram of an exemplary electronic device capable of performing the operations described in this document.

[0016] FIG. 2 is a drawing illustrating an electronic device according to one embodiment.

[0017] FIG. 3 is a block diagram of an electronic device according to one embodiment.

[0018] FIG. 4 is a drawing illustrating an electronic device according to one embodiment.

[0019] FIG. 5 is an operation flowchart of an electronic device according to one embodiment.

[0020] FIG. 6 is a diagram illustrating a feature vector according to one embodiment.

[0021] FIG. 7 is a diagram illustrating the operation of an electronic device according to one embodiment determining a similarity correction value.

[0022] FIG. 8 is a diagram illustrating the recognition allowance range of an electronic device according to one embodiment.

[0023] FIG. 9 is a flowchart of an operation method of an electronic device according to one embodiment.

[0024] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0025] FIG. 1 is a block diagram of an exemplary electronic device (100) capable of performing the operations described in this document.

[0026] Referring to FIG. 1, the electronic device (100) may be one of various forms of electronic devices, such as a notebook (190), smartphones (191) having various form factors (e.g., a bar-type smartphone (191-1), a foldable-type smartphone (191-2), or a sliderable (or rollable)-type smartphone (191-3)), a tablet (192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 1 are illustrative only and are not intended to limit the implementations described or claimed herein. The electronic device (100) may be referred to as a mobile device, a user device, a multifunction device, a portable device, or a server.

[0027] The electronic device (100) may include components comprising at least one processor (110) (hereinafter referred to as processor (110)), at least one memory (120) (hereinafter referred to as memory (120)), at least one display (140) (hereinafter referred to as display (140)), at least one image sensor (150) (hereinafter referred to as image sensor (150)), at least one communication circuit (160) (hereinafter referred to as communication circuit (160)), and / or at least one sensor (170) (hereinafter referred to as sensor (170)). The components are merely exemplary. For example, the electronic device (100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuit, antenna, rechargeable battery, or input / output interface). For example, some components may be omitted from the electronic device (100). For example, some components may be integrated into a single component.

[0028] The processor (110) may be implemented as one or more integrated circuit (or circuitry) chips and may perform various data processing operations. The processor (110) may include at least one electrical circuit and may process instructions (or programs, data) stored in memory (120) individually or collectively in a distributed manner. The processor (110) may include a processor assembly comprising one or more processing circuits. The processor (110) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (100) (e.g., memory (120), display (140), image sensor (150), communication circuit (160), and / or sensor (170)). For example, the processor (110) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (110) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (110) may include one or more processing circuits. For example, the processor (110) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (110) may be included in a first chip of the electronic device (100), and at least another portion of the processor (110) may be included in a second chip of the electronic device (100) different from the first chip of the electronic device (100).

[0029] For example, the processor (110) may include a central processing unit (111), a graphics processing unit (112), a neural processing unit (113), an image signal processor (114), a display controller (115), a memory controller (116), a storage controller (117), a communication processor (118), and / or a sensor interface (119). These components of the processor (110) are merely exemplary. For example, the processor (110) may include other components. For example, some components of the processor (110) may be omitted from the processor (110). For example, some components of the processor (110) may be included as separate components of the electronic device (100) outside of the processor (110). For example, some components of the processor (110) (e.g., memory controller (116)) may be included in other components (e.g., at least part of memory (120), an interface (e.g. available for connection to at least one component of the electronic device (100)), a display (140) and / or an image sensor (150)).

[0030] The processor (110) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (120). The CPU (111) (or central processing circuit) may be configured to control the components of the processor (110) based on the execution of instructions stored in memory (120) (e.g., volatile memory (121) and / or non-volatile memory (122)). The GPU (112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (113) (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). An ISP (114) (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor (150) into a format suitable for a component within the electronic device (100) or a component of the processor (110). A display controller (115) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from a CPU (111), GPU (112), ISP (114), or memory (120) (e.g., volatile memory (121)) into a format suitable for a display (140). A memory controller (116) (or memory control circuit) may be configured to control reading data from the volatile memory (121) and writing data to the volatile memory (121). A storage controller (117) (or storage control circuit) may be configured to control reading data from the non-volatile memory (122) and writing data to the non-volatile memory (122).The CP (118) (communication processing circuit) may be configured to process data obtained from a component of the processor (110) into a format suitable for transmitting to another electronic device via the communication circuit (160), or to process data obtained from another electronic device via the communication circuit (160) into a format suitable for processing by the component of the processor (110). For example, the communication circuit (160) may include one or more communication circuits. The sensor interface (119) (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (100) and / or the state around the electronic device (100), obtained through the sensor (170), into a format suitable for the component of the processor (110).

[0031] Memory (120) may include one or more storage media (or one or more storage devices). For example, memory (120) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, a permanent memory such as flash memory, read-only memory (ROM) (e.g., non-volatile memory (122)), a semi-permanent memory such as random access memory (RAM) (e.g., volatile memory (121)), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (120) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As an example not limited to, the cache memory may be included within the processor (110). The memory (120) may be fixedly embedded within the electronic device (100) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (100).

[0032] For example, memory (120) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (110). For example, memory (120) may store instructions that can be called by an application programming interface (API). For example, memory (120) may store instructions within a library.

[0033] FIG. 2 is a drawing illustrating an electronic device according to one embodiment.

[0034] The electronic device (200) can perform user authentication using an image containing a face. User authentication may refer to an action of verifying whether the face included in the image is a user registered with the electronic device (200).

[0035] An electronic device (200) may use a feature vector corresponding to the face of a user (e.g., Person A, Person B) to perform user authentication. The feature vector may include multiple feature information representing features that can be identified in the face. The feature information may be converted into numerical values ​​and included in the feature vector. The electronic device (200) may perform user authentication by comparing the feature vector (210, 220, 230) corresponding to the user's face with a first feature vector extracted from a face included in an image input during user authentication.

[0036] For example, the electronic device (200) may determine (or calculate) the similarity between a feature vector corresponding to a user's face and a first feature vector, and determine user authentication as successful if the calculated similarity is greater than or equal to a threshold value. The threshold value may refer to a pre-set criterion for determining user authentication as successful. The electronic device (200) may calculate the similarity using cosine similarity and / or Euclidean distance.

[0037] The electronic device (200) can perform user authentication using a plurality of feature vectors (210, 220, 230) corresponding to a user's face. For example, the electronic device (200) can determine (or calculate) a similarity corresponding to each of the plurality of feature vectors (210, 220, 230) by comparing each of the plurality of feature vectors (210, 220, 230) corresponding to a user's face with a first feature vector. The electronic device (200) can determine that user authentication of the first feature vector is successful if at least one of the similarities obtained by comparing each of the plurality of feature vectors (210, 220, 230) corresponding to a user's face with the first feature vector is greater than or equal to a threshold value.

[0038] A plurality of feature vectors (210, 220, 230) may include a predetermined reference feature vector (210) based on a plurality of feature information of the face of a user of an electronic device and a second feature vector (220, 230) calculated based on a plurality of feature information of the face included in an image that has been authenticated as a user of an electronic device. In FIG. 2, two second feature vectors are shown, but there may be three or more second feature vectors.

[0039] The reference feature vector (210) may be a vector determined to be most similar to the user's face among the user's face images. According to one example, the electronic device (200) may determine the feature vector with the highest similarity to the user's other feature vectors as the reference feature vector. Similarity to other feature vectors may refer to the average value of similarities with each of the other feature vectors. The electronic device (200) may also determine the feature vector of the user's face (image) that was first registered as the reference feature vector.

[0040] A plurality of feature vectors (210, 220, 230) may reflect a plurality of feature information representing a face when at least some of the values ​​included in each of the plurality of feature vectors (210, 220, 230) are determined.

[0041] The second feature vector (220, 230) and the reference feature vector (210) correspond to the face of the same user (e.g., Person A), but at least one of the multiple values ​​included in the second feature vector may differ from the value included in the reference feature vector. Since the second feature vector (220, 230) and the reference feature vector (210) are vectors based on feature information extracted from the user's face included in the image, if at least some of the feature information representing the user's face included in the image is different, at least some of the values ​​included in each of the multiple feature vectors (210, 220, 230) may differ. At least some of the feature information included in the second feature vector may change as the state of the face (e.g., brightness, pose, or accessories (e.g., glasses, hat)) is deformed compared to the face corresponding to the reference feature vector.

[0042] Among the values ​​of the second feature vector (220, 230), the values ​​that differ from the values ​​of the reference feature vector (210) may represent feature information that differs from the feature information of the reference feature vector (210) among the feature information corresponding to the second feature vector (220, 230).

[0043] Since the electronic device (200) performs user authentication based on multiple feature vectors (210, 220, 230) that reflect different feature information, it can increase the TAR compared to when user authentication is performed based on a single feature vector (e.g., reference feature vector (210)).

[0044] The degree of difference (225, 235) between feature vectors according to a change in the state of the face (e.g., brightness, pose, or accessories (e.g., glasses, hat)) may vary from user to user. In one example, the degree of difference may refer to the magnitude of the difference vector between two vectors. The degree of difference (225, 235) between feature vectors according to a change in the state of the face may refer to the degree of difference between a reference feature vector (210) and a second feature vector (220, 230) obtained in a situation where there is a change in the state of the face. For example, even if the first change (e.g., brightness difference 3dB) is the same, the degree of difference (235, 265) of the feature vector according to the first change (e.g., brightness difference 3dB) may differ between the first user (e.g., Person A) and the second user (e.g., Person B). For example, even if the second variation (e.g., glasses) is the same, the degree of difference (225, 255) of the feature vector according to the second variation (e.g., glasses) may differ between the first user (e.g., person A) and the second user (e.g., person B).

[0045] Each of the second feature vectors (220, 230) may have a different degree of difference from the reference feature vector (210). For example, the degree of difference (225, 235) between the second feature vectors (220, 230) and the reference feature vector according to the state of the deformed face relative to the reference feature vector (210) may differ for each of the second feature vectors (220, 230).

[0046] The electronic device (200) may have limitations in increasing TAR if it does not reflect the degree of difference between the reference feature vector (210) and the second feature vectors (220, 230) in user authentication.

[0047] The electronic device (200) can determine different authentication results for each registered user (e.g., Person A, Person B) even if it performs user authentication based on substantially the same deformation of the face state. The electronic device (200) can perform user authentication by extracting a first feature vector (not shown) from a face included in an image that reflects a deformation of the face state (e.g., a brightness difference of 3 dB) when performing user authentication. Since the degree of difference in the feature vector based on whether the face state is deformed varies depending on the user, the user authentication result based on the deformation of the face state (e.g., a brightness difference of 3 dB) may vary depending on the user (e.g., Person A, Person B). For example, a first user (e.g., Person A) whose degree of difference (235) based on whether the first deformation (e.g., a brightness difference of 3 dB) is greater than the degree of difference (265) of other users (e.g., Person B) may have a greater difference in the first feature vector based on the first deformation (e.g., a brightness difference of 3 dB) than other deformations when performing user authentication. A first feature vector reflecting a first variation (e.g., a brightness difference of 3dB) may have a similarity level below a threshold value even when compared with a user's second feature vector (230) reflecting a first variation (e.g., a brightness difference of 3dB) because the degree of difference in the first feature vector due to the first variation (e.g., a brightness difference of 3dB) is greater than the degree of difference due to a variation of the other face condition (e.g., a second variation). Therefore, the authentication success rate due to the variation of the face condition may vary from user to user.

[0048] The electronic device (200) can lower the threshold value to increase TAR by compensating for the degree of difference (225, 235) of the feature vector according to the deformation of the face state. However, if the electronic device (200) lowers the threshold value, a problem may occur in which FAR increases. However, the electronic device (200) may lower the threshold value to increase TAR even if FAR increases.

[0049] The electronic device (200) can perform user authentication based on the degree of difference (225, 235) of the feature vector according to the deformation of the face state. For example, the electronic device (200) can correct the similarity of the first feature vector compared with the second feature vector (220, 230) to a larger value as the degree of difference (225, 235) between the second feature vector (220, 230) and the reference feature vector (210) increases. However, as the similarity is corrected to a larger value, a problem may occur in which the FAR increases.

[0050] The electronic device (200) can perform user authentication based on the direction of the difference vector between the first feature vector and the reference feature vector (first difference vector) and the direction of the difference vector between the second feature vector and the reference feature vector (second difference vector) to resolve the problem of FAR increasing as similarity is corrected to a larger value. For example, the electronic device (200) can increase the degree of correction as the direction of the difference vector between the first feature vector and the reference feature vector (first difference vector) and the direction of the difference vector between the second feature vector and the reference feature vector (second difference vector) are similar. From FIG. 3 onwards, an embodiment is described in which the electronic device (200) prevents the problem of FAR increasing while increasing TAR when performing user authentication.

[0051] FIG. 3 is a block diagram of an electronic device according to one embodiment.

[0052] An electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (200) of FIG. 2) (300) may include a processor (310) and memory (320). Various embodiments of this document may be implemented even if some of the illustrated configurations are omitted or replaced with other configurations. In addition to the illustrated configurations, the electronic device (300) may further include at least some of the configurations and / or functions of the electronic device (100) of FIG. 1, the electronic device (200) of FIG. 2, or the electronic device (400) of FIG. 4. At least some of each configuration of the illustrated (or unillustrated) electronic device (300) may be operatively, functionally, and / or electrically connected.

[0053] The processor (310) may include at least one processing circuitry, and the processor (310) may include at least one processor (310). For example, the processor (310) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core. The operations described in FIGS. 1 to 9 may be performed individually or collectively by at least one processor (310) included in the processor (310). The processor (310) may perform the operations of the face detection unit (410), the face anti-spoofing (FAS) unit (420), the feature vector extraction unit (430), the face registration unit (431), the face authentication unit (440), the similarity calculation unit (450), and the similarity correction unit (460) included in FIG. 4.

[0054] The memory (320) can store at least one computer program, and at least one computer program may include instructions that can be executed by the processor (310). The operations described in FIGS. 1 through 9 can be performed according to the execution of instructions contained in the memory (320).

[0055] FIG. 4 is a drawing illustrating an electronic device according to one embodiment.

[0056] In the following embodiments, the operations of the electronic device (400) of FIG. 4 may be performed in a secure area implemented in the electronic device (400). By performing the operations of FIG. 4 in the secure area, the electronic device (400) can restrict external access to the user's fingerprint image.

[0057] For example, the secure area may be an area separately provided within the processor (e.g., the processor (110) of FIG. 1 or the processor (310) of FIG. 3). The secure area may be an area separately provided outside the processor (110) (e.g., an embedded secure element (eSE), a secure processor). For example, the secure area is ARM TM Trustzone developed by Saga TM This may apply. For example, the security area can be implemented as a hypervisor.

[0058] The electronic device (400) may include a face detection unit (410), a face anti-spoofing (FAS) unit (420), a feature vector extraction unit (430), a face registration unit (431), a face authentication unit (440), and a feature vector database (470). The face authentication unit (440) may include a similarity calculation unit (450) and a similarity correction unit (460).

[0059] The feature vector database (470) may include a reference feature vector (210), which is a feature vector of a face determined to be most similar to the user's face among the user's face images, and at least one second feature vector, which is a feature vector of the user's face. The feature vector database (470) may be stored in the memory (320) of the electronic device (400) or stored on an external server.

[0060] The electronic device (400) can receive an image containing a face. The electronic device (400) may include a camera (e.g., image sensor (150) of FIG. 1) and / or a sensor (e.g., sensor (170) of FIG. 1) for receiving an image containing a face. The electronic device (400) can perform user authentication and / or face registration.

[0061] The face detection unit (410) can detect a face from an image containing a face. Detecting a face may refer to detecting an area corresponding to a face among the areas included in the image. The face detection unit (410) can transmit the detected face to the FAS unit (420).

[0062] The FAS unit (420) can check whether the detected face is forged. If the detected face is not forged, the FAS unit (420) can transmit the face to the feature vector extraction unit (430). If the detected face is forged, the FAS unit (420) can determine that user authentication has failed and not transmit the face to the feature vector extraction unit. According to one example, the FAS unit (420) can check whether the detected face is forged based on the feature vector stored in the feature vector database (470) or check whether the face is forged using a pre-trained artificial intelligence model.

[0063] The feature vector extraction unit (430) can extract a first feature vector, which is a feature vector of the detected face, from the detected face. The feature vector may include a plurality of feature information representing features that can be identified on the face. For example, the feature information may be determined based on landmark points on the face (e.g., location of eyes, nose, mouth, jawline), distances between landmark points, skin texture information (e.g., texture, wrinkles, blemishes, spots), skin color information (e.g., color histogram), the shape of the face, and / or the symmetry of the face. However, the feature information is not limited to the above examples and may be determined based on information that can be identified on the face. Each feature information may be converted into a numerical value and included in the feature vector.

[0064] The feature vector extraction unit (430) may use a face recognition model to extract a first feature vector from a detected face. The face recognition model may be an artificial intelligence (e.g., deep learning) model trained to extract a feature vector (e.g., [f1 f2 f3 쪋 fN] (e.g., [0.03 0.21 0.17 쪋 0.2]) which is a coordinate in a multidimensional space from a detected face. The feature vector extraction unit (430) may transmit the extracted first feature vector to the face authentication unit (440).

[0065] The face authentication unit (440) can perform authentication of the first feature vector by comparing the first feature vector and the user's feature vector (e.g., reference feature vector, second feature vector) included in the feature vector database (470). The face authentication unit (440) can perform authentication of the first feature vector by comparing each of the user's feature vectors (e.g., reference feature vector, second feature vector) included in the feature vector database (470) with the first feature vector. The face authentication unit (440) can determine that the authentication of the first feature vector is successful by comparing it with at least one of the user's feature vectors (e.g., reference feature vector, second feature vector) included in the feature vector database (470).

[0066] The face authentication unit (440) may determine that the authentication of the first feature vector is successful if the first similarity calculated by the similarity calculation unit (450) and / or the second similarity corrected by the similarity correction unit (460) is greater than or equal to a threshold value. According to one example, the face authentication unit (440) may determine that the authentication of the first feature vector is successful if the first similarity calculated by the similarity calculation unit (450) is greater than or equal to a threshold value, and control the similarity correction unit (460) not to perform similarity correction. The face authentication unit (440) may control the similarity correction unit (460) to perform similarity correction if the first similarity calculated by the similarity calculation unit (450) is less than or equal to a threshold value. According to another example, the face authentication unit (440) may omit the operation of checking whether the first similarity calculated by the similarity calculation unit (450) is greater than or equal to a threshold value, and check whether the second similarity corrected by the similarity correction unit is greater than or equal to a threshold value.

[0067] This disclosure describes only the authentication of a first feature vector in comparison with a second feature vector, but the same applies when comparing with a reference feature vector. Additionally, the second feature vector may refer to a feature vector that is not the reference feature vector among the feature vectors included in the feature vector database (470).

[0068] The similarity calculation unit (450) can determine (or calculate) the similarity between the second feature vector and the first feature vector. The similarity calculation unit (450) can calculate the similarity using cosine similarity and / or Euclidean distance. For example, the similarity calculation unit (440) calculates the cosine similarity between the second feature vector and the first feature vector, and the calculated similarity can be determined as a value between 0.0 and 1.0.

[0069] The similarity correction unit (460) can determine a similarity correction value based on a first feature vector, a reference feature vector, and a second feature vector. The similarity correction unit (460) can determine a second similarity by adding a similarity correction value to the first similarity. The operation of the similarity correction unit (460) determining whether to correct the similarity and determining the similarity correction value will be explained in FIGS. 5 to 7.

[0070] The face registration unit (431) may include the first feature vector determined as successful authentication by the face authentication unit (440) in the feature vector database (470). Alternatively, the face registration unit (431) may include the first feature vector in the feature vector database (470) in response to user input instructing to register a face even if the face authentication unit (440) does not determine successful authentication.

[0071] The face register (431) can redetermine a reference feature vector when the first feature vector is included in the feature vector database (470). The face register (431) can calculate the similarity between the first feature vector, the reference feature vector, and the second feature vector, and determine the feature vector with the highest similarity to other feature vectors as the reference feature vector. The similarity to other feature vectors may refer to the average value of the similarities with each of the other feature vectors.

[0072] FIG. 5 is an operation flowchart of an electronic device according to one embodiment.

[0073] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0074] According to one embodiment, operations 510 to 552 may be understood to be performed in a processor (e.g., processor 110 of FIG. 1, processor 310 of FIG. 3) of an electronic device (e.g., electronic device (100) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4).

[0075] According to one embodiment, an electronic device (e.g., electronic device (100) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4) can calculate a first similarity between a first feature vector and a second feature vector in operation 510. The electronic device (400) can determine (or calculate) the similarity between the second feature vector and the first feature vector. The electronic device (400) can calculate the similarity using cosine similarity and / or Euclidean distance. For example, the electronic device (400) calculates the cosine similarity between the second feature vector and the first feature vector, and the calculated similarity can be determined as a value between 0.0 and 1.0.

[0076] According to one embodiment, the electronic device (400) can calculate a first characteristic value of a second difference vector representing the difference between a second feature vector and a reference feature vector in operation 520. The first characteristic value of the second difference vector may be determined based on the degree of difference between the reference feature vector and the second feature vector. According to one example, the electronic device (400) can calculate a second difference vector representing the difference between the second feature vector and the reference feature vector for each of the second feature vectors. The first characteristic value of the second difference vector may be determined by the magnitude of the second difference vector.

[0077] According to one embodiment, the electronic device (400) can check whether the first characteristic value of the second difference vector is greater than or equal to a set value in operation 530.

[0078] The electronic device (400) can correct the first similarity to the second similarity in operation 540 when the first characteristic value of the second difference vector is greater than or equal to the set value. The electronic device (400) can correct the first similarity of the second feature vector when the first characteristic value of the second difference vector, which is the degree of difference between the second feature vector and the reference feature vector, is greater than or equal to the set value. For example, the electronic device (400) can correct the first similarity of the second feature vector only when the magnitude of the second difference vector is greater than or equal to the set value. The electronic device (400) can correct the first similarity to the second similarity by adding a similarity correction value to the first similarity.

[0079] The electronic device (400) can determine a similarity correction value based on a first characteristic value of a second difference vector, which is the degree of difference between a reference feature vector and a second feature vector. For example, the electronic device (400) can determine the similarity correction value to be proportional to the degree of difference between the second feature vector and the reference feature vector (e.g., the magnitude of the second difference vector).

[0080] The electronic device (400) can determine the second characteristic value of the second difference vector (e.g., the direction of the second difference vector) and the second characteristic value of the first difference vector (e.g., the direction of the first difference vector). The first difference vector may refer to a difference vector representing the difference between the first feature vector and the reference feature vector. The electronic device (400) can determine a similarity correction value based on the similarity between the second characteristic value of the second feature vector and the second characteristic value of the first difference vector. The second characteristic value of the second difference vector may be determined as the direction of the second difference vector representing the difference between the second feature vector and the reference feature vector.

[0081] According to one embodiment, the electronic device (400) can determine whether the first similarity or the second similarity in operation 550 is greater than or equal to a threshold value. The threshold value may be a value indicating a match with the user's face.

[0082] For example, the electronic device (400) can check whether the second similarity is greater than or equal to a threshold value when similarity correction is performed, and whether the first similarity is greater than or equal to a threshold value when similarity correction is not performed.

[0083] According to one embodiment, the electronic device (400) may determine the user authentication result of the first feature vector as authentication success in operation 551 when the first similarity or the second similarity is greater than or equal to a threshold value.

[0084] According to one embodiment, the electronic device (400) may determine the user authentication result of the first feature vector as an authentication failure in operation 552 when the first similarity or the second similarity is less than a threshold value.

[0085] FIG. 6 is a diagram illustrating a feature vector according to one embodiment.

[0086] A feature vector database (e.g., the feature vector database (470) of FIG. 4) may include a reference feature vector (601) and a second feature vector (610, 620, 630). The reference feature vector (601) and the second feature vector (610, 620, 630) may be vectors determined based on feature information (e.g., [f1 f2 f3 ↳ fN] (e.g., [0.03 0.21 0.17 ↳ 0.2]). Although shown on the axes of f1, f2, and f3 in FIG. 6, the reference feature vector (601) and the second feature vector (610, 620, 630) may refer to three or more multidimensional coordinates.

[0087] The reference feature vector (601) may refer to a feature vector of a face determined to be most similar to the user's face among the user's face images. The reference feature vector (601) may be set by user input. For example, the reference feature vector (601) may be a feature vector corresponding to the user's face (image) entered by the user when first registering the face.

[0088] The second feature vector (610, 620, 630) may refer to a feature vector among the feature vectors of the user's face excluding the reference feature vector (601).

[0089] When performing user authentication, an electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (300) of FIG. 3, and the electronic device (400) of FIG. 4) may perform user authentication for each of the reference feature vector (601) and the second feature vectors (610, 620, 630). The electronic device (400) may calculate a first similarity for each of the reference feature vector (601) and the second feature vectors (610, 620, 630) by comparing each of the reference feature vector (601) and the second feature vectors (610, 620, 630) with a first feature vector, and determine whether to correct the first similarity. However, the first similarity of the reference feature vector (601) may not be subject to similarity correction.

[0090] The electronic device (400) can determine whether to correct the first similarity of the second feature vector (610, 620, 630) based on the reference feature vector (601). The electronic device (400) can correct the first similarity of the second feature vector when the first characteristic value of the second difference vector (e.g., the magnitude of the second difference vector (611, 621, 631)), which is the degree of difference between the second feature vector and the first feature vector, is greater than or equal to a set value. Correcting the first similarity of the second feature vector may refer to determining the second similarity by adding a similarity correction value to the first similarity.

[0091] According to one example, the electronic device (400) can calculate a second difference vector (611, 621, 631) between the second feature vector and the reference feature vector (601) for each of the second feature vectors. The electronic device (400) can determine a first characteristic value of the second feature vector as a value corresponding to the magnitude of the second difference vector between the second feature vector and the reference feature vector (601), and can determine whether to correct the first similarity based on the first characteristic value of the second feature vector. For example, the electronic device (400) can decide to correct the first similarity of the second feature vector only when the magnitude of the second difference vector (611, 621, 631) is greater than or equal to a set value.

[0092] According to another example, the electronic device (400) may calculate the similarity between a reference feature vector (601) and a second feature vector (610, 620, 630), and determine to correct the first similarity of the second feature vector only when the calculated similarity is less than a reference value.

[0093] The setting values ​​and reference values ​​of the present disclosure may be set by the manufacturer of the electronic device.

[0094] The electronic device (400) can perform similarity correction only on the feature vectors among the second feature vectors (610, 620, 630) that are determined to correct the first similarity based on the reference feature vector (601). The electronic device (400) can prevent the problem of FAR increasing due to similarity correction by selectively performing similarity correction without performing similarity correction on the first similarity of all second feature vectors.

[0095] FIG. 7 is a diagram illustrating the operation of an electronic device according to one embodiment determining a similarity correction value.

[0096] An electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (300) of FIG. 3, and the electronic device (400) of FIG. 4) can determine a similarity correction value based on a first feature vector (702), a reference feature vector (701), and a second feature vector (720). The electronic device (400) can determine a second similarity by adding a similarity correction value to the first similarity.

[0097] The electronic device (400) can identify a first difference vector representing the difference between a reference feature vector (701) based on multiple feature information of a user's face of a predetermined electronic device and the first feature vector, and a second difference vector (N1, N2) representing the difference between the reference feature vector (701) and the second feature vector (710, 720).

[0098] According to one example, the electronic device (400) can determine the magnitude of a second difference vector (N2), which represents the difference between a second feature vector (720) and a reference feature vector (701), as the first feature value of the second feature vector (720).

[0099] The electronic device (400) may determine a similarity correction value based on a first characteristic value of a second difference vector (N1, N2), which is the degree of difference between a reference feature vector (701) and a second feature vector (710, 720). For example, the electronic device (400) may determine the similarity correction value to be proportional to the first characteristic value of the second difference vector (N1, N2), which is the degree of difference between the second feature vector (710, 720) and the reference feature vector (701) (e.g., the magnitude of the second difference vector (N1, N2)). The second feature vector (710, 720) may include at least one feature information that reflects a variation in the state of the face relative to the face corresponding to the reference feature vector (e.g., brightness, pose, or accessory (e.g., glasses, hat)). The second feature vector (710, 720) corresponds to the same user's face, but at least one of the multiple values ​​included in the second feature vector (710, 720) may differ from one another. Accordingly, the first characteristic value of the second difference vector (N1, N2) (e.g., the magnitude of the second difference vector (N1, N2)) may differ for each second feature vector (710, 720). The electronic device (400) can determine a similarity correction value as large as the first characteristic value of the second difference vector (N1, N2) (e.g., the magnitude of the second difference vector (N1, N2)) is large, and can determine a similarity correction value as small as the first characteristic value of the second difference vector (N1, N2) (e.g., the magnitude of the second difference vector (N1, N2)) is small. However, the electronic device (400) may not perform correction of the second feature vector (710, 720) when the first characteristic value of the second difference vector (N1, N2) (e.g., the magnitude of the second difference vector (N1, N2)) is less than or equal to a set value.

[0100] The electronic device (400) can increase TAR even if the degree of difference between the first feature vector and the reference feature vector due to the deformation of the face state varies depending on the user, by determining the similarity correction value to be proportional to the degree of difference between the second feature vector (720) and the reference feature vector (701). The electronic device (400) can determine different authentication results for each user even if user authentication is performed based on substantially the same deformation of the face state. The electronic device (400) can perform user authentication by extracting the first feature vector from the face included in the image reflecting the deformation of the face state at the time of user authentication. Since the degree of difference between the first feature vector and the reference feature vector due to the deformation of the face state varies depending on the user, the user authentication result based on the deformation of the face state may vary depending on the user. The electronic device (400) can increase TAR by determining the second similarity to be greater than the first similarity, even if the difference between the user's feature vector (e.g., second feature vector, first feature vector) and reference feature vector according to a specific deformation condition (e.g., brightness difference of 3dB) is greater than other deformations of the face condition.

[0101] However, the electronic device (400) may increase the FAR by determining the similarity correction value to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the second difference vector (N2) representing the difference between the second feature vector (720) and the reference feature vector (701). The electronic device (400) may also determine the user authentication result of the first feature vector of another person as successful by determining the similarity correction value to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the difference vector (N2) between the second feature vector (720) and the reference feature vector (701) even when performing user authentication of the first feature vector of another person. Below, an operation is described in which the electronic device (400) determines the similarity correction value based on the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector, thereby preventing the problem in which the FAR may increase by determining the similarity correction value to be proportional to the first characteristic value of the second difference vector.

[0102] The electronic device (400) can determine the second characteristic value of the second difference vector (N2) and the second characteristic value of the first difference vector (NP). The second characteristic value can be determined based on a deformation of the face state that causes a difference from the reference feature vector (701). The second characteristic value of the second difference vector (N2) can be determined based on a deformation of the face state corresponding to the second feature vector (720) that causes a difference from the reference feature vector (701). Thus, the second characteristic value of the second difference vector (N2) can represent a deformation of the face state corresponding to the second feature vector (720). For example, if the second feature vector (720) is obtained according to a third deformation (e.g., a hat), the second characteristic value of the second difference vector (N2) can reflect the third deformation (e.g., a hat). The second characteristic value of the first difference vector (702) may represent a deformation of the face state corresponding to the feature information included in the first feature vector (702). For example, if the second feature vector (720) is obtained according to the fourth deformation, the second characteristic value of the second difference vector (N2) may reflect the fourth deformation.

[0103] The electronic device (400) may determine a similarity correction value based on the similarity between the second characteristic value of the second difference vector (N2) and the second characteristic value of the first difference vector (NP). For example, the electronic device (400) may determine the similarity to be proportional to the similarity between the second characteristic value of the second difference vector (N2) and the second characteristic value of the first difference vector (NP). The similarity between the second characteristic value of the second difference vector (N2) and the second characteristic value of the first difference vector (NP) may refer to cosine similarity. The similarity between the second characteristic value of the second difference vector (N2) and the second characteristic value of the first difference vector (NP) may correspond to the degree of similarity between the deformation of the face state reflected in the second characteristic vector (720) and the deformation of the face state reflected in the first characteristic vector (702).

[0104] According to one embodiment, the electronic device (400) may determine a second characteristic value of the second difference vector (N2) based on the direction of the second difference vector (N2) representing the difference between the second feature vector (720) and the reference feature vector (701). For example, the electronic device (400) may determine that the second characteristic value of the second difference vector (N2) has a value equal to the direction of the second difference vector (N2). According to one example, the second characteristic value of the second difference vector (N2) may be in the form of a vector representing the direction.

[0105] According to one embodiment, the electronic device (400) may determine a second characteristic value of the first difference vector (NP) based on the direction of the first difference vector (NP) representing the difference between the first feature vector (702) and the reference feature vector (701). For example, the electronic device (400) may determine the second characteristic value of the first difference vector (NP) to have the same value as the direction of the first difference vector (NP). According to one example, the second characteristic value of the first difference vector (NP) may be in the form of a vector representing the direction.

[0106] The electronic device (400) may determine the similarity correction value to be proportional to the similarity of the direction of the second difference vector (N2) representing the difference between the second feature vector (720) and the reference feature vector, and the direction of the first difference vector (NP) representing the difference between the first feature vector (702) and the reference feature vector (701). The similarity between the direction of the second difference vector (N2) and the direction of the first difference vector (NP) may be determined to be a larger value as the angle between the second difference vector (N2) and the first difference vector (NP) becomes smaller. According to one embodiment, the electronic device (400) may determine the similarity correction value to 0 when the similarity between the second difference vector (N2) of the second feature vector (720) and the reference feature vector (701) and the first difference vector (NP) of the first feature vector (702) and the reference feature vector (701) is less than a set value.

[0107] The electronic device (400) can prevent the problem of FAR increasing as the similarity correction value is determined to be proportional to the first characteristic value of the second feature vector (e.g., the magnitude of the second difference vector (N2) of the second feature vector (720) and the reference feature vector) by determining the similarity correction value based on the similarity of the second characteristic value of the second feature vector (e.g., the direction of the second difference vector (N2) of the second feature vector (720) and the reference feature vector) and the second characteristic value of the first difference vector (e.g., the direction of the first difference vector (NP) of the first feature vector (702) and the reference feature vector (701). For example, when the second feature vector (720) corrects the first similarity by reflecting only the first characteristic value, the recognition allowance range (721) may be larger than that of another second feature vector (710), but as the first similarity is corrected by reflecting both the first characteristic value and the second characteristic value, the recognition allowance range (721) may be adjusted to a limited recognition allowance range (722).

[0108] According to one embodiment, the electronic device (400) may determine a similarity correction value proportional to the first similarity. By setting the second similarity higher as the first similarity increases, the electronic device (400) can prevent the problem of the first feature vector with low first similarity succeeding in user authentication and the FAR increasing.

[0109] According to one example, the similarity correction value can be expressed as [Equation 1]. In [Equation 1], a is a constant, I is a reference feature vector, is the second feature vector, is the first feature vector, Is It can refer to the similarity of.

[0110]

[0111] According to one example, the similarity correction value can be expressed as in [Equation 2]. In [Equation 1], a and b are constants, and I is a reference feature vector, is the second feature vector, is the first feature vector, Is It can refer to the similarity of. For example, the similarity correction value is, In this case, it can be expressed as [Equation 2].

[0112]

[0113] [Equation 1] and [Equation 2] represent only similarity correction values ​​determined by the electronic device (400) according to one example, and it should be understood that the similarity correction values ​​of the present disclosure are not limited to the similarity correction values ​​of [Equation 1] or [Equation 2].

[0114] FIG. 8 is a diagram illustrating the recognition allowance range of an electronic device according to one embodiment.

[0115] The recognition acceptance range (e.g., recognition acceptance range before correction (810, 820), recognition acceptance range after correction (815, 825)) may refer to a range in which the electronic device determines the user authentication result of the first feature vector as successful. The electronic device (e.g., the electronic device of FIG. 1 (100), the electronic device of FIG. 2 (200), the electronic device of FIG. 3 (300), the electronic device of FIG. 4 (400)) may determine the user authentication result of the first feature vector as successful if the first feature vector is included in the recognition acceptance range (810, 820, 815, 825).

[0116] The electronic device (400) can extend the recognition allowance range (810, 820) corresponding to some second feature vectors by determining the similarity correction value based on the first characteristic value of the second difference vector (e.g., the magnitude of the second difference vector), which is the degree of difference between the reference feature vector and the second feature vector. Accordingly, a user's first feature vector (801) that is not included in the recognition allowance range (810) before similarity correction of a specific second feature vector may be included in the recognition allowance range (815) after similarity correction.

[0117] The electronic device (400) can limit the direction of expansion of the recognition allowance range (815, 825) corresponding to some second feature vectors by determining the similarity correction value based on the similarity between the second feature value of the second difference vector and the second feature value of the first difference vector. For example, the recognition allowance range (815, 825) corresponding to some second feature vectors can be limited according to the second feature vector and the second difference vector of the reference feature vector. Thus, it is possible to limit the first feature vector (802) of another person, which may be included when similarity correction is based only on the first feature value of the second feature vector, so that it is not included in the recognition allowance range (815).

[0118] FIG. 9 is a flowchart of an operation method of an electronic device according to one embodiment.

[0119] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0120] According to one embodiment, operations 910 to 950 may be understood to be performed in a processor (e.g., processor 110 of FIG. 1, processor 310 of FIG. 3) of an electronic device (e.g., electronic device (100) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4).

[0121] An electronic device (e.g., electronic device (100) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4) can acquire an image including a face in operation 910. The electronic device (400) can receive an image including a face. The electronic device (400) may include a camera and / or sensor for receiving an image including a face. The electronic device (400) can perform user authentication and / or face registration.

[0122] A feature vector database (e.g., the feature vector database (470) of FIG. 4) may include a reference feature vector and a second feature vector. The reference feature vector and the second feature vector may be vectors determined based on feature information (e.g., [f1 f2 f3 ⪋ fN] (e.g., [0.03 0.21 0.17 ⪋ 0.2]). Although shown on the axes of f1, f2, and f3 in FIG. 6, the reference feature vector and the second feature vector may refer to three or more multidimensional coordinates.

[0123] The reference feature vector may refer to the feature vector of a face determined to be most similar to the user's face among the user's face images. The reference feature vector may also be set by user input. For example, the reference feature vector may be a feature vector corresponding to the user's face (image) entered by the user when initially registering the face.

[0124] The second feature vector may refer to the feature vectors of the user's face excluding the reference feature vector.

[0125] According to one embodiment, the electronic device can acquire an image including a face in response to a user authentication request in operation 910.

[0126] The electronic device (400) may include a reference feature vector, which is a feature vector of a face determined to be most similar to the user's face among the user's face images, and at least one second feature vector, which is a feature vector of the user's face. The feature vector database may be stored in the memory of the electronic device or stored on an external server.

[0127] The electronic device (400) can receive an image containing a face. The electronic device (400) may include a camera and / or a sensor for receiving an image containing a face. The electronic device (400) can perform user authentication and / or face registration.

[0128] The electronic device (400) can detect a face from an image containing a face. Detecting a face may refer to detecting an area corresponding to a face among the areas included in the image.

[0129] The electronic device (400) can check whether the detected face is forged. If the detected face is forged, the electronic device (400) may determine user authentication to fail and not transmit the face to the feature vector extraction unit. According to one example, the electronic device (400) can check whether the detected face is forged based on feature vectors stored in the feature vector database (470) or check whether the face is forged using a pre-trained artificial intelligence model.

[0130] According to one embodiment, the electronic device (400) can identify a first feature vector based on a plurality of feature information of the face from an image including the face in operation 920.

[0131] The electronic device (400) can perform user authentication using an image containing a face. User authentication may refer to an action of verifying whether the face included in the image is a user registered with the electronic device (400).

[0132] The electronic device (400) may use a feature vector corresponding to the user's face to perform user authentication. The feature vector may include multiple feature information representing features that can be identified in the face. The feature information may be converted into numerical values ​​and included in the feature vector. The electronic device (400) may perform user authentication by comparing the feature vector corresponding to the user's face with a first feature vector extracted from a face included in an image input during user authentication.

[0133] For example, the electronic device (400) may determine (or calculate) the similarity between a feature vector corresponding to a user's face and a first feature vector, and determine user authentication as successful if the calculated similarity is greater than or equal to a threshold value. The threshold value may refer to a pre-set criterion for determining user authentication as successful. The electronic device (400) may calculate the similarity using cosine similarity and / or Euclidean distance.

[0134] The electronic device (400) can extract a first feature vector, which is a feature vector of the detected face, from the detected face. The feature vector may include a plurality of feature information representing features that can be identified on the face. For example, the feature information may be determined based on landmark points on the face (e.g., location of eyes, nose, mouth, jawline), distances between landmark points, skin texture information (e.g., texture, wrinkles, blemishes, spots), skin color information (e.g., color histogram), the shape of the face, and / or the symmetry of the face. Each feature information may be converted into a numerical value and included in the feature vector.

[0135] The electronic device (400) may use a face recognition model to extract a first feature vector from a detected face. The face recognition model may be an artificial intelligence (e.g., deep learning) model trained to extract a feature vector (e.g., [f1 f2 f3 �� fN] (e.g., [0.03 0.21 0.17 �� 0.2]) which is a coordinate in a multidimensional space from a detected face. The electronic device may transmit the extracted first feature vector to the face authentication unit (440).

[0136] According to one embodiment, the electronic device can, in operation 930, determine a first similarity between a second feature vector calculated based on a plurality of feature information of a face included in an image that has been authenticated as a user of the electronic device and the first feature vector.

[0137] The electronic device (400) can calculate a first similarity between a first feature vector and a second feature vector. The electronic device (400) can determine (or calculate) the similarity between the second feature vector and the first feature vector. The electronic device (400) can calculate the similarity using cosine similarity and / or Euclidean distance. For example, the electronic device (400) calculates the cosine similarity between the second feature vector and the first feature vector, and the calculated similarity can be determined as a value between 0.0 and 1.0.

[0138] When performing user authentication, the electronic device (400) may perform user authentication for each of the reference feature vector and the second feature vector. The electronic device (400) may calculate a first similarity for each of the reference feature vector and the second feature vector by comparing each of the reference feature vector and the second feature vector with a first feature vector, and determine whether to correct the first similarity. However, the first similarity of the reference feature vector may not be subject to similarity correction.

[0139] According to one embodiment, in operation 940, the electronic device can identify a first difference vector representing the difference between a reference feature vector and a first feature vector based on a plurality of feature information of the user's face of the electronic device that is predetermined, and a second difference vector representing the difference between the reference feature vector and the second feature vector.

[0140] According to one example, the electronic device (400) can calculate a second difference vector between the second feature vector and the reference feature vector for each of the second feature vectors. The electronic device (400) can determine a first characteristic value of the second feature vector as a value corresponding to the magnitude of the second difference vector between the second feature vector and the reference feature vector, and can determine whether to correct the first similarity based on the first characteristic value of the second feature vector. For example, the electronic device (400) can decide to correct the first similarity of the second feature vector only when the magnitude of the difference vector of the reference feature vector is greater than or equal to a set value. According to another example, the electronic device (400) can calculate the similarity between the reference feature vector and the second feature vector, and decide to correct the first similarity of the second feature vector only when the calculated similarity is less than a reference value.

[0141] The setting values ​​of the present disclosure may be set by the manufacturer of the electronic device.

[0142] According to one embodiment, the electronic device can correct the confirmed first similarity when, in operation 950, the first characteristic value of the second difference vector is greater than a predetermined set value.

[0143] The electronic device (400) can determine whether to correct the first similarity of the second feature vector based on a reference feature vector. The electronic device (400) can correct the first similarity of the second feature vector when the first characteristic value of the second difference vector, which is the degree of difference between the second feature vector and the first feature vector (e.g., the magnitude of the second difference vector), is greater than or equal to a set value. Correcting the first similarity of the second feature vector may refer to determining the second similarity by adding a similarity correction value to the first similarity.

[0144] The electronic device (400) can calculate a first characteristic value of the second difference vector. The first characteristic value of the second difference vector can be determined based on the degree of difference between the reference feature vector and the second feature vector. According to one example, the electronic device (400) can calculate a second difference vector representing the difference between the second feature vector and the reference feature vector for each of the second feature vectors, and determine the magnitude of the second difference vector as the first characteristic value of the second difference vector.

[0145] The electronic device (400) can check whether the first characteristic value of the second difference vector is greater than or equal to the set value.

[0146] The electronic device (400) can correct the first similarity to the second similarity in operation 540 when the first characteristic value of the second difference vector is greater than or equal to the set value. The electronic device (400) can correct the first similarity of the second feature vector when the first characteristic value of the second difference vector, which is the degree of difference between the second feature vector and the first feature vector, is greater than or equal to the set value. For example, the electronic device (400) can correct the first similarity of the second feature vector only when the magnitude of the second difference vector is greater than or equal to the set value. The electronic device (400) can correct the first similarity to the second similarity by adding a similarity correction value to the first similarity.

[0147] The electronic device (400) may determine a similarity correction value based on a first characteristic value of a second difference vector, which is the degree of difference between a reference feature vector and a second feature vector. For example, the electronic device (400) may determine the similarity correction value to be proportional to the degree of difference between the second feature vector and the reference feature vector (e.g., the magnitude of the second difference vector). The electronic device (400) may determine a second characteristic value of the second difference vector (e.g., the direction of the second difference vector) and a second characteristic value of the first difference vector (e.g., the direction of the first difference vector). The first difference vector may refer to a difference vector representing the difference between the first feature vector and the reference feature vector. The electronic device (400) may determine the similarity correction value based on the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector. The electronic device (400) may determine the second characteristic value of the second difference vector as the difference vector between the second feature vector and the reference feature vector.

[0148] The electronic device (400) can check whether the first similarity or the second similarity is greater than or equal to a threshold value. The threshold value may be a value indicating a match with the user's face. For example, the electronic device (400) can check whether the second similarity is greater than or equal to the threshold value when similarity correction is performed, and whether the first similarity is greater than or equal to the threshold value when similarity correction is not performed.

[0149] The electronic device (400) may determine the user authentication result of the first feature vector as authentication success if the first similarity or the second similarity is greater than or equal to a threshold value. The electronic device (400) may determine the user authentication result of the first feature vector as authentication failure if the first similarity or the second similarity is less than a threshold value.

[0150] The electronic device (400) can perform similarity correction only on the feature vectors determined to correct the first similarity based on the reference feature vector among the second feature vectors. The electronic device (400) can prevent the problem of FAR increasing due to similarity correction by selectively performing similarity correction without performing similarity correction on the first similarity of all second feature vectors.

[0151] The electronic device (400) can determine a similarity correction value based on a first feature vector, a reference feature vector, and a second feature vector. The electronic device (400) can determine a second similarity by adding the similarity correction value to the first similarity.

[0152] The electronic device (400) may determine a similarity correction value based on a first characteristic value of a second difference vector, which is the degree of difference between a reference feature vector and a second feature vector. For example, the electronic device (400) may determine the similarity correction value to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the second difference vector), which is the degree of difference between the second feature vector and the reference feature vector. The second feature vector may include at least one feature information that reflects a change in the state of the face (e.g., brightness, pose, or accessories (e.g., glasses, hat)). The second feature vector corresponds to the face of the same user, but at least one of the multiple values ​​included in the second feature vector may differ from one another. Therefore, the first characteristic value of the second difference vector may differ for each second feature vector. The electronic device (400) may determine the similarity correction value to be larger as the first characteristic value of the second difference vector is larger, and may determine the similarity correction value to be smaller as the first characteristic value of the second feature vector is smaller. However, the electronic device (400) may not perform correction of the second feature vector if the degree of difference between the second feature vector and the reference feature vector is less than or equal to a set value.

[0153] According to one example, the electronic device (400) can determine the magnitude of the difference vector between the second feature vector and the reference feature vector as the first feature value of the second difference vector.

[0154] The electronic device (400) can increase the self-acceptance rate even if the degree of difference between the first feature vector and the reference feature vector due to the deformation of the face state varies depending on the user, by determining the similarity correction value to be proportional to the degree of difference between the second feature vector and the reference feature vector. The electronic device (400) can determine the authentication result differently for each user even if it performs user authentication based on substantially the same deformation of the face state. The electronic device (400) can perform user authentication by extracting the first feature vector from the face included in the image reflecting the deformation of the face state when performing user authentication. Since the degree of difference between the first feature vector and the reference feature vector due to the deformation of the face state varies depending on the user, the user authentication result based on the deformation of the face state may vary depending on the user. The electronic device (400) can increase TAR by determining the second similarity to be greater than the first similarity, even if the difference between the user's feature vector (e.g., second feature vector, first feature vector) and reference feature vector according to a specific variation of the face condition (e.g., brightness difference of 3 dB) is greater than other variations of the face condition.

[0155] However, the electronic device (400) may increase the FAR by determining the similarity correction value to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the difference vector between the second feature vector and the reference feature vector). Even when performing user authentication of the first feature vector of a person other than the user, the electronic device (400) may determine the user authentication result of the first feature vector of a person other than the user as successful by determining the similarity correction value to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the difference vector between the second feature vector and the reference feature vector).

[0156] The electronic device (400) can prevent the problem of FAR increasing by determining the similarity correction value based on the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector, and determining the similarity correction value to be proportional to the first characteristic value of the second difference vector.

[0157] The electronic device (400) can determine a second characteristic value of the second difference vector and a second characteristic value of the first difference vector. The second characteristic value can be determined based on a deformation of the face state that causes a difference from the reference feature vector. The second characteristic value of the second difference vector can be determined based on a deformation of the face state corresponding to the second feature vector that causes a difference from the reference feature vector. Accordingly, the second characteristic value of the second difference vector can represent a deformation of the face state corresponding to the second feature vector. For example, if the second feature vector is obtained according to a third deformation (e.g., a hat), the second characteristic value of the second difference vector can reflect the third deformation (e.g., a hat). The second characteristic value of the first difference vector can represent a deformation of the face state corresponding to the feature information included in the first feature vector. For example, if the second feature vector is obtained according to a fourth deformation, the second characteristic value of the second difference vector can reflect the fourth deformation.

[0158] The electronic device (400) may determine a similarity correction value based on the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector. For example, the electronic device (400) may determine it to be proportional to the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector. The similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector may refer to cosine similarity. The similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector may correspond to the degree of similarity between the deformation of the face state reflected in the second characteristic vector and the deformation of the face state reflected in the first characteristic vector.

[0159] According to one embodiment, the electronic device (400) may determine a second characteristic value of a second difference vector based on the direction of the second difference vector representing the difference between the second feature vector and the reference feature vector. For example, the electronic device (400) may determine that the second characteristic value of the second difference vector has a value equal to the direction of the second difference vector. According to one example, the second characteristic value of the second difference vector may be in the form of a vector representing the direction.

[0160] According to one embodiment, the electronic device (400) may determine a second characteristic value of a first difference vector based on the direction of the first difference vector representing the difference between the first feature vector and the reference feature vector. For example, the electronic device (400) may determine that the second characteristic value of the first difference vector has a value equal to the direction of the first difference vector. According to one example, the second characteristic value of the first difference vector may be in the form of a vector representing the direction.

[0161] The electronic device (400) may determine a similarity correction value proportional to the similarity of the direction of the second difference vector representing the difference between the second feature vector and the reference feature vector and the direction of the first difference vector representing the difference between the first feature vector and the reference feature vector. The similarity between the direction of the second difference vector and the direction of the first difference vector may be determined to be a larger value as the angle between the second difference vector and the first difference vector becomes smaller. According to one embodiment, the electronic device (400) may determine the similarity correction value to 0 when the similarity between the second difference vector of the second feature vector and the reference feature vector and the first difference vector of the first feature vector and the reference feature vector is less than a set value.

[0162] The electronic device (400) can prevent the problem of the FAR increasing as the similarity correction value is determined to be proportional to the first characteristic value of the second difference vector (e.g., the magnitude of the second difference vector between the second feature vector and the reference feature vector) by determining the similarity correction value based on the similarity between the second characteristic value of the second difference vector (e.g., the direction of the second difference vector between the second feature vector and the reference feature vector) and the second characteristic value of the first difference vector (e.g., the direction of the first difference vector between the first feature vector and the reference feature vector). For example, when the first similarity is corrected by reflecting only the first characteristic value of the second feature vector, the recognition allowable range (721) may be larger than that of other second feature vectors (710), but as the first similarity is corrected by reflecting both the first characteristic value and the second characteristic value, the recognition allowable range (721) may be adjusted to a limited recognition allowable range (722).

[0163] According to one embodiment, the electronic device (400) may determine a similarity correction value proportional to the first similarity. By setting the second similarity higher as the first similarity increases, the electronic device (400) can prevent the problem of the first feature vector with low first similarity succeeding in user authentication and the FAR increasing.

[0164] The recognition allowance range may refer to the range in which the electronic device determines the user authentication result of the first feature vector as successful. The electronic device (400) may determine the user authentication result of the first feature vector as successful if the first feature vector is included in the recognition allowance range.

[0165] The electronic device (400) can extend the recognition allowance range corresponding to some second feature vectors by determining the similarity correction value based on the first characteristic value of the second difference vector (e.g., the magnitude of the difference vector between the reference feature vector and the second feature vector), which is the degree of difference between the reference feature vector and the second feature vector. Accordingly, a user's first feature vector that is not included in the recognition allowance range before similarity correction of a specific second feature vector may be included in the recognition allowance range after similarity correction.

[0166] The electronic device (400) can limit the direction of expansion of the recognition allowance range corresponding to some second feature vectors by determining the similarity correction value based on the similarity between the second feature value of the second difference vector and the second feature value of the first difference vector. For example, the recognition allowance range corresponding to some second feature vectors can be limited according to the second feature vector and the second difference vector of the reference feature vector. Thus, it is possible to limit the first feature vector of another person, which may be included when similarity correction is based only on the first feature value of the second difference vector, so that it is not included in the recognition allowance range.

[0167] An electronic device according to one embodiment may include a memory that stores at least one computer program including instructions. An electronic device according to one embodiment may include at least one processor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may acquire an image including a face in response to a user authentication request. When the instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first feature vector based on a plurality of feature information of the face from the image including the face. When the instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first similarity between a second feature vector calculated based on a plurality of feature information of the face included in an image of a user of the electronic device who has already been authenticated as a user of the electronic device and the first feature vector. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify a first difference vector representing the difference between a reference feature vector and a first feature vector based on a plurality of feature information of the user's face of the electronic device that is predetermined, and a second difference vector representing the difference between the reference feature vector and the second feature vector. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may correct the identified first similarity if the first characteristic value of the second difference vector is greater than a predetermined set value.

[0168] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may calculate a similarity correction value based on a first characteristic value of the second difference vector, a second characteristic value of the second difference vector, and a second characteristic value of the first difference vector. When the instructions are executed individually or collectively by the at least one processor, the electronic device may correct the first similarity to a second similarity based on the similarity correction value. When the instructions are executed individually or collectively by the at least one processor, the electronic device may perform user authentication based on the second similarity.

[0169] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may determine the similarity correction value in proportion to the first characteristic value of the second difference vector.

[0170] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may determine the similarity correction value in proportion to the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector.

[0171] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether the second similarity exceeds a threshold value and determine the authentication of the first feature vector as successful in response to the second similarity exceeding the threshold value.

[0172] In an electronic device according to one embodiment, a first characteristic value of the second difference vector may be determined based on the magnitude of the second difference vector. A second characteristic value of the second difference vector may be determined based on the direction of the second difference vector. A second characteristic value of the first difference vector may be determined based on the direction of the first difference vector.

[0173] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may determine the similarity correction value to be proportional to the first similarity.

[0174] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, if the electronic device determines that the authentication of the first feature vector is successful, the first feature vector may be stored in the memory.

[0175] In an electronic device according to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may calculate the similarity between the reference feature vector, the first feature vector, and the second feature vector. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine the feature vector with the highest average value of similarity with other vectors as the reference feature vector.

[0176] In an electronic device according to one embodiment, the first similarity may refer to the cosine similarity of the first feature vector and the second feature vector.

[0177] In an electronic device according to one embodiment, the reference feature vector may be a feature vector of a face image determined to be most similar to the user's face among the user's face images.

[0178] A method of operation of an electronic device according to one embodiment may include an operation of acquiring an image including a face in response to a user authentication request. The method of operation of the electronic device may include an operation of identifying a first feature vector based on a plurality of feature information of the face from the image including the face. The method of operation of the electronic device may include an operation of identifying a first similarity between a second feature vector calculated based on a plurality of feature information of the face included in an image of a user of the electronic device who has already been authenticated as such and the first feature vector. The method of operation of the electronic device may include an operation of identifying a first difference vector representing the difference between a reference feature vector based on a plurality of feature information of the face of a user of the electronic device that is predetermined and the first feature vector, and a second difference vector representing the difference between the reference feature vector and the second feature vector. The method of operation of the electronic device may include an operation of correcting the identified first similarity when the first characteristic value of the second difference vector is greater than a predetermined set value.

[0179] A method of operation of an electronic device according to one embodiment may include an operation of calculating a similarity correction value based on a first characteristic value of the second difference vector, a second characteristic value of the second difference vector, and a second characteristic value of the first difference vector. It may include an operation of correcting the first similarity to a second similarity based on the similarity correction value. It may include an operation of performing user authentication based on the second similarity.

[0180] A method of operation of the electronic device according to one embodiment may include an operation of determining the similarity correction value in proportion to the first characteristic value of the second difference vector.

[0181] A method of operation of the electronic device according to one embodiment may include an operation of determining the similarity correction value in proportion to the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector.

[0182] A method of operation of the electronic device according to one embodiment may include checking whether the second similarity exceeds a threshold value, and determining the authentication of the first feature vector as successful in response to the second similarity exceeding the threshold value.

[0183] In a method of operation of the electronic device according to one embodiment, a first characteristic value of the second difference vector may be determined based on the magnitude of the second difference vector. A second characteristic value of the second difference vector may be determined based on the direction of the second difference vector. A second characteristic value of the first difference vector may be determined based on the direction of the first difference vector.

[0184] In a method of operating the electronic device according to one embodiment, the operation of determining the similarity correction value to be proportional to the first similarity may be included.

[0185] In a method of operation of the electronic device according to one embodiment, when the authentication of the first feature vector is determined to be successful, the operation of storing the first feature vector in the memory may be included.

[0186] In a method of operation of the electronic device according to one embodiment, the method may include an operation of calculating the mutual similarity between the reference feature vector, the first feature vector, and the second feature vector. The method of operation of the electronic device may include an operation of determining the feature vector having the highest average value of similarity with other vectors as the reference feature vector.

[0187] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0188] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any possible combination of items listed together in the corresponding phrase. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0189] As used in this document, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed as a whole, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0190] Various embodiments of the present document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine (e.g., an electronic device (100)). For example, a processor (e.g., a processor (110)) of the machine (e.g., an electronic device (100)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' merely means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0191] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0192] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, Memory for storing at least one computer program including instructions; At least one processor; When the above instructions are executed individually or collectively by the at least one processor, the electronic device, In response to a user authentication request, acquire an image including a face, and Identify a first feature vector based on a plurality of feature information of the face from an image including the face, and Confirming a first similarity between a second feature vector calculated based on multiple feature information of a face included in an image of a user of the electronic device already authenticated, and the first feature vector, Identifying a first difference vector representing the difference between a reference feature vector and a first feature vector based on a plurality of feature information of the face of a user of the electronic device that is predetermined, and a second difference vector representing the difference between the reference feature vector and the second feature vector, An electronic device that corrects the first similarity confirmed above when the first characteristic value of the second difference vector is greater than a predetermined set value.

2. In claim 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device, A similarity correction value is calculated based on the first characteristic value of the second difference vector, the second characteristic value of the second difference vector, and the second characteristic value of the first difference vector. Based on the above similarity correction value, the above first similarity is corrected to a second similarity, and An electronic device that performs user authentication based on the above second similarity.

3. An electronic device according to paragraph 2, wherein, when the instructions are executed individually or collectively by at least one processor, the electronic device determines the similarity correction value in proportion to the first characteristic value of the second difference vector.

4. In paragraph 2, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that determines a similarity correction value in proportion to the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector.

5. In claim 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that checks whether the second similarity exceeds a threshold value and determines the authentication of the first feature vector as successful in response to the second similarity exceeding the threshold value.

6. In claim 1, the first characteristic value of the second difference vector is determined based on the magnitude of the second difference vector, and The second characteristic value of the second difference vector is determined based on the direction of the second difference vector, and An electronic device in which the second characteristic value of the first difference vector is determined based on the direction of the first difference vector.

7. In claim 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that determines the above similarity correction value in proportion to the above first similarity.

8. In claim 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device An electronic device that stores the first feature vector in the memory when the authentication of the first feature vector is determined to be successful.

9. In claim 8, when the instructions are executed individually or collectively by the at least one processor, the electronic device Calculate the mutual similarity between the reference feature vector, the first feature vector, and the second feature vector, and An electronic device that determines the feature vector with the highest average value of similarity with other vectors as the reference feature vector.

10. An electronic device in which, in claim 1, the first similarity refers to the cosine similarity of the first feature vector and the second feature vector.

11. An electronic device in which, in claim 1, the reference feature vector is the feature vector of a face image determined to be most similar to the user's face among the user's face images.

12. In a method of operating an electronic device, An action of acquiring an image including a face in response to a user authentication request; An operation to identify a first feature vector based on a plurality of feature information of the face from an image including the face; An operation to confirm a first similarity between a second feature vector calculated based on multiple feature information of a face included in an image of a user of the electronic device already authenticated, and the first feature vector; An operation to identify a first difference vector representing the difference between a reference feature vector and a first feature vector based on a plurality of feature information of a user's face of the electronic device that is predetermined, and a second difference vector representing the difference between the reference feature vector and the second feature vector; and A method of operation of an electronic device comprising an operation to correct the first similarity confirmed when the first characteristic value of the second difference vector is greater than a predetermined set value.

13. In Clause 12, the method of operation of the electronic device is, An operation of calculating a similarity correction value based on the first characteristic value of the second difference vector, the second characteristic value of the second difference vector, and the second characteristic value of the first difference vector; The operation of correcting the first similarity to a second similarity based on the above similarity correction value; and A method of operation of an electronic device including an operation to perform user authentication based on the above second similarity.

14. In paragraph 13, the method of operating the electronic device is, A method of operation of an electronic device comprising determining the similarity correction value in proportion to the first characteristic value of the second difference vector.

15. In paragraph 13, the method of operation of the electronic device is, A method of operation of an electronic device comprising determining a similarity correction value in proportion to the similarity between the second characteristic value of the second difference vector and the second characteristic value of the first difference vector.

Images