Electronic device, and method for managing registered fingerprints of electronic device

Abstract

This electronic device comprises: a sensor; a memory for storing at least one computer program including instructions and a plurality of fingerprint templates corresponding to each of a plurality of registered fingerprint images; and at least one processor, wherein the plurality of registered fingerprint images includes forged fingerprint images obtained by forging the fingerprint of a user, abnormal state fingerprint images including an obstacle element that interferes with fingerprint authentication of the fingerprint of the user, and a normal state fingerprint image that does not include an obstacle element of the fingerprint of the user. When executed individually or collectively by the at least one processor, the instructions can instruct the electronic device to: use the sensor to acquire an input fingerprint image, which is a fingerprint image input by the user; identify, on the basis of the plurality of fingerprint templates, whether the input fingerprint image matches some of the plurality of registered fingerprint images; if the input fingerprint image matches some of the plurality of registered fingerprint images, generate at least one virtual fingerprint image on the basis of the input fingerprint image matching some of the plurality of fingerprint images; store templates of the input fingerprint image and the at least one virtual fingerprint image in the plurality of fingerprint templates; and train a fingerprint generation artificial intelligence model on the basis of the input fingerprint image and the at least one virtual fingerprint image.

Description

How to manage electronic devices and registered fingerprints on electronic devices

[0001] The present disclosure relates to an electronic device and an operating method of the electronic device, and relates to a technique for managing a registered fingerprint of the electronic device.

[0002] Electronic devices can maintain the security of user information within the electronic device through security authentication technologies, including fingerprint authentication. Fingerprint authentication technology registers a fingerprint image via a sensor and performs fingerprint authentication on an input fingerprint image based on the registered fingerprint image. The electronic device can determine that fingerprint authentication is successful if the input fingerprint image matches the registered fingerprint image.

[0003] The registered fingerprint image used for fingerprint authentication may be the fingerprint image at the time the user registers the fingerprint. Unless the user registers a new fingerprint, the registered fingerprint image may not be updated. A user's fingerprint image may change over time, and the registered fingerprint image may not reflect the changes in the user's fingerprint image.

[0004] As fingerprint authentication technology advances, it is now possible to accurately capture a user's fingerprint image and compare it with a registered fingerprint image. Electronic devices can accurately compare registered and input fingerprint images, but fingerprint authentication accuracy may be reduced for input fingerprint images that are not accurately captured.

[0005] An electronic device may include an anti-spoofing protection (ASP) module. The ASP module may include a model for distinguishing between fake and non-fake fingerprint images. The electronic device may include the same ASP module regardless of the user.

[0006] The above information may be provided as background information to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0007] The registered fingerprint image may contain information about a fingerprint image from a time prior to the time of fingerprint authentication. A user's fingerprints can change over time. If the template for the registered fingerprint image is not updated, the electronic device cannot perform fingerprint authentication based on the user's fingerprint changes. The accuracy of fingerprint authentication devices may deteriorate over time.

[0008] When a user performs fingerprint authentication, there may be an obstruction in the user's fingerprint. The input fingerprint image may be unclear due to the obstruction. For fingerprint authentication to succeed, the user must input a fingerprint without any obstructions. However, the success rate of fingerprint authentication may vary depending on the obstruction in each authentication situation.

[0009] The ASP module may include a model for determining whether an unspecified fingerprint image is forged. Because the ASP module determines whether a fingerprint image is forged without considering the characteristics of the user's fingerprint image, it may incorrectly determine whether some of the user's fingerprint images are forged, resulting in the image being judged as forged. Even when the user inputs a normal fingerprint image, the ASP module may determine the fingerprint authentication result as a failed result by judging the image as forged.

[0010] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0011] An electronic device according to one embodiment may include a sensor. The electronic device may include at least one computer program including instructions and a memory storing a plurality of fingerprint templates indicating embedding vectors of each of a plurality of registered fingerprint images. The electronic device may include at least one processor. The plurality of registered fingerprint images may include a forged fingerprint image that is a forged fingerprint image of a user's fingerprint, an abnormal fingerprint image that includes a hindrance element that interferes with fingerprint authentication among the user's fingerprints, and a normal fingerprint image that does not include the hindrance element among the user's fingerprints. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire an input fingerprint image, which is a fingerprint image input by a user, using the sensor. The instructions may cause the electronic device to determine whether the input fingerprint image matches a portion of the plurality of registered fingerprint images based on the plurality of fingerprint templates. If the input fingerprint image matches a portion of the plurality of registered fingerprint images, the instructions may cause the electronic device to generate at least one virtual fingerprint image based on the input fingerprint image that matches a portion of the plurality of fingerprint images. The above instructions may cause templates of the input fingerprint image and the at least one virtual fingerprint image to be stored in the plurality of fingerprint templates. The instructions may cause a fingerprint generation artificial intelligence model to be trained based on the input fingerprint image and the at least one virtual fingerprint image.

[0012] According to one embodiment, a method of operating an electronic device may include an operation of acquiring an input fingerprint image, which is a fingerprint image input by a user, using the sensor. The method of operating the electronic device may include an operation of determining whether the input fingerprint image matches a portion of the plurality of registered fingerprint images based on a plurality of fingerprint templates indicating an embedding vector of each of the plurality of registered fingerprint images. The method of operating the electronic device may include an operation of generating at least one virtual fingerprint image based on the input fingerprint image matching a portion of the plurality of fingerprint images, when the input fingerprint image matches a portion of the plurality of registered fingerprint images. The method of operating the electronic device may include an operation of storing templates of the input fingerprint image and the at least one virtual fingerprint image in the plurality of fingerprint templates. The method of operating the electronic device may include an operation of training a fingerprint generation artificial intelligence model based on the input fingerprint image and the at least one virtual fingerprint image.

[0013] A non-transitory computer-readable recording medium according to one embodiment of the present disclosure may store at least one command and / or instruction that, when executed, causes an electronic device to perform the method or operation of the electronic device described above. The operation of the electronic device may include: using a sensor, obtaining an input fingerprint image corresponding to a fingerprint image input by a user; identifying whether the input fingerprint image matches at least a portion of a plurality of registered fingerprint images based on a plurality of fingerprint templates; generating at least one virtual fingerprint image based on the input fingerprint image matching the portion of the plurality of fingerprint images when the input fingerprint image matches at least a portion of the plurality of registered fingerprint images; storing a template of the input fingerprint image and a template of the at least one virtual fingerprint image in the plurality of fingerprint templates; and generating a fingerprint image similar to the at least one virtual fingerprint image or the input fingerprint image by training a fingerprint generation artificial intelligence model.

[0014] An electronic device can perform fingerprint authentication based on the results of comparing an input fingerprint image with a normal fingerprint image, an abnormal fingerprint image containing a defect, and / or a fake fingerprint image. By comparing the abnormal fingerprint image with the input fingerprint, the electronic device can accurately obtain fingerprint authentication results even if the input fingerprint image is a fingerprint image obtained with a defect or a fake fingerprint image.

[0015] An electronic device can perform fingerprint authentication based on an input fingerprint image as well as an enrolled fingerprint image by storing the input template in a fingerprint template database. Since the electronic device performs fingerprint authentication based on a recently input user fingerprint image, it can perform accurate fingerprint authentication even if the user's fingerprint changes over time. Furthermore, the electronic device can generate a virtual fingerprint image that maintains the structural characteristics of the input fingerprint image and perform fingerprint authentication based on the template of the virtual fingerprint image. Because the electronic device can compare a limited number of fingerprint images with the input fingerprint image, it can perform accurate fingerprint authentication.

[0016] An electronic device can be trained to determine whether a fingerprint image is forged based on a user's fingerprint image. Since the electronic device can generate a large number of virtual fingerprint images based on an input fingerprint image, it can acquire a large number of user fingerprint images for training the ASP module. An ASP module trained based on a large number of user fingerprint images can increase the accuracy of determining whether a user's fingerprint image is forged.

[0017] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0018] The above and other aspects, features and advantages of specific embodiments of the present disclosure can be clearly explained by the following description provided together with the accompanying drawings.

[0019] FIG. 1A is a block diagram of an exemplary electronic device capable of performing the operations described herein.

[0020] Figure 1b is a schematic diagram of an exemplary artificial intelligence system.

[0021] FIG. 2 is a drawing illustrating a fingerprint authentication device according to an example.

[0022] FIG. 3 is a diagram illustrating a fingerprint authentication system of an electronic device according to one embodiment.

[0023] FIG. 4A is a flowchart of an operation of an electronic device classifying an input fingerprint image, according to one embodiment.

[0024] FIG. 4b is a diagram illustrating a result of an electronic device classifying an input fingerprint image according to an embodiment.

[0025] FIG. 5 is a flowchart of an operation of an electronic device updating a fingerprint template database according to one embodiment.

[0026] FIG. 6 is a diagram illustrating an operation of an electronic device updating a fingerprint template according to one embodiment.

[0027] FIG. 7 is a diagram illustrating a fingerprint generation artificial intelligence model of an electronic device according to one embodiment.

[0028] Figure 8 is a flowchart of an operation for learning an ASP module according to one embodiment.

[0029] FIG. 9 is a diagram illustrating an ASP module according to one embodiment.

[0030] FIG. 10 is a flowchart of an operation of an electronic device performing fingerprint authentication according to one embodiment.

[0031] FIG. 11 is a diagram illustrating a fingerprint authentication result of an input fingerprint image according to one embodiment.

[0032] FIG. 12 is a diagram illustrating an electronic device supporting multiple users according to one embodiment.

[0033] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment.

[0034] FIG. 14 is a diagram illustrating an operation flow chart of an electronic device according to one embodiment.

[0035] It should be noted that throughout the drawings, the same reference numbers are used to describe identical or similar elements, features and structures.

[0036] The following description, with reference to the attached drawings, is provided to facilitate a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. While it includes numerous specific details to facilitate this understanding, these are to be considered merely exemplary. Accordingly, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, descriptions of well-known functions and structures may be omitted for clarity and brevity.

[0037] The terms and words used in this specification and claims are not limited to their bibliographic meanings, but are used by the inventors solely to facilitate a clear and consistent understanding of the present invention. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and is not intended to limit the present disclosure as defined by the appended claims and their equivalents.

[0038] Unless the context clearly indicates otherwise, the singular form should be understood to include the plural referent. Thus, for example, reference to "component surface" includes reference to one or more of such surfaces.

[0039] It should be understood that the blocks and combinations of the flowcharts in each flowchart can be performed by one or more computer programs containing instructions. The entirety of one or more computer programs may be stored in a single memory device, or the one or more computer programs may be divided into different portions stored in multiple different memory devices.

[0040] Any function or operation described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing and includes circuits such as an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphics processing unit (GPU), a neural processing unit (NPU), an artificial intelligence (AI) chip), a wireless fidelity (Wi-Fi) chip, a Bluetooth® chip, a global positioning system (GPS) chip, a near field communication (NFC) chip, a connection chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display driver integrated circuit (IC), an audio codec chip, a universal serial bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system on a chip (SoC), an IC, or a similar chip.

[0041] For convenience of explanation, the following example assumes that the biometric information is a fingerprint. However, the examples can be equally applied to various biometric information that can be recognized in the form of images, such as veins, irises, and faces.

[0042] FIG. 1A is a block diagram of an exemplary electronic device (100) capable of performing the operations described herein.

[0043] Referring to FIG. 1A, the electronic device (100) may be one of various forms of electronic devices, such as a notebook (190), smartphones (191) having various form factors (e.g., a bar-type smartphone (191-1), a foldable-type smartphone (191-2), or a sliderable (or rollable) type smartphone (191-3)), a tablet (192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 1A are exemplary only and do not limit the implementations described or claimed in this document. The electronic device (100) may be referred to as a mobile device, a user device, a multi-function device, a portable device, or a server.

[0044] The electronic device (100) may include components including at least one processor (110) (hereinafter, referred to as processor (110)), at least one memory (120) (hereinafter, referred to as memory (120)), at least one display (140) (hereinafter, referred to as display (140)), at least one image sensor (150) (hereinafter, referred to as image sensor (150)), at least one communication circuit (160) (hereinafter, referred to as communication circuit (160)), and / or at least one sensor (170) (hereinafter, referred to as sensor (170)). The above components are merely exemplary. For example, the electronic device (100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuitry, an antenna, a rechargeable battery, or an input / output interface). For example, some components may be omitted from the electronic device (100). For example, some components may be integrated into one component.

[0045] The processor (110) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing. The processor (110) may include at least one electrical circuit and may individually or collectively perform distributed processing of instructions (or programs, data, etc.) stored in the memory (120). The processor (110) may include a processor assembly including one or more processing circuits. The processor (110) may include any processing circuit operative to control the performance and operations of one or more components (e.g., the memory (120), the display (140), the image sensor (150), the communication circuit (160), and / or the sensor (170)) of the electronic device (100). For example, the processor (110) (e.g., the application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or a chipset). For example, the processor (110) may be implemented with multiple cores (or at least one core circuit), multiple chips, or multiple chipsets. For example, the processor (110) may include one or more processing circuits. For example, the processor (110) may include one or more processing circuits configured to individually and / or collectively perform various functions of the present disclosure. As a non-limiting example, at least a portion of the processor (110) may be included in a first chip of the electronic device (100), and at least another portion of the processor (110) may be included in a second chip of the electronic device (100) that is different from the first chip of the electronic device (100).

[0046] For example, the processor (110) may include a central processing unit (CPU) (111), a graphics processing unit (GPU) (112), a neural processing unit (NPU) (113), an image signal processor (ISP) (114), a display controller (115), a memory controller (116), a storage controller (117), a communication processor (CP) (118), and / or a sensor interface (119). These components of the processor (110) are merely exemplary. For example, the processor (110) may further include other components. For example, some components of the processor (110) may be omitted from the processor (110). For example, some components of the processor (110) may be included as separate components of the electronic device (100) outside the processor (110). For example, some components of the processor (110) (e.g., memory controller (116)) may be included within other components (e.g., at least a portion of memory (120), an interface (e.g., available for connection to at least one component of the electronic device (100)), a display (140) and / or an image sensor (150)).

[0047] The processor (110) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in the memory (120). The CPU (111) (or central processing circuit) may be configured to control components of the processor (110) based on the execution of instructions stored in the memory (120) (e.g., volatile memory (121) and / or non-volatile memory (122)). The GPU (112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (113) (or neural processing circuit, or artificial intelligence (AI) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). The ISP (114) (or image signal processing circuit) may be configured to process a raw image acquired through the image sensor (150) into a format suitable for a component within the electronic device (100) or a component of the processor (110). The display controller (115) (or display control circuit, or display processing unit (DPU)) may be configured to process an image acquired from the CPU (111), the GPU (112), the ISP (114), or the memory (120) (e.g., the volatile memory (121)) into a format suitable for the display (140). The memory controller (116) (or memory control circuit) may be configured to control reading data from the volatile memory (121) and writing data to the volatile memory (121). The storage controller (117) (or storage control circuit) may be configured to control reading data from the nonvolatile memory (122) and writing data to the nonvolatile memory (122).The CP (118) (communication processing circuit) may be configured to process data acquired from a component of the processor (110) into a format suitable for transmission to another electronic device via the communication circuit (160), or to process data acquired from another electronic device via the communication circuit (160) into a format suitable for processing by the component of the processor (110). For example, the communication circuit (160) may include one or more communication circuits. The sensor interface (119) (or sensing data processing circuit, sensor hub) may be configured to process data about the state of the electronic device (100) and / or the state of the surroundings of the electronic device (100), acquired via the sensor (170), into a format suitable for the component of the processor (110).

[0048] The memory (120) may include one or more storage media (or one or more storage devices). For example, the memory (120) may include a memory assembly including one or more storage media. For example, the one or more storage media may include permanent memory (e.g., non-volatile memory (122)) such as a hard drive, flash memory, read-only memory (ROM), semi-permanent memory (e.g., volatile memory (121)) such as random access memory (RAM), any other suitable type of storage (or storage assembly), or any combination thereof. The memory (120) may include cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As a non-limiting example, the cache memory may be included within the processor (110). The memory (120) may be fixedly embedded within the electronic device (100) or incorporated into one or more suitable types of components (e.g., a subscriber identity module (SIM) card and / or a secure digital (SD) card) that may be repeatedly inserted into and removed from the electronic device (100).

[0049] For example, the memory (120) may store one or more software applications, such as an operating system (or system) software application, a firmware software application, a driver software application, a plug-in (e.g., add-in, add-on, and / or applet) software application, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (110). For example, the memory (120) may store instructions callable by an application programming interface (API). For example, the memory (120) may store instructions within a library.

[0050] Figure 1b is a schematic diagram of an exemplary artificial intelligence system.

[0051] Referring to FIG. 1b, an artificial intelligence system according to one embodiment may include a user interface (10100), a database (10500), an applications / service component (10600), an AI framework (10200), and a generative AI model (10300).

[0052] A user interface (10100) can receive a user query. The input can include user input and / or data acquired or generated by an electronic device (e.g., the electronic device (100) described above, the electronic device (300) of FIG. 3, or the electronic device (1300) of FIG. 13). The data can include images, videos, and / or sensor data generated by at least one processor of the electronic device (e.g., at least one processor (110) or the processor (1310) of FIG. 13) (e.g., illuminance data around the electronic device acquired from a sensor (170) or a sensor hub, posture data (or orientation data) of the electronic device, a temperature inside the electronic device (e.g., a temperature of the display (140) or a temperature of the at least one processor (110)), size information of a display area of ​​the display (140), and / or an image acquired through an image sensor (150) of the electronic device). For example, a user query may be in the form of natural language, touch data acquired through touch circuitry included in the display (140) (e.g., used to identify input from a finger and / or stylus), images, audio, and / or video. Additionally, context information may be transmitted together with the user query. The context information may include various side information related to the time at which the user query is input to the artificial intelligence system. For example, the context information may include information about the application currently being used by the user or information about the user's location. As another example, the user query may also be a non-natural language input that does not generate natural language, such as a design request or modification.Additionally, a mixed form of natural language, images, sounds, and contextual information as described above is also possible. Furthermore, the user interface (10100) can output the output of the artificial intelligence system to the user. The output may include results (or result information) generated or acquired by the artificial intelligence system, at least in part based on the input. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. For example, the output may have a format according to the user settings of the electronic device.

[0053] The AI ​​framework (10200) can receive a user query and coordinate and control each component necessary to execute the user's intent. The AI ​​framework (10200) may include a prompt design component (10210), an application and plug-in management component (APIs / Plugins Management component (10230), and an output modification component (10250).

[0054] A user query or action entered in the user interface (10100) can be sent to a prompt design component (10210). The prompt design component (10210) can be used to generate prompts suitable for input into a large language model (LLM), a large vision model (LVM), or a large multimodal model. The prompt design component (10210) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (10210) can access a database (10500) (e.g., a knowledge repository) containing user preference data, a prompt library, and prompt examples to generate prompts and pass them to the large language model (LLM), the large vision model (LVM), and / or the large multimodal model (LMM).

[0055] The application and plugin management component (10230) can communicate with external information when there is a request for additional information when passing user input as input to the generative AI model (10300). The application and plugin management component (10230) establishes a channel for communicating with the outside of the artificial intelligence system through an application programming interface (API), thereby enabling access to various data sources. For example, the application and plugin management component (10230) can be used to request another component (e.g., the application and service component (10600)) to perform feedback (or response) according to the prompt. The acquired information can be used to generate a prompt by the prompt design component (10210) together with the user input, or can be used as input to the generative AI model (10300). In addition, the application and plugin management component (10230) can request an action through the API if the application or service needs to perform an action that ultimately performs a user query, rather than an intermediate result. Information obtained from the outside can be passed as input to the generative AI model (10300) along with user input.

[0056] The output modification component (10250) can fine-tune (or adjust) (or change) the output from the generative AI model (10300). For example, the output modification component (10250) can determine the relevance (e.g., score) between the output (e.g., content) of the generative AI model and the user input. For example, the output modification component (10250) can verify that the content generated through a large language model (LLM), a large vision model (LVM), or a large multi-modal model (LMM) does not contain the above-described relevance, biased information (e.g., selective information), or harmful information (e.g., violent content or profanity). In addition, the output modification component (10250) can determine to what extent the output matches the user's desired result and can proceed with additional processes if necessary. Additionally, the output modification component (10250) can configure and provide hints to the user to avoid unwanted output.

[0057] A generative AI model (10300) can generally refer to an artificial intelligence neural network that generates new types of data based on user input information. A generative AI model (10300) may include an image generation model and / or a language generation model. The image generation model may include a generative adversarial network (GAN) and / or a variational autoencoder (VAE). An example of an image generation model is a diffusion-based generative model that uses the structure of a VAE and a transformer. In addition, a language generation model is a model trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, the generative AI model may include large multimodal models (LMMs) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0058] In one embodiment, the AI ​​framework (10200) and / or the generative AI model (10300) may be included in an AI module (e.g., including a processing circuit) within the electronic device. For example, the AI ​​module may be operatively coupled with at least one processor (e.g., at least one processor (110) or processor (1310)) of the electronic device. For example, the AI ​​module may be operatively coupled with a sensor hub of the electronic device for one or more sensors within the electronic device.

[0059] FIG. 2 is a drawing illustrating a fingerprint authentication device according to an example.

[0060] A fingerprint authentication device (200) may be an example of an electronic device that manages a user's registered fingerprint. The fingerprint authentication device (200) may perform fingerprint authentication by comparing a user's registered fingerprint image with an input fingerprint image (215) input through a sensor (210). The fingerprint authentication device (200) may perform fingerprint authentication by checking whether the input fingerprint image (215) matches the registered fingerprint image and checking whether the input fingerprint image (215) is forged. For example, the fingerprint authentication device (200) may include a fingerprint authentication unit (241) including a comparator (240) that checks whether the input fingerprint image (215) matches the registered fingerprint image, and an anti-spoofing-protection (ASP) module (250) that checks whether the input fingerprint image (215) is forged.

[0061] The fingerprint authentication device (200) can store a registered fingerprint template (230) that converts a user's registered fingerprint image into an embedding vector and compare it with an input template (225) that converts a user's input fingerprint image into an embedding vector. The registered fingerprint template (230) is a fingerprint template corresponding to the registered fingerprint image at the time the user registers the fingerprint image, and may be information about a fingerprint image older than the time at which fingerprint authentication is performed. Since the user's fingerprint may change over time, if the registered fingerprint template (230) is not updated, the fingerprint authentication device (200) cannot perform fingerprint authentication by reflecting the changed user fingerprint. The accuracy of fingerprint authentication of the fingerprint authentication device (200) may decrease over time.

[0062] In a situation where a user performs fingerprint authentication, there may be an obstacle on the user's fingerprint. The input fingerprint image (215) may be acquired unclearly due to the obstacle. The fingerprint authentication device (200) may store an enrolled fingerprint template (230) including a template of an enrolled fingerprint image input and stored during fingerprint registration and a fingerprint image that has successfully undergone fingerprint authentication. The fingerprint authentication device (200) may store a fingerprint having a fingerprint recognition rate higher than a threshold value as an enrolled fingerprint template (230) and may store a template of a fingerprint image that matches the enrolled fingerprint image as an enrolled fingerprint template (230). The fingerprint images that the fingerprint authentication device (200) compares with the fingerprint image input for fingerprint authentication may be a fingerprint image that has successfully been registered and a fingerprint image that matches the enrolled fingerprint image during authentication. Since the fingerprint authentication device (200) does not separately store a user's fingerprint image that includes an obstacle (e.g., a wet fingerprint image, a dry fingerprint image), it cannot perform fingerprint authentication based on a fingerprint image that includes an obstacle. The fingerprint authentication device (200) may determine the fingerprint authentication result for an input fingerprint image (215) as failed due to an obstacle. Since fingerprint authentication can only be successful if a fingerprint image without obstacles is input, the user may experience the inconvenience of having to check and / or remove obstacles present in the fingerprint for each fingerprint authentication situation.

[0063] A fingerprint authentication device may include an ASP module (250). The ASP module (250) may be a model that distinguishes between a fake fingerprint image and a non-fake fingerprint image. The ASP module (250) may be a model that determines whether a fingerprint image of an unspecified person is fake. Since the ASP module (250) determines whether a fingerprint image is fake without reflecting the characteristics of the user's fingerprint image, it may incorrectly determine whether some of the user's fingerprint images are fake and determine them as fake fingerprint images. Even in a situation where the user inputs a fingerprint image normally, the ASP module (250) may determine the fingerprint authentication result as a failure by determining that the fingerprint image is a fake fingerprint image.

[0064] Below, in FIGS. 3, 4a, 4b, 5 to 14, a method for managing registered fingerprints by an electronic device (300) to solve the above problem is described.

[0065] FIG. 3 is a diagram illustrating a fingerprint authentication system of an electronic device according to one embodiment.

[0066] An electronic device (300) may include a sensor (310) for detecting a user's fingerprint (e.g., sensor (170) of FIG. 1A). The electronic device (300) may obtain an input fingerprint image (315) showing the user's fingerprint through the sensor (310). The electronic device (300) may convert the input fingerprint image (315) obtained by the sensor (310) into an embedding vector to obtain an input fingerprint template (325). The electronic device (300) may compare the input fingerprint template (325) with a template of a registered fingerprint image of the user. The template may refer to a form of storing data obtained by extracting feature information of a fingerprint image in a memory. The template may be data in the form of numbers (or characters) corresponding to the feature information of the fingerprint image. For example, an electronic device (e.g., the electronic device (100) of FIG. 1A) may include, as feature information, minutiae such as ridge ending points, bifurcation points, or smooth areas of an input fingerprint image (315). For example, the electronic device may identify the directionality of a fingerprint by finding the core and delta, which are the central points of the fingerprint, or identify the shape of the fingerprint (e.g., minutiae, bifurcation, isolated, or connected), as feature information.

[0067] The template can be converted from a fingerprint image by a processor (e.g., processor (1310) of FIG. 13) included in an electronic device (e.g., electronic device (1300) of FIG. 13) (300).

[0068] The fingerprint template database (330) may be a database that stores a user's fingerprint template. The fingerprint template database (330) may store a template of a fingerprint image recognized as the user's fingerprint. The fingerprint template database (330) may not store fingerprint images that are not recognized as the user's fingerprint. For example, the fingerprint template database (330) may store a template of a user's fingerprint image input at the time of fingerprint registration or fingerprint authentication. The fingerprint template database (330) may store a fingerprint template including a template of a fingerprint image input and stored at the time of fingerprint registration and a template of a fingerprint image that has successfully undergone fingerprint authentication. The fingerprint template database (330) may store a template of a fingerprint whose fingerprint recognition rate is above a threshold value, and may store a template of a fingerprint image that matches the template of a stored fingerprint image as a registered fingerprint template.

[0069] The electronic device (300) may store a fingerprint image with a recognition rate higher than a threshold value as a registered fingerprint image during the fingerprint registration process, and may not store a fingerprint image with a recognition rate lower than the threshold value. For example, a wet fingerprint may have a lower recognition rate in the image recognized by the sensor (310) than a dry fingerprint. If a wet fingerprint image is input, the electronic device (300) may determine that the fingerprint registration has failed because the recognition rate is lower than the threshold value, and may not store the fingerprint in the fingerprint template database (330).

[0070] The fingerprint template database (330) may be stored in a memory (e.g., memory (1320) of FIG. 13) included in an electronic device (300) (e.g., electronic device (1300) of FIG. 13). The fingerprint template database (330) may include a normal fingerprint template (301), an abnormal fingerprint template (302), and a fake fingerprint template (303). A fingerprint image recognized as a user's fingerprint may be classified into one of the normal fingerprint template (301), the abnormal fingerprint template (302), and the fake fingerprint template (303) and stored in the fingerprint template database (330). The operation of the electronic device (300) classifying a fingerprint image is described in FIG. 4A.

[0071] The normal state fingerprint template (301) may refer to a fingerprint template converted from a normal state fingerprint image among fingerprint images. The normal state fingerprint image may refer to a normal state fingerprint image acquired by the sensor (310). The normal state may refer to a state of a fingerprint that has been successfully registered and a fingerprint that matches (or coincides) with a registered fingerprint. For example, the normal state may refer to a state of a fingerprint whose recognition rate of the fingerprint image is above a threshold value.

[0072] Some of the fingerprint images input through the sensor may contain interference factors. The interference factors may distort the fingerprint images input through the sensor so that the fingerprint images input through the sensor do not match the registered fingerprint images (or the fingerprint images that have been authenticated by matching the registered fingerprint images). The interference factors may include any external factors (e.g., moisture) that interfere with the recognition of the user's fingerprint. The interference factors may include not only external factors but also factors that interfere with fingerprint recognition (e.g., scratches, dryness) occurring on the user's fingerprint.

[0073] The abnormal fingerprint template (302) may refer to a fingerprint template that has been converted from an abnormal fingerprint image among fingerprint images. The abnormal state may refer to a state in which an obstacle exists in the fingerprint of a user performing fingerprint authentication. The abnormal fingerprint image may be recognized differently from the registered fingerprint image (or a fingerprint image that matches (or coincides) with the registered fingerprint image) due to an obstacle that interferes with fingerprint recognition (e.g., an external element, an element that interferes with fingerprint recognition occurring on the fingerprint).

[0074] An abnormal fingerprint image may be a fingerprint image of a user recognized by a sensor (310) in a state where there is a problem. For example, an abnormal fingerprint image may include a fingerprint image input when the user's fingerprint is wet, dry, or has a wound.

[0075] According to one example, the abnormal state fingerprint template (302) may include a template of an abnormal state fingerprint image of a user performing fingerprint authentication, a virtual abnormal state fingerprint image generated based on the abnormal state fingerprint image, and an abnormal state fingerprint image generated based on a normal state fingerprint image input during fingerprint registration (or fingerprint authentication).

[0076] The fake fingerprint template (303) may refer to a fingerprint template converted from a fake fingerprint image. A fake fingerprint may refer to a fingerprint input by an object other than the user's finger among fingerprint images recognized as the user's fingerprint. The fingerprint template database (330) may store fingerprint images recognized as the user's fingerprint, and the fake fingerprint image may not be the user's fingerprint image, but may be a template of a fingerprint image recognized as the user's fingerprint. For example, the fake fingerprint image may include an image that is a forged fingerprint image matching the user's fingerprint, made of a material including gelatin, clay, wood, and silicone.

[0077] According to one example, the fake fingerprint template (303) may include a template of a fake fingerprint image for performing fingerprint authentication, a virtual fake fingerprint image generated based on the fake fingerprint image, and a fake fingerprint image generated based on a normal fingerprint image input during fingerprint registration (or fingerprint authentication).

[0078] The comparator (340) can output a comparison value, which is a value indicating the degree of similarity between two templates, based on two templates. For example, the comparator (340) can compare an input fingerprint template (325) of an input fingerprint image (315) input through a sensor (310) with fingerprint templates (301, 302, 303) included in a fingerprint template database (330), and output a comparison value indicating the degree of similarity between the compared templates.

[0079] The ASP (anti-spoofing-protection) module (350) can verify whether a fingerprint image is forged. For example, the ASP module (350) can verify whether an input fingerprint image (315) input through a sensor (310) is a forged fingerprint. According to an example, the ASP module (350) may include an artificial intelligence model. The artificial intelligence model included in the ASP module (350) may be trained to determine whether a user's fingerprint image is forged based on the user's fingerprint image.

[0080] The fingerprint authentication unit (360) can determine whether fingerprint authentication is successful or not using a comparator (340) and an ASP module (350). The fingerprint authentication unit (360) can determine whether fingerprint authentication is successful or not based on a comparison value output from the comparator (340) and a threshold value indicating the success of fingerprint authentication.

[0081] The fingerprint authentication unit (360) can classify the input fingerprint image (315) into one of a normal fingerprint image, an abnormal fingerprint image, and a fake fingerprint image based on a threshold value indicating the success of fingerprint authentication based on the comparison value output from the comparator (340). The threshold value may be a value indicating that the fingerprint templates corresponding to the comparison value are fingerprints of the same person.

[0082] The fingerprint generation artificial intelligence model (370) can generate a virtual fingerprint image similar to the fingerprint image based on the fingerprint image. For example, the fingerprint generation artificial intelligence model (370) can receive an input fingerprint image (315) and generate at least one virtual fingerprint image. The virtual fingerprint image can be generated to represent a fingerprint image of the same person as the input fingerprint image (315). The fingerprint generation artificial intelligence model (370) can generate a virtual fingerprint image having a shape similar to the target image based on the target image. When the input fingerprint image (315) is a normal fingerprint image, the fingerprint generation artificial intelligence model (370) can generate a virtual abnormal fingerprint image and a virtual fake fingerprint image based on the input fingerprint image (315). When the input fingerprint image (315) is an abnormal fingerprint image, the fingerprint generation artificial intelligence model (370) can generate a virtual abnormal fingerprint image based on the input fingerprint image (315). The fingerprint generation artificial intelligence model (370) can generate a virtual fake fingerprint image based on the input fingerprint image (315) if the input fingerprint image (315) is a fake fingerprint image. The template of the generated virtual fingerprint image can be stored in the fingerprint template database (330).

[0083] FIG. 4A is a flowchart of an operation of an electronic device classifying an input fingerprint image, according to one embodiment.

[0084] The electronic device (300) can obtain an input fingerprint image (315) in operation 410.

[0085] The electronic device (300) can obtain an input fingerprint image (315) corresponding to the user's fingerprint by using a sensor (310) included in the electronic device (300).

[0086] The electronic device (300) can convert an input fingerprint image (315) into an input fingerprint template (325), which is an embedding vector, in operation 420.

[0087] A template may refer to text data that extracts feature information from a fingerprint image. The template may be in a form that allows for the verification of similarity between distinct fingerprint images.

[0088] In operation 430, the electronic device (300) can obtain a comparison value by comparing an input fingerprint template (325) and a fingerprint template included in a fingerprint template database (330) (e.g., fingerprint templates (301, 302, 303) of FIG. 3).

[0089] The fingerprint templates included in the fingerprint template database (330) (e.g., fingerprint templates (301, 302, 303) of FIG. 3) may refer to a normal fingerprint template (301), an abnormal fingerprint template (302), and a fake fingerprint template (303). The comparison values ​​(e.g., a first comparison value, a second comparison value, a third comparison value) may refer to values ​​indicating a similarity between two templates. For example, the comparison values ​​(e.g., a first comparison value, a second comparison value, a third comparison value) may be set to have a higher value as the similarity between the two templates increases. The comparison values ​​(e.g., a first comparison value, a second comparison value, a third comparison value) may be set to have a lower value as the similarity between the two templates decreases.

[0090] The electronic device (300) can obtain a first comparison value by comparing the input fingerprint template (325) and the normal state fingerprint template (301). The electronic device (300) can obtain a second comparison value by comparing the input fingerprint template (325) of the input fingerprint image (315) with the abnormal state fingerprint template (302). The electronic device (300) can obtain a third comparison value by comparing the input fingerprint template (325) of the input fingerprint image (315) with a fake fingerprint template (303). The fingerprint template database (330) includes a plurality of fingerprint templates, and there can be a plurality of fingerprint templates compared with the input fingerprint template (325) of the input fingerprint image (315) to obtain the comparison values ​​(e.g., the first comparison value, the second comparison value, the third comparison value).

[0091] The electronic device (300) can obtain comparison values ​​(e.g., a first comparison value, a second comparison value, a third comparison value) based on the result of classifying (or scoring) the fingerprint templates included in the normal state fingerprint template. For example, the electronic device (300) can perform matching between fingerprint templates included in the normal state fingerprint template (301). Matching may refer to a comparison value obtained by comparing fingerprint templates exceeding a threshold value. The electronic device (300) can classify (or score) the fingerprint templates included in the normal state fingerprint template (301) in descending order of the number of successful matching attempts. For example, the electronic device (300) can set the fingerprint template with the highest number of successful matching attempts as the first normal state fingerprint template, and set the fingerprint template with the number of successful matching attempts corresponding to a median value among the fingerprint templates as the average normal state fingerprint template. When performing fingerprint authentication, the electronic device (300) can obtain a first comparison value by comparing a first normal state fingerprint template with an input fingerprint template (325). When performing fingerprint authentication, the electronic device (300) can obtain a first comparison value by comparing an average normal state fingerprint template with an input fingerprint template (325). The electronic device (300) can perform fingerprint authentication based on a fingerprint template that satisfies a predetermined condition among the fingerprint templates included in the normal state fingerprint template (301) as well as the first normal state fingerprint template and the average normal state fingerprint template. The electronic device (300) can also perform the above operation for the abnormal state fingerprint template (302) and the fake fingerprint template (303), and a redundant description will be omitted.

[0092] The electronic device (300) may store the template of the input fingerprint image (315) and the template of the virtual fingerprint image generated based on the input fingerprint image in the fingerprint template database (330) when the input fingerprint image (315) is successfully authenticated. When the input fingerprint template (325) is stored in the fingerprint template database (330), the electronic device (300) may perform matching between the fingerprint templates to reclassify (or score) the fingerprint template. The first comparison value may be a value indicating the degree of similarity between the input fingerprint image (315) and the user's normal state fingerprint image (401). The higher the first comparison value, the more similar the input fingerprint image (315) and the user's normal state fingerprint image (401) may be. The second comparison value may be a value indicating the degree of similarity between the input fingerprint image (315) and the user's abnormal state fingerprint image (402). The higher the second comparison value, the more similar the input fingerprint image (315) and the user's abnormal fingerprint image (402) may be. The third comparison value may be a value indicating the degree of similarity between the input fingerprint image (315) and the user's fake fingerprint image. The higher the third comparison value, the more similar the input fingerprint image (315) and the user's fake fingerprint image (403) may be.

[0093] The electronic device (300) can, in operation 440, determine whether the first comparison value is greater than or equal to a threshold value. The threshold value may be a value indicating that two fingerprint images corresponding to the comparison values ​​are fingerprints of the same person. If the first comparison value is greater than or equal to the threshold value, the electronic device (300) can recognize the input fingerprint image (315) as a fingerprint image of the same person as the user of the normal fingerprint image (401). If the first comparison value is greater than or equal to the threshold value, the electronic device (300) can determine that the input fingerprint image (315) matches the normal fingerprint image (401).

[0094] The electronic device (300) can, in operation 450, determine whether the third comparison value is greater than or equal to the first comparison value in response to the first comparison value being greater than or equal to the threshold value.

[0095] The electronic device (300) can determine the input fingerprint image (315) as a normal fingerprint image (401) in response to the first comparison value being greater than or equal to a threshold value and the third comparison value being less than the first comparison value. The electronic device (300) can recognize that the input fingerprint image (315) does not match the fake fingerprint image (403) in response to the third comparison value being less than the first comparison value. The electronic device (300) can determine the input fingerprint image that matches the user's fingerprint image of the normal fingerprint image (401) and does not match the fake fingerprint image (403) as the normal fingerprint image (401). However, the operation of the electronic device (300) comparing the third comparison value and the first comparison value is only one example of the operation of determining whether the fake fingerprint image (403) and the input fingerprint image (315) do not match. As an example, the electronic device may replace the 450 operation with an operation to determine whether the third comparison value is greater than or equal to the threshold, in response to the first comparison value being greater than or equal to the threshold.

[0096] The electronic device (300) can, in operation 455, check whether the number of comparisons of the third comparison value and the first comparison value exceeds the maximum number of repetitions in response to the first comparison value being greater than or equal to the threshold value and the third comparison value being greater than or equal to the first comparison value.

[0097] The electronic device (300) can recognize that the input fingerprint image (315) matches the user's fake fingerprint image (403) in response to the third comparison value being greater than or equal to the first comparison value. If the number of comparisons for comparing the third comparison value and the first comparison value exceeds the maximum number of repetitions, the electronic device (300) can determine the input fingerprint image (315) as a fake fingerprint. If the number of comparisons for comparing the third comparison value and the first comparison value does not exceed the maximum number of repetitions, the electronic device (300) can add 1 to the number of comparisons (the number of comparisons for comparing the third comparison value and the first comparison value) in operation 456 and re-perform the operation of classifying the input fingerprint image (315) from operation 410.

[0098] In operation 460, the electronic device (300) can determine whether the second comparison value is greater than or equal to the threshold value in response to the first comparison value being less than or equal to the threshold value.

[0099] The electronic device (300) may determine the input fingerprint image (315) as an unregistered fingerprint image (404) in response to the first comparison value being less than the threshold value and the second comparison value being less than the threshold value. The unregistered fingerprint image (404) is a fingerprint image that is not registered in the electronic device (300) and may refer to a fingerprint image other than the user's fingerprint. The electronic device (300) may recognize that the input fingerprint image (315) does not match the user's abnormal fingerprint image (402) in response to the second comparison value being less than the threshold value. The electronic device (300) may determine the input fingerprint image that does not match the user's fingerprint image of the normal fingerprint image (401) and does not match the abnormal fingerprint image (402) as an unregistered fingerprint image (404).

[0100] In operation 465, the electronic device (300) can determine whether the number of comparisons between the second comparison value and the threshold value exceeds the maximum number of repetitions in response to the first comparison value being less than the threshold value and the second comparison value being greater than or equal to the threshold value. The maximum number of repetitions of operation 465 may be a value different from the maximum number of repetitions of operation 455.

[0101] The electronic device (300) can recognize that the input fingerprint image (315) matches the user's abnormal fingerprint image (402) in response to the second comparison value being greater than or equal to the threshold value. If the number of comparisons for comparing the second comparison value and the threshold value exceeds the maximum number of repetitions, the electronic device (300) can determine the input fingerprint image (315) as an abnormal fingerprint. If the number of comparisons for comparing the second comparison value and the threshold value does not exceed the maximum number of repetitions, the electronic device (300) can add 1 to the number of comparisons (the number of comparisons for comparing the second comparison value and the threshold value) in operation 466, and re-perform the operation of classifying the input fingerprint image (315) from operation 410.

[0102] The method described above is merely a method for an electronic device (300) according to an example to classify an input fingerprint image, and the electronic device (300) can classify an input fingerprint image based on various methods. The electronic device (300) can classify an input fingerprint image into an unregistered fingerprint image (404), a normal fingerprint image (401), an abnormal fingerprint image (402), and a fake fingerprint image (403) based on the first comparison value, the second comparison value, and the third comparison value.

[0103] According to one embodiment, the electronic device (300) may classify the input fingerprint image (315) according to the operation of FIG. 4A, and may determine the fingerprint authentication result based on the classification result. The electronic device (300) may determine the fingerprint authentication as failed if the input fingerprint image (315) is an unregistered fingerprint image (404) or a fake fingerprint image (403). The electronic device (300) may determine the fingerprint authentication as successful if the input fingerprint image (315) is a normal fingerprint image (401) or an abnormal fingerprint image (402).

[0104] The electronic device (300) can perform fingerprint authentication based on the results of comparing the input fingerprint image (315) with not only the normal fingerprint image (401), but also the abnormal fingerprint image (402) and the fake fingerprint image (403). By comparing the abnormal fingerprint image (402) with the input fingerprint image (315), the electronic device (300) can accurately obtain a fingerprint authentication result even if the input fingerprint image (315) is a fingerprint image acquired with an obstacle element. By comparing the fake fingerprint image (403) with the input fingerprint image (315), the electronic device (300) can accurately obtain a fingerprint authentication result even if the input fingerprint image (315) is a fake fingerprint image (403).

[0105] FIG. 4b is a diagram illustrating a result of an electronic device (300) classifying an input fingerprint image (315) according to one embodiment.

[0106] The electronic device (300) can determine an input fingerprint image (315) in which the first comparison value is greater than or equal to a threshold value and the third comparison value is less than the first comparison value as a normal state fingerprint image (401).

[0107] The electronic device (300) can determine an input fingerprint image (315) in which the first comparison value is greater than or equal to a threshold value and the third comparison value is greater than or equal to the first comparison value as a fake fingerprint image (403).

[0108] The electronic device (300) can determine an input fingerprint image (315) in which the first comparison value is less than the threshold value and the second comparison value is less than the threshold value as an unregistered fingerprint image (404).

[0109] The electronic device (300) can determine an input fingerprint image (315) in which the first comparison value is less than the threshold value and the second comparison value is greater than the threshold value as an abnormal fingerprint image (402).

[0110] The electronic device (300) may determine fingerprint authentication as failed if the input fingerprint image (315) is an unregistered fingerprint image (404) or a fake fingerprint image (403). The electronic device (300) may determine fingerprint authentication as successful if the input fingerprint image (315) is a normal fingerprint image (401) or an abnormal fingerprint image (402).

[0111] FIG. 5 is a flowchart of an operation of an electronic device updating a fingerprint template database according to one embodiment.

[0112] The electronic device (300), in operation 510, can store an input fingerprint template (325), which is a template of the input fingerprint image (315), in a fingerprint template database (330) when the input fingerprint image (315) is a normal fingerprint image (401), an abnormal fingerprint image (402), or a fake fingerprint image (403).

[0113] If the input fingerprint image (315) is a normal fingerprint image (401), the electronic device (300) can store the input template (325) as a normal fingerprint template (301) in the fingerprint template database (330). If the input fingerprint image (315) is an abnormal fingerprint image (402), the electronic device (300) can store the input fingerprint template (325) as an abnormal fingerprint template (302) in the fingerprint template database (330). If the input fingerprint image (315) is a fake fingerprint image (403), the electronic device (300) can store the input fingerprint template (325) as a fake fingerprint template (303) in the fingerprint template database (330).

[0114] The electronic device (300), in operation 520, can load a fingerprint generation artificial intelligence model (370) and a target image (e.g., target image (730) of FIG. 7) into a secure area when the input fingerprint image (315) is a normal fingerprint image (401), an abnormal fingerprint image (402), or a fake fingerprint image (403).

[0115] A target image (e.g., target image (730) of FIG. 7) may be data indicating environmental characteristics of a virtual fingerprint image generated by a fingerprint generation artificial intelligence model (370). Environmental characteristics may refer to environmental characteristics at the time of fingerprint registration or fingerprint authentication, and may include, for example, environmental characteristics related to dryness or moisture, or the material of an object into which a fingerprint is input. For example, environmental characteristics may include characteristics related to the time of fingerprint authentication (e.g., morning and afternoon). For example, environmental characteristics may include characteristics related to the weather (e.g., cloudy, clear, rainy, snowy, foggy, cloudy) when fingerprint authentication is performed. For example, environmental characteristics may include characteristics related to the season (e.g., spring, summer, fall, winter) when fingerprint authentication is performed. For example, environmental characteristics may include characteristics related to the user's behavior (e.g., showering, exercising, hiking, driving, cooking, washing dishes) when fingerprint authentication is performed. For example, the environmental characteristics may include characteristics related to the location where fingerprint authentication is performed (e.g., indoor / outdoor, desert / tropical region / Mediterranean Sea / Arctic, airplane, countryside / city). For example, the environmental characteristics may include characteristics related to the health status of the user performing fingerprint authentication (e.g., obesity / normal weight, blood sugar, heart rate). The electronic device (300) may identify the environmental characteristics at the time when the user performs fingerprint authentication based on the user information included in the electronic device (e.g., the user's health information of a health application), and may generate a virtual fingerprint image based on a target image (e.g., the target image (730) of FIG. 7) corresponding to the identified environmental characteristics.

[0116] For example, if the target image (e.g., the target image (730) of FIG. 7) is an abnormal (or fake) fingerprint image, the fingerprint generation artificial intelligence model (370) may generate an abnormal (or fake) virtual fingerprint image. However, since the fingerprint generation artificial intelligence model (370) generates a virtual fingerprint image of a user corresponding to the input fingerprint image (315), the target image may not instruct the fingerprint generation artificial intelligence model (370) to generate a virtual fingerprint image of a person having a fingerprint corresponding to the target image (e.g., the target image (730) of FIG. 7).

[0117] An area on a memory (e.g., memory (1320) of FIG. 13) included in an electronic device (300) may include a secure area and a general area. The secure area may refer to an area in which an operation of the electronic device (300) can be performed while maintaining security. The electronic device (300) may restrict external access to a virtual fingerprint image generated by the fingerprint generation artificial intelligence model (370) by loading a fingerprint generation artificial intelligence model (370) and a target image (e.g., target image (730) of FIG. 7) into the secure area. For example, the secure area may be an area separately provided within the processor (110). The secure area may be an area (e.g., an embedded secure element (eSE), a secure processor) separately provided in addition to the processor (110). For example, the secure area may be an ARM TM Trustzone developed by Saga TM ) may apply. For example, the security zone may be implemented as a hypervisor.

[0118] The electronic device (300), in operation 530, can generate a virtual fake fingerprint image based on the input fingerprint image (315) when the input fingerprint image (315) is a normal fingerprint image (401) or a fake fingerprint image (403), and store a template of the virtual fake fingerprint image as a fake fingerprint template (303).

[0119] In operation 540, the electronic device (300) can generate a virtual abnormal state fingerprint image based on the input fingerprint image (315) when the input fingerprint image (315) is a normal state fingerprint image (401) or an abnormal state fingerprint image (402), and store a template of the virtual abnormal state fingerprint image as an abnormal state fingerprint template (302).

[0120] The electronic device (300) can generate a virtual fingerprint image representing a fingerprint of the same person as the input fingerprint image (315). The electronic device (300) can input the input fingerprint image (315) into a fingerprint generation artificial intelligence model (370) and generate a virtual fingerprint image that maintains the structural characteristics of the input fingerprint image (315). Since the structural characteristics of the input fingerprint image (315) are maintained, the virtual fingerprint image can be a fingerprint image representing a fingerprint of the same person as the input fingerprint image (315). The electronic device (300) can input the input fingerprint image (315) into the fingerprint generation artificial intelligence model (370) and generate a virtual fingerprint image that maintains the environmental characteristics of the target image.

[0121] In operation 550, the electronic device (300) can unload the fingerprint generation artificial intelligence model (370) and the target image from the secure area if the input fingerprint image (315) is a normal fingerprint image (401), an abnormal fingerprint image (402), or a fake fingerprint image (403).

[0122] It should be understood that the operation of the electronic device (300) generating a virtual fingerprint image using the fingerprint generation artificial intelligence model (370) in the secure area is an operation to maintain security for the generated virtual fingerprint image, and that the fingerprint generation artificial intelligence model (370) is not only usable in the secure area. For example, the fingerprint generation artificial intelligence model (370) can generate a virtual fingerprint image even in the general area.

[0123] FIG. 6 is a diagram illustrating an operation of an electronic device updating a fingerprint template according to one embodiment.

[0124] The update judgement unit (611, 612, 613, 621, 622, 631, 632) can perform an operation of checking the image quality of an input fingerprint image (315) (e.g., a normal fingerprint image (401), an abnormal fingerprint image (402), a fake fingerprint image (403)). The image quality can be determined based on the degree to which the structural features of the input fingerprint image (315) can be recognized. The update judgement unit (611, 612, 613, 621, 622, 631, 632) can determine the image quality of the fingerprint image (e.g., input fingerprint image (315), virtual fingerprint image (412, 413, 422, 433)) as an image quality that instructs the generation of the virtual fingerprint image (412, 413, 422, 433) if the fingerprint image (e.g., input fingerprint image (315), virtual fingerprint image (412, 413, 422, 433)) can recognize structural features.

[0125] The update decision unit (611, 612, 613, 621, 622, 631, 632) can determine whether to store a fingerprint image (e.g., an input fingerprint image (315), a virtual fingerprint image (412, 413, 422, 433)) in the fingerprint template database (330) based on the image quality of the fingerprint image (e.g., an input fingerprint image (315), a virtual fingerprint image (412, 413, 422, 433)).

[0126] The update judgement unit (611, 612, 613, 621, 622, 631, 632) can compare an input fingerprint image (315) (e.g., a normal fingerprint image (401), an abnormal fingerprint image (402), a fake fingerprint image (403)) with a fingerprint template (e.g., a normal fingerprint template (301), an abnormal fingerprint template (302), a fake fingerprint template (303)) included in a fingerprint template database (330). The update judgement unit (611, 612, 613, 621, 622, 631, 632) can determine a fingerprint image for generating a virtual fingerprint image (412, 413, 422, 433) based on the input fingerprint image (315) if the similarity between the input fingerprint image (315) and the fingerprint templates (e.g., normal fingerprint template (301), abnormal fingerprint template (302), fake fingerprint template (303)) included in the fingerprint template database (330) is greater than a threshold value. The update judgement unit (611, 612, 613, 621, 622, 631, 632) can determine whether to store the template of the fingerprint image (e.g., input fingerprint image (315) and virtual fingerprint image (412, 413, 422, 433)) in the fingerprint template database (330) based on whether the similarity of the fingerprint image (e.g., input fingerprint image (315) and virtual fingerprint image (412, 413, 422, 433)) compared with the fingerprint template included in the fingerprint template database is greater than a threshold value.

[0127] The update decision unit (611, 612, 613, 621, 622, 631, 632) can determine whether to store a fingerprint image (e.g., an input fingerprint image (315), a virtual fingerprint image (412, 413, 422, 433)) in the fingerprint template database based on at least one of the similarity and image quality with a fingerprint template (e.g., a normal fingerprint template (301), an abnormal fingerprint template (302), a fake fingerprint template (303)) included in the fingerprint template database (330). The update judgement unit (611, 612, 613, 621, 622, 631, 632) can determine whether to generate a virtual fingerprint image (412, 413, 422, 433) based on a fingerprint image (e.g., an input fingerprint image (315), a virtual fingerprint image (412, 413, 422, 433)) based on at least one of the similarity and image quality with a fingerprint template (e.g., a normal fingerprint template (301), an abnormal fingerprint template (302), a fake fingerprint template (303)) included in the fingerprint template database (330). The similarity and image quality of the fingerprint templates (e.g., normal fingerprint template (301), abnormal fingerprint template (302), fake fingerprint template (303)) included in the fingerprint template database described above are only examples, and the update judgement unit (611, 612, 613, 621, 622, 631, 632) can determine whether to generate a virtual fingerprint image and whether to store the fingerprint image (e.g., input fingerprint image (315), virtual fingerprint image (412, 413, 422, 433)) in the fingerprint template database (330) based on a combination of various conditions including examples.

[0128] The fingerprint generation artificial intelligence model (615, 616, 625, 635) can generate a virtual fingerprint image (412, 413, 422, 433) based on an input fingerprint image (315) having an image quality higher than a specified value. In addition, the fingerprint image that can be input to the fingerprint generation artificial intelligence model (615, 616, 625, 635) is not limited to the input fingerprint image (315). The fingerprint generation artificial intelligence model (615, 616, 625, 635) can also generate a virtual fingerprint image that maintains the structural characteristics of the generated virtual fingerprint image (412, 413, 422, 433).

[0129] The operations of the update judgement unit (611, 612, 613, 621, 622, 631, 632) and the fingerprint generation artificial intelligence model (615, 616, 625, 635) can be executed by a processor included in the electronic device.

[0130] According to FIG. 4A, an input fingerprint image (315) can be classified into an abnormal fingerprint image (402), a normal fingerprint image (401), or a fake fingerprint image (403) depending on an operation of the electronic device to classify the input fingerprint image (315). Hereinafter, an operation of the electronic device (300) to generate virtual fingerprint images (412, 413, 422, 433) based on an input fingerprint image (315) classified as an abnormal fingerprint image (402), a normal fingerprint image (401), or a fake fingerprint image (403) will be described.

[0131] The electronic device (300) can determine the image quality of the first fingerprint image (401) based on the update decision unit (611) if the input fingerprint image (315) is a first fingerprint image (401) classified as a normal fingerprint image (401). If the first fingerprint image (401) is clearly acquired and structural features can be recognized, the electronic device (300) can determine the first fingerprint image (401) as having an image quality that instructs generation of a virtual fingerprint image. The electronic device (300) can store the first fingerprint image (401) whose determined image quality is equal to or higher than a specified value as a normal fingerprint template (301) in the fingerprint template database (330).

[0132] The operation of the update judgement device (611) determining whether to generate a virtual fingerprint image based on the image quality of the first fingerprint image (401) and the operation of determining whether to store the first fingerprint image (401) in the fingerprint template are exemplary, and the update judgement device (611) can determine whether to generate a virtual fingerprint image or whether to store the first fingerprint image (401) in the normal state fingerprint template (301) based on set conditions.

[0133] The electronic device (300) can generate a virtual abnormal fingerprint image (412) based on a first fingerprint image (401) having an image quality equal to or higher than a specified value using a first fingerprint generation artificial intelligence model (615). In addition, the electronic device (300) can additionally generate a virtual abnormal fingerprint image (412) based on the generated virtual abnormal fingerprint image (412) using the first fingerprint generation artificial intelligence model (615). The electronic device (300) can verify the image quality of the generated virtual abnormal fingerprint image (412) based on an update judgement unit (612). If the verified image quality is equal to or higher than a specified value, the electronic device (300) can store the generated virtual abnormal fingerprint image (412) as an abnormal fingerprint template (302) of a fingerprint template database (330). The operation of the update judgement unit (612) determining whether to generate a virtual fingerprint image based on the image quality of the virtual abnormal state fingerprint image (412) and the operation of determining whether to store the virtual abnormal state fingerprint image (412) in the fingerprint template are exemplary, and the update judgement unit (612) can determine whether to generate a virtual fingerprint image or whether to store the virtual abnormal state fingerprint image (412) in the abnormal state fingerprint template (302) based on a set condition.

[0134] The electronic device (300) can train the first fingerprint generation artificial intelligence model (615) to generate an image similar to the generated virtual abnormal state fingerprint image (412) based on the generated virtual abnormal state fingerprint image (412) when the verified image quality is higher than a specified value.

[0135] The electronic device (300) can generate a virtual fake fingerprint image (413) based on a first fingerprint image (401) having an image quality equal to or higher than a specified value using a second fingerprint generation artificial intelligence model (616). In addition, the electronic device (300) can additionally generate a virtual fake fingerprint image (413) based on the generated virtual fake fingerprint image (413) using the second fingerprint generation artificial intelligence model (616). The electronic device (300) can verify the image quality of the generated virtual fake fingerprint image (413) based on an update judgement unit (613). If the verified image quality is equal to or higher than a specified value, the electronic device (300) can store the generated virtual fake fingerprint image (413) as a fake fingerprint template (303) in the fingerprint template database (330). The operation of the update judgement unit (613) determining whether to generate a virtual fingerprint image based on the image quality of the virtual fake fingerprint image (413) and the operation of determining whether to store the virtual fake fingerprint image (413) in the fingerprint template are exemplary, and the update judgement unit (613) can determine whether to generate a virtual fingerprint image or whether to store the virtual fake fingerprint image (413) in the fake fingerprint template (303) based on set conditions.

[0136] The electronic device (300) can train a second fingerprint generation artificial intelligence model (616) to generate an image similar to the generated fake state fingerprint image (413) based on the generated virtual fake fingerprint image (413) when the verified image quality is higher than a specified value.

[0137] The electronic device (300) can, based on the update decision unit (621), determine the image quality of the second fingerprint image (402) when the input fingerprint image (315) is a second fingerprint image (402) classified as an abnormal fingerprint image (402). If the second fingerprint image (402) is clearly acquired and structural features can be recognized, the electronic device (300) can determine the second fingerprint image (402) to have an image quality that instructs generation of a virtual fingerprint image. The electronic device (300) can store the second fingerprint image (402) whose determined image quality is equal to or higher than a specified value as an abnormal fingerprint template (302) in the fingerprint template database (330). The operation of the update judgement device (621) determining whether to generate a virtual fingerprint image based on the image quality of the second fingerprint image (402) and the operation of determining whether to store the second fingerprint image (402) in the abnormal state fingerprint template (302) are exemplary, and the update judgement device (621) can determine whether to generate a virtual fingerprint image or whether to store the second fingerprint image (402) in the abnormal state fingerprint template (302) based on set conditions.

[0138] The electronic device (300) can generate a virtual abnormal fingerprint image (422) based on a second fingerprint image (402) having an image quality equal to or higher than a specified value using a third fingerprint generation artificial intelligence model (625). In addition, the electronic device (300) can additionally generate a virtual abnormal fingerprint image (422) based on the generated virtual abnormal fingerprint image (422) using the third fingerprint generation artificial intelligence model (625). The electronic device (300) can verify the image quality of the generated virtual abnormal fingerprint image (422) based on an update judgement unit (622). If the verified image quality is equal to or higher than a specified value, the electronic device (300) can store the generated virtual abnormal fingerprint image (422) as an abnormal fingerprint template (302) in the fingerprint template database (330). The operation of the update judgement unit (622) determining whether to generate a virtual fingerprint image based on the image quality of the virtual abnormal state fingerprint image (422) and the operation of determining whether to store the virtual abnormal state fingerprint image (422) in the abnormal state fingerprint template (302) are exemplary, and the update judgement unit (622) can determine whether to generate a virtual fingerprint image or whether to store the virtual abnormal state fingerprint image (422) in the abnormal state fingerprint template (302) based on a set condition.

[0139] The electronic device (300) can train a third fingerprint generation artificial intelligence model (625) to generate an image similar to the generated virtual abnormal state fingerprint image (422) based on the generated virtual abnormal state fingerprint image (422) when the verified image quality is higher than a specified value.

[0140] The electronic device (300) can, based on the update decision unit (631), determine the image quality of the third fingerprint image (403) if the input fingerprint image (315) is a third fingerprint image (403) classified as a fake fingerprint image (403). If the third fingerprint image (403) is clearly acquired and structural features can be recognized, the electronic device (300) can determine the third fingerprint image (403) to have an image quality that instructs generation of a virtual fingerprint image. The electronic device (300) can store the third fingerprint image (403) whose determined image quality is equal to or higher than a specified value as a fake fingerprint template (303) in the fingerprint template database (330). The operation of the update judgement unit (631) determining whether to generate a virtual fingerprint image based on the image quality of the third fingerprint image (403) and the operation of determining whether to store the third fingerprint image (403) in the fake fingerprint template (303) are exemplary, and the update judgement unit (631) can determine whether to generate a virtual fingerprint image or whether to store the third fingerprint image (403) in the fake fingerprint template (303) based on set conditions.

[0141] The electronic device (300) can generate a virtual fake fingerprint image (433) based on a third fingerprint image (403) having an image quality equal to or higher than a specified value using a fourth fingerprint generation artificial intelligence model (635). In addition, the electronic device (300) can additionally generate a virtual fake fingerprint image (433) based on the generated virtual fake fingerprint image (433) using the fourth fingerprint generation artificial intelligence model (635). The electronic device (300) can verify the image quality of the generated virtual fake fingerprint image (433) based on an update judgement unit (632). If the verified image quality is equal to or higher than a specified value, the electronic device (300) can store the generated virtual fake fingerprint image (433) as a fake fingerprint template (303) in the fingerprint template database (330). The operation of the update judgement unit (632) determining whether to generate a virtual fingerprint image based on the image quality of the virtual fake fingerprint image (433) and the operation of determining whether to store the virtual fake fingerprint image (433) in the fake fingerprint template (303) are exemplary, and the update judgement unit (632) can determine whether to generate a virtual fingerprint image or whether to store the virtual fake fingerprint image (433) in the fake fingerprint template (303) based on set conditions.

[0142] The electronic device (300) can train the fourth fingerprint generation artificial intelligence model (635) to generate an image similar to the generated virtual fake fingerprint image (433) based on the generated virtual fake fingerprint image (433) when the verified image quality is higher than a specified value.

[0143] The electronic device (300) can perform fingerprint authentication based on not only the registered fingerprint image but also the input fingerprint image (315) by storing the input template in the fingerprint template database (330). Since the electronic device (300) performs fingerprint authentication based on the recently input user fingerprint image (315), it can perform accurate fingerprint authentication even if the user's fingerprint changes over time. In addition, the electronic device (300) can generate virtual fingerprint images (412, 413, 422, 433) that maintain the structural characteristics of the input fingerprint image (315), and perform fingerprint authentication based on the templates of the virtual fingerprint images (412, 413, 422, 433). Since the number of virtual fingerprint images (412, 413, 422, 433) that can be generated is not limited, the electronic device (300) can secure a sufficient amount of fingerprint templates. In particular, although it is difficult to obtain data related to abnormal state fingerprint images (402) and fake fingerprint images (403), the electronic device (300) can generate virtual abnormal state fingerprint images (422, 412) and virtual fake fingerprint images (433, 413) based on the normal state fingerprint image (401).

[0144] The electronic device (300) can train a fingerprint generation artificial intelligence model (615, 616, 625, 635) using an input fingerprint image (315) and / or a virtual fingerprint image (412, 413, 422, 433) as learning data. Since the model is trained based on the user's input fingerprint image (315) or a virtual fingerprint image (412, 413, 422, 433) similar to the user's input fingerprint image, the performance of the fingerprint generation artificial intelligence model (615, 616, 625, 635) can be gradually improved depending on the user's fingerprint authentication attempt.

[0145] FIG. 7 is a diagram illustrating a fingerprint generation artificial intelligence model of an electronic device according to one embodiment.

[0146] A fingerprint generation artificial intelligence model (720) (e.g., fingerprint generation artificial intelligence model (370) of FIG. 3, fingerprint generation artificial intelligence models (615, 616, 625, 635) of FIG. 6) can receive an input fingerprint image (710) (e.g., input fingerprint image (315) of FIG. 3) and generate a virtual fingerprint image (740) (e.g., virtual fingerprint images (412, 413, 422, 433) of FIG. 6) that maintains the structural characteristics of the input fingerprint image (710). For example, the fingerprint generation artificial intelligence model (720) can generate a virtual fingerprint image (740) that represents a fingerprint of the same person as the input fingerprint image (710). The fingerprint generation artificial intelligence model (720) can receive one input fingerprint image (710) and generate at least one virtual fingerprint image (740). Additionally, the fingerprint generation artificial intelligence model (720) can generate a separate virtual fingerprint image (740) based on the generated virtual fingerprint image (740).

[0147] The fingerprint generation artificial intelligence model (720) can generate a virtual fingerprint image (740) that maintains the environmental characteristics of the target image (730). The environmental characteristics may refer to environmental characteristics at the time of fingerprint registration or fingerprint authentication, and may include, for example, environmental characteristics of dryness, moisture, or the material of the object into which the fingerprint is input. For example, if the target image (730) is an abnormal (or fake) fingerprint image, the fingerprint generation artificial intelligence model (720) can generate an abnormal (or fake) virtual fingerprint image (740).

[0148] The fingerprint generation artificial intelligence model (720) can be trained based on an input fingerprint image (710) and / or a virtual fingerprint image (740). For example, the fingerprint generation artificial intelligence model (720) can be trained to generate an image similar to the input fingerprint image (710) and / or the virtual fingerprint image (740). As the fingerprint generation artificial intelligence model (720) is trained based on the input fingerprint image (710) and / or the virtual fingerprint image (740) each time a new input fingerprint image (710) is input, the fingerprint generation artificial intelligence model (720) can generate a virtual fingerprint image (740) having a high similarity to the user's fingerprint.

[0149] The fingerprint generation artificial intelligence model (720) may be trained externally or within the electronic device. The fingerprint generation artificial intelligence model (720) may be a pre-trained artificial intelligence model and may be additionally trained externally or within the electronic device (300). If additional training is performed externally, the fingerprint generation artificial intelligence model (720) within the electronic device (300) may be updated with the additionally trained fingerprint generation artificial intelligence model from an external server. However, information included in the fingerprint generation artificial intelligence (720) within the electronic device may be combined with the updated fingerprint generation artificial intelligence model. The updated fingerprint generation artificial intelligence model may apply the training results of the existing fingerprint generation artificial intelligence model (720).

[0150] The fingerprint generation artificial intelligence model (720) may be adjusted by removing parameters with low importance from the fingerprint generation artificial intelligence model (720), thereby reducing the number of parameters included in the fingerprint generation artificial intelligence model (720). In addition, the fingerprint generation artificial intelligence model (720) may be adjusted according to knowledge distillation, low-rank approximation, neural architecture search, quantization, and pruning during training methods.

[0151] An electronic device may generate a prompt to generate a virtual image (e.g., a virtual fingerprint image (740)) based on at least one image (e.g., a fingerprint image), an index related to at least one image (e.g., a fingerprint image), and parameter information related to the image (e.g., a fingerprint image). The image (e.g., a fingerprint image, a virtual fingerprint image (740)) may be used as a prompt source for generating the virtual fingerprint image (740). The image may include image-based content. The electronic device may generate a prompt to generate the virtual fingerprint image (740) based on an input fingerprint image (710) input through a sensor (170, 310).

[0152] An electronic device can generate (or obtain) a virtual fingerprint image (740) in relation to a generated prompt. The electronic device can input a prompt source (e.g., an input fingerprint image (710)) related to the generation of the virtual fingerprint image (740) based on an interaction with a user, and can generate (e.g., regenerate or reconstruct) a virtual fingerprint image (740) in a server or on-device based on the prompt source. The electronic device can provide a prompt to a generative artificial intelligence of the on-device and / or the server to execute a process for generating a virtual fingerprint image (740) based on the generated prompt. The electronic device can provide a prompt (e.g., a question or instruction input to the generative artificial intelligence) requesting the generation of a virtual fingerprint image (740) to the generative artificial intelligence. The electronic device can generate (or obtain) a virtual fingerprint image (740) according to the process for generating a virtual fingerprint image (740) executed in relation to the prompt in the on-device artificial intelligence. The electronic device can receive (or acquire) a virtual fingerprint image (740) from the server according to a virtual fingerprint image (740) generation process executed in connection with a prompt in the server artificial intelligence.

[0153] Figure 8 is a flowchart of an operation for learning an ASP module according to one embodiment.

[0154] The electronic device (300) can, in operation 810, generate a virtual abnormal fingerprint image (412, 422) and a virtual fake fingerprint image (413, 433) and store them in the fingerprint template database (330).

[0155] The electronic device (300) can store an input fingerprint image (315) and a template of a virtual fingerprint image generated based on the input fingerprint image (315) in a fingerprint template database (330). The operation of generating a virtual fingerprint image based on the classification of the input fingerprint image is omitted as it has been described in FIGS. 5 and 6.

[0156] The electronic device (300) can load the ASP module (350) into the secure area in operation 820.

[0157] The electronic device (300) can train the ASP module (350) based on the normal state fingerprint template (301), the abnormal state fingerprint template (302), and the fake fingerprint template (303) in operation 830.

[0158] The electronic device (300) can train the ASP module (350) based on the templates of the input fingerprint images (315) corresponding to the normal state fingerprint template (301), the abnormal state fingerprint template (302), and the fake fingerprint template (303) and the virtual fingerprint images generated based on the input fingerprint images (315). The data that the electronic device (300) uses to train the ASP module (350) will be described in FIG. 9.

[0159] The electronic device (300) can update the existing ASP module (350) with the learned ASP module (350) in operation 840.

[0160] The electronic device (300) can, in operation 850, unload the learned ASP module (350) from the secure area.

[0161] FIG. 9 is a diagram illustrating an ASP module (350) according to one embodiment.

[0162] The ASP module (350) may be a model that distinguishes between a fake fingerprint image (903) and non-fake fingerprint images (901, 902). The ASP module (910) is a pre-trained artificial intelligence model, but may be fine-tuned within the electronic device (300). The following description of the ASP module (350) training is a description of the fine-tuning. The ASP module (350) may be trained based on a fingerprint image of a user of the electronic device (300). The ASP module (910) before training may be an artificial intelligence model that determines whether an unspecified fingerprint image is fake. The ASP module (910) before training may incorrectly determine whether some of the user's fingerprint images are fake and determine them as fake fingerprint images (903). For example, if the user's fingerprint has characteristics similar to a fingerprint forged with gelatin even though the user's fingerprint is not forged, the ASP module (910) before learning may determine the user's fingerprint image as a forged fingerprint image (903).

[0163] The electronic device (300) can train the ASP module (350) based on a normal state fingerprint template (301), an abnormal state fingerprint template (302), and a fake fingerprint template (303). The electronic device (300) can train the ASP module (350) based on a template of an input fingerprint image (315) corresponding to the normal state fingerprint template (301), the abnormal state fingerprint template (302), and the fake fingerprint template (303) and a virtual fingerprint image generated based on the input fingerprint image (315).

[0164] The electronic device (300) can train the ASP module (350) to determine that a normal fingerprint image (901) and an abnormal fingerprint image (902) are not forged fingerprint images based on a normal fingerprint template (301) and an abnormal fingerprint template (302). The abnormal fingerprint template (302) can include a template corresponding to not only the abnormal fingerprint image (902), which is an input fingerprint image, but also a virtual abnormal fingerprint image (e.g., the virtual abnormal fingerprint images (412, 422) of FIG. 6).

[0165] The electronic device (300) can train the ASP module (350) to determine a fake fingerprint image (903) as a fake fingerprint image (903) based on a fake fingerprint template (303). The fake fingerprint template (303) can include a template corresponding to a virtual fake fingerprint image (e.g., a virtual fake fingerprint image (413, 433) of FIG. 6) as well as a fake fingerprint image (903) which is an input fingerprint image.

[0166] The electronic device (300) can be trained to determine whether a fingerprint image is forged based on a user's fingerprint image. Since the electronic device (300) can generate a large number of virtual fingerprint images based on an input fingerprint image (315), it can acquire a large number of user fingerprint images for training the ASP module (350). The ASP module (920) trained based on a large number of user fingerprint images can have a high accuracy in determining whether a user's fingerprint image is forged.

[0167] FIG. 10 is a flowchart of an operation of an electronic device performing fingerprint authentication according to one embodiment.

[0168] The electronic device (300) can obtain an input fingerprint image in operation 1110. Operation 1110 may be the same as operation 410 described in FIG. 4A.

[0169] The electronic device (300) can convert an input fingerprint image into an input fingerprint template (325), which is an embedding vector, in operation 1120. Operation 1120 may be the same as operation 420 described in FIG. 4A.

[0170] In operation 1130, the electronic device (300) can obtain a comparison value by comparing the input fingerprint template (325) with a fingerprint template included in the fingerprint template database. Operation 1130 may be the same as operation 430 described in FIG. 4A.

[0171] The electronic device (300) can determine, in operation 1140, whether the first comparison value is greater than or equal to a threshold value. Operation 1140 may be the same as operation 440 described in FIG. 4A.

[0172] In operation 1150, the electronic device (300) can determine whether the third comparison value is greater than or equal to the first comparison value in response to the first comparison value being greater than or equal to the threshold value. Operation 1150 may be the same operation as operation 450 described in FIG. 4A.

[0173] The electronic device (300), in operation 1155, may change the threshold value to a change threshold value in response to the third comparison value being greater than or equal to the first comparison value. The electronic device (300) may set the change threshold value to have a larger value than the threshold value. The electronic device (300) may set the change threshold value to have a larger value as the difference between the third comparison value and the first comparison value increases. By setting the change threshold value to have a larger value than the threshold value, the electronic device (300) may determine that authentication has failed if the first comparison value is less than the change threshold value set to a value greater than the threshold value. The change threshold value may be expressed as in Mathematical Expression 1.

[0174]

[0175] If the first comparison value is greater than or equal to the threshold value and the third comparison value is greater than or equal to the first comparison value, the input fingerprint image may be similar to a fake fingerprint image. If the input fingerprint image is similar to a fake fingerprint image, the electronic device can determine the fingerprint authentication result as failed even in a situation where the degree of similarity between the input fingerprint image and the registered fingerprint image exceeds the threshold value by setting the threshold value high.

[0176] The electronic device (300) can, in operation 1160, determine whether the second comparison value is greater than or equal to the threshold value in response to the first comparison value being less than or equal to the threshold value. Operation 1160 may be the same as operation 460 described in FIG. 4A.

[0177] In operation 1165, the electronic device (300) may change the threshold value to a change threshold value in response to the second comparison value being greater than or equal to the threshold value. The electronic device (300) may set the change threshold value to have a value lower than the threshold value. The electronic device (300) may set the change threshold value to have a lower value as the difference between the first comparison value and the second comparison value increases. By setting the change threshold value to have a lower value than the threshold value, the electronic device (300) may determine that authentication is successful if the first comparison value is greater than or equal to the change threshold value set to a value lower than the threshold value. The change threshold value may be expressed as in mathematical expression 2.

[0178]

[0179] The electronic device (300) can, in operation 1170, check whether the first comparison value is greater than or equal to the change threshold value.

[0180] The electronic device (300) can determine the fingerprint authentication result as successful in response to the first comparison value being greater than or equal to the change threshold value.

[0181] The electronic device (300) can, in operation 1175, check whether the number of change threshold comparisons is less than the maximum number of repetitions when the first comparison value is less than the change threshold.

[0182] The electronic device (300) may determine the fingerprint authentication result as a failure if the number of change threshold comparisons is greater than or equal to the maximum number of repetitions.

[0183] The electronic device (300), in operation 1176, if the number of change threshold comparisons is less than the maximum number of repetitions, adds 1 to the number of change threshold comparisons and can perform the operation of performing fingerprint authentication again from operation 1110.

[0184] The electronic device (300) may determine that the input fingerprint image may be similar to the abnormal fingerprint image if the second comparison value is greater than or equal to the threshold value in operation 1160. If the input fingerprint image is similar to the abnormal fingerprint image, the electronic device may determine the fingerprint authentication result as successful even in a situation where the degree of similarity between the input fingerprint image and the registered fingerprint image does not exceed the threshold value by setting the threshold value lower (change threshold value in operation 1165).

[0185] FIG. 11 is a diagram illustrating a fingerprint authentication result of an input fingerprint image according to one embodiment.

[0186] The electronic device (300) can determine the fingerprint authentication result as successful when the first comparison value is greater than or equal to the threshold value and the third comparison value is less than the first comparison value.

[0187] The electronic device (300) may change the threshold value to a change threshold value having a value greater than the threshold value when the first comparison value is greater than or equal to the threshold value and the third comparison value is greater than or equal to the first comparison value. The electronic device (300) may determine fingerprint authentication as successful when the first comparison value is greater than or equal to the change threshold value, and the electronic device (300) may determine fingerprint authentication as failed when the first comparison value is less than the change threshold value.

[0188] The electronic device (300) may determine the fingerprint authentication result as failed if the first comparison value is less than the threshold value and the second comparison value is less than the threshold value. For example, the electronic device (300) may determine the fingerprint authentication result of an unregistered fingerprint image (1104) as failed.

[0189] The electronic device (300) may change the threshold value to a change threshold value having a value lower than the threshold value when the first comparison value is less than the threshold value and the second comparison value is greater than or equal to the threshold value. The electronic device (300) may determine the fingerprint authentication as successful in response to the first comparison value being greater than or equal to the change threshold value, and the electronic device (300) may determine the fingerprint authentication result as failed in response to the first comparison value being less than the change threshold value.

[0190] According to the fingerprint authentication result, a normal fingerprint image (1101) can be successfully authenticated. An abnormal fingerprint image (1102) can be authenticated because the first comparison value exceeds the change threshold even if it is below the threshold. A forged fingerprint image (1103) can fail to authenticate because the first comparison value is below the change threshold even if it is above the threshold. An unregistered fingerprint image (1104) can fail to authenticate.

[0191] FIG. 12 is a diagram illustrating an electronic device supporting multiple users according to one embodiment.

[0192] The electronic device (300) may include a plurality of authenticators corresponding to a plurality of users (e.g., a first user (1210), a second user (1220), and a third user (1230)). The electronic device (300) may include, for each authenticator, a fingerprint template database, a fingerprint generation artificial intelligence model (370), a comparator, and an ASP module (350). The electronic device (300) may perform the operations described in FIGS. 3 , 4A, 4B, 5 to 11 for each authenticator. For example, each authenticator may include a fingerprint generation artificial intelligence model (370) learned according to a fingerprint image of a corresponding user, and an ASP module (350). For example, a first authenticator (1211) may perform fingerprint authentication for a first user (1210) based on a first fingerprint template database (1212) corresponding to the first user (1210). For example, the second authenticator (1221) can perform fingerprint authentication for the second user (1220) based on the second fingerprint template database (1222) corresponding to the second user (1220). For example, the third authenticator (1231) can perform fingerprint authentication for the third user (1230) based on the third fingerprint template database (1232) corresponding to the third user (1230).

[0193] The electronic device (300) can perform a fingerprint authentication operation for each authenticator in response to a user input attempting fingerprint authentication. The electronic device (300) can identify a user corresponding to an authenticator that successfully authenticates the user. For example, a first user (1210) can successfully authenticate with a first authenticator (1211), but can fail to authenticate with a second authenticator (1221) and a third authenticator (1231). The electronic device (300) can identify a user based on the fingerprint authentication result, even without a user input specifying the user.

[0194] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment.

[0195] The electronic device (1300) may include a processor (1310), a memory (1320), and a sensor (1330). Even if some of the illustrated components are omitted or replaced with other components, various embodiments of the present document may be implemented. In addition to the illustrated components, the electronic device may further include at least some of the components and / or functions of the electronic device (100) of FIG. 1A. At least some of the components of the illustrated (or not illustrated) electronic device may be operatively, functionally, and / or electrically connected to each other.

[0196] The processor (1310) may include at least one processing circuitry, and the processor (1310) may include at least one processor. The operations described in FIGS. 3, 4A, 4B, 5 to 12 may be individually or collectively performed by at least one processor (1310) included in the processor (1310). The processor (1310) may perform operations of a comparator (e.g., comparator (340) of FIG. 3), an ASP module (e.g., ASP module (350) of FIG. 3, ASP module (920) of FIG. 9), a fingerprint authentication unit (e.g., fingerprint authentication unit (360) of FIG. 3), and / or a fingerprint generation artificial intelligence model (e.g., fingerprint generation artificial intelligence model (370) of FIG. 3, fingerprint generation artificial intelligence models (615, 616, 625, 635) of FIG. 6, fingerprint generation artificial intelligence model (720) of FIG. 7) included in FIG. 3. The processor (1310) may convert an input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) acquired by a sensor (e.g., sensor (310) of FIG. 3) into an input template (325), which is an embedding vector.

[0197] The processor (1310) may include a neural processing unit (NPU) (e.g., the NPU (113) of FIG. 1A). The NPU (e.g., the NPU (113) of FIG. 1A) may perform processing optimized for a deep-learning algorithm of artificial intelligence. According to one embodiment, the NPU (e.g., the NPU (113) of FIG. 1A) is a processor (320) optimized for deep-learning algorithm operations (e.g., artificial intelligence operations) and may process big data quickly and efficiently like a human neural network. For example, the NPU (e.g., the NPU (113) of FIG. 1A) may be mainly used for artificial intelligence operations. According to one embodiment, the NPU (e.g., the NPU (113) of FIG. 1A) may generate a virtual fingerprint image having structural characteristics similar to an input fingerprint image. In one embodiment, an NPU (e.g., NPU (113) of FIG. 1A) may perform processing to generate (e.g., regenerate or reconstruct) an image based on given information (e.g., an image and / or a prompt).

[0198] The memory (1320) can store at least one computer program, and the at least one computer program can include instructions that can be executed by the processor (1310). The operations of the processor (1310) described in FIGS. 3, 4A, 4B, and 5 to 12 can be performed according to the execution of instructions included in the memory (1320). The memory can store a fingerprint template database (330) including a normal state fingerprint template (301), an abnormal state fingerprint template (302), and a fake fingerprint template (303) as illustrated in FIG. 3. The memory can store the ASP module illustrated in FIG. 3 (e.g., the ASP module (350) of FIG. 3, the ASP module (920) of FIG. 9), and the fingerprint generation artificial intelligence model (e.g., the fingerprint generation artificial intelligence model (370) of FIG. 3, the fingerprint generation artificial intelligence models (615, 616, 625, 635) of FIG. 6, and the fingerprint generation artificial intelligence model (720) of FIG. 7).

[0199] The sensor (1330) can acquire a user's fingerprint image using various methods. For example, an optical fingerprint recognition sensor can detect a user's fingerprint by detecting light emitted from a light source. An electrostatic fingerprint recognition sensor can recognize a fingerprint by detecting the electrostatic capacitance formed by a human fingerprint using a semiconductor device sensitive to voltage and current. An ultrasonic fingerprint recognition sensor can detect a user's fingerprint by generating high-frequency sound waves and measuring the cycle of the sound waves reflecting off the fingerprint and returning.

[0200] Figure 14 is a flowchart of the operation of an electronic device according to one embodiment.

[0201] According to one embodiment, the electronic device may, in operation 1410, obtain an input fingerprint image, which is a fingerprint image input by a user, using a sensor.

[0202] The electronic device can convert an input fingerprint image (315) into an input fingerprint template (325). The template may refer to text data from which feature information of the fingerprint image is extracted. The template may be in a form that can verify the similarity between distinct fingerprint images. For example, the template may be information that converts an image into an embedding vector.

[0203] In one embodiment, the electronic device may, at operation 1420, determine whether an input fingerprint image matches some (or at least one fingerprint image) of a plurality of registered fingerprint images based on a plurality of fingerprint templates.

[0204] The electronic device can determine whether an input fingerprint image matches some of a plurality of registered fingerprint images based on a plurality of fingerprint templates, according to the operational flow diagram illustrated in FIG. 4A. The details described in FIG. 4A are omitted in FIG. 14.

[0205] According to one embodiment, the electronic device, in operation 1430, may generate at least one virtual fingerprint image based on the input fingerprint image that matches some of the plurality of registered fingerprint images, if the input fingerprint image matches some of the plurality of registered fingerprint images.

[0206] The electronic device may generate at least one virtual fingerprint image based on an input fingerprint image that matches a portion of the plurality of registered fingerprint images, if the input fingerprint image matches a portion of the plurality of registered fingerprint images, as described in FIGS. 5 and 6. The descriptions in FIGS. 5 and 6 are omitted in FIG. 14.

[0207] In one embodiment, the electronic device may store, in operation 1440, a template of an input fingerprint image and at least one virtual fingerprint image in at least one fingerprint template.

[0208] The electronic device can store an input fingerprint image and at least one virtual fingerprint image in at least one fingerprint template, according to the descriptions in FIGS. 5 and 6. The descriptions in FIGS. 5 and 6 are omitted in FIG. 14.

[0209] In one embodiment, the electronic device may train a fingerprint generation artificial intelligence model to generate a fingerprint image similar to an input fingerprint image and at least one virtual fingerprint image, in operation 1450.

[0210] The electronic device can train a fingerprint generation artificial intelligence model to generate fingerprint images similar to an input fingerprint image and at least one virtual fingerprint image, as described in FIG. 6. The description in FIG. 6 is omitted in FIG. 14.

[0211] The fingerprint generation artificial intelligence model may be an artificial intelligence model learned based on an input fingerprint image and a virtual fingerprint image, according to FIG. 7.

[0212] The electronic device can train the ASP module based on a normal state fingerprint template, an abnormal state fingerprint template, and a fake fingerprint template, according to FIGS. 8 and 9.

[0213] The electronic device can determine a fingerprint authentication result of an input fingerprint image according to FIGS. 10 to 11.

[0214] The electronic device may include multiple authenticators corresponding to multiple users according to FIG. 12.

[0215] The registered fingerprint image may be information about a fingerprint image from a time prior to the time of fingerprint authentication. A user's fingerprint may change over time. If the input template (255) is not updated, the electronic device cannot perform fingerprint authentication by reflecting the changing user fingerprint. The fingerprint authentication device (200) may experience a decrease in fingerprint authentication accuracy over time.

[0216] When a user performs fingerprint authentication, there may be an obstruction in the user's fingerprint. The input fingerprint image (215) may be acquired in an unclear manner due to the obstruction. For fingerprint authentication to be successful, the user must input a fingerprint without any obstruction. However, the success rate of fingerprint authentication may vary depending on the obstruction in the fingerprint depending on the situation in which fingerprint authentication is performed.

[0217] The ASP module (e.g., the ASP module (250) of FIG. 2) may be a model that determines whether an unspecified fingerprint image is forged. Since the ASP module (e.g., the ASP module (350) of FIG. 3) determines whether an input fingerprint image (e.g., the input fingerprint image (215) of FIG. 2) is forged without reflecting the characteristics of the user's fingerprint image, it may incorrectly determine whether some of the user's fingerprint images are forged and determine them as forged fingerprint images. The ASP module (e.g., the ASP module (250) of FIG. 2) may determine the fingerprint authentication result as a failure by determining that the fingerprint image is a forged fingerprint image even in a situation where the user inputs the fingerprint image normally.

[0218] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0219] An electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) according to one embodiment may include a sensor (e.g., the sensor (310) of FIG. 3, the sensor (1330) of FIG. 13). The electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) may include a memory (e.g., the memory (1320) of FIG. 13) that stores at least one computer program including instructions and a plurality of fingerprint templates corresponding to each of a plurality of registered fingerprint images. The electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) may include at least one processor (e.g., the processor (1310) of FIG. 13). The plurality of registered fingerprint images may include a forged fingerprint image that is a forged fingerprint of a user (e.g., a forged fingerprint image (403) of FIG. 4B), an abnormal state image that includes a hindrance element that interferes with fingerprint authentication among the user's fingerprints (e.g., an abnormal state fingerprint image (402) of FIG. 4B), and a normal state image that does not include a hindrance element among the user's fingerprints (e.g., a normal state fingerprint image (401) of FIG. 4B). The instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) to obtain an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3), which is a fingerprint image input by the user, by using the sensor (e.g., the sensor (310) of FIG. 3, the sensor (1330) of FIG. 13). The above instructions may include instructions for checking whether the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matches some of the plurality of registered fingerprint images based on the plurality of fingerprint templates.If the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matches some of the plurality of registered fingerprint images, the method may include instructions for generating at least one virtual fingerprint image (e.g., the virtual abnormal state fingerprint image (412), the virtual abnormal state fingerprint image (422), the virtual fake fingerprint image (413), the virtual fake fingerprint image (433) of FIG. 6, and the virtual fingerprint image (740) of FIG. 7) based on the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matching some of the plurality of fingerprint images. The above instructions may include instructions for storing templates of the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) and the at least one virtual fingerprint image (e.g., the virtual abnormal state fingerprint image (412) of FIG. 6, the virtual abnormal state fingerprint image (422), the virtual fake fingerprint image (413), the virtual fake fingerprint image (433), and the virtual fingerprint image (740) of FIG. 7) in the plurality of fingerprint templates. The above instructions may include instructions for training a fingerprint generation artificial intelligence model (e.g., fingerprint generation artificial intelligence model (370) of FIG. 3, fingerprint generation artificial intelligence models (615, 616, 625, 635) of FIG. 6, fingerprint generation artificial intelligence model (720) of FIG. 7) based on the input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) and at least one virtual fingerprint image (e.g., virtual abnormal state fingerprint image (412) of FIG. 6, virtual abnormal state fingerprint image (422), virtual fake fingerprint image (413) of FIG. 6, virtual fake fingerprint image (433), virtual fingerprint image (740) of FIG. 7).

[0220] An electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) can perform fingerprint authentication based on the result of comparing an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) with not only a normal fingerprint image but also an abnormal fingerprint image (e.g., an abnormal fingerprint image (402) of FIG. 4B) and / or a fake fingerprint image (e.g., a fake fingerprint image (403) of FIG. 4B). By comparing an abnormal fingerprint image with an input fingerprint, the electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) can accurately obtain a fingerprint authentication result even if the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) is a fingerprint image obtained with a fault element or a fake fingerprint image.

[0221] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) to determine whether the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) is forged based on an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images and the at least one virtual fingerprint image (e.g., the virtual abnormal state fingerprint image (412), the virtual abnormal state fingerprint image (422), the virtual fake fingerprint image (413), the virtual fake fingerprint image (433) of FIG. 6, the virtual fingerprint image (740) of FIG. 7), the ASP module (e.g., the ASP module (350) of FIG. 3), It can be made to learn ASP module (920) of 9.

[0222] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) to determine whether an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matching some of the plurality of registered fingerprint images is forged using the ASP (Anti-Spoofing-Protection) module. The instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) to determine fingerprint authentication as failed if the electronic device determines that an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of fingerprint images is a fake fingerprint image (e.g., the fake fingerprint image (403) of FIG. 4B). The above instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3)) to determine that fingerprint authentication is successful if the electronic device determines that an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images is not a forged fingerprint image.

[0223] An electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) can be trained to determine whether a fingerprint image is forged based on a user's fingerprint image. Since the electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) can generate a large number of virtual fingerprint images (e.g., a virtual abnormal fingerprint image (412) of FIG. 6, a virtual abnormal fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7)) based on an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3), the electronic device can obtain a large number of user's fingerprint images for training an ASP module (e.g., an ASP module (350) of FIG. 3, an ASP module (920) of FIG. 9). An ASP module trained based on a large number of user fingerprint images (e.g., ASP module (350) of FIG. 3, ASP module (920) of FIG. 9) can increase the accuracy of determining whether a user's fingerprint image is forged.

[0224] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3) to compare templates of the plurality of fingerprint templates (e.g., the normal state fingerprint template (301) of FIG. 3, the abnormal state fingerprint template (302) of FIG. 3, the fake fingerprint template (303) of FIG. 3)) and the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3)) to obtain a plurality of comparison values. The above instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3) to determine whether the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matches some of the plurality of registered fingerprint images based on a result of comparing the plurality of comparison values ​​and threshold values.

[0225] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3)) to classify, based on the plurality of comparison values, an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) matching some of the plurality of registered fingerprint images as one of the fake fingerprint image (e.g., the fake fingerprint image (403) of FIG. 4B), the abnormal state image (e.g., the abnormal state fingerprint image (402) of FIG. 4B), and the normal state image (e.g., the normal state fingerprint image (401) of FIG. 4B).

[0226] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3) to generate a virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7) corresponding to the classified input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3)) based on the classified input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3). The instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3) to store a template of the generated virtual fingerprint image (e.g., the virtual abnormal state fingerprint image (412) of FIG. 6, the virtual abnormal state fingerprint image (422), the virtual fake fingerprint image (413), the virtual fake fingerprint image (433) of FIG. 6, the virtual fingerprint image (740) of FIG. 7) as a corresponding template among the template of the fake fingerprint image (e.g., the fake fingerprint image (403) of FIG. 4B), the template of the abnormal state image (e.g., the abnormal state fingerprint image (402) of FIG. 4B), and the template of the normal state image (e.g., the normal state fingerprint image (401) of FIG. 4B).

[0227] An electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) can perform fingerprint authentication based on an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) as well as a registered fingerprint image by storing an input template in a fingerprint template database (e.g., a fingerprint template database (330) of FIG. 3). Since the electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) performs fingerprint authentication based on a recently input user's fingerprint image, accurate fingerprint authentication can be performed even if the user's fingerprint changes over time. In addition, an electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) can generate a virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7) in which structural features are maintained, and perform fingerprint authentication based on a template of the virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7). Because a limited number of fingerprint images can be compared with an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3), an electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) can perform accurate fingerprint authentication.

[0228] In an electronic device according to one embodiment (e.g., electronic device (100) of FIG. 1, electronic device (300) of FIG. 3), the abnormal state image (e.g., abnormal state fingerprint image (402) of FIG. 4B) may include a wet state fingerprint image and a dry state fingerprint image.

[0229] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3)) to determine that fingerprint authentication of the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) fails if the input fingerprint image does not match some of the plurality of registered fingerprint images.

[0230] In an electronic device according to one embodiment (e.g., electronic device (100) of FIG. 1A, electronic device (300) of FIG. 3), the processor (e.g., processor (1310) of FIG. 13) may include at least one NPU (e.g., NPU (113) of FIG. 1) (Neural Processing Unit). In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3), the instructions, when individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), may cause the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3)) to generate the at least one virtual fingerprint image (e.g., the virtual abnormal state fingerprint image (412) of FIG. 6, the virtual abnormal state fingerprint image (422), the virtual fake fingerprint image (413), the virtual fake fingerprint image (433) of FIG. 6, the virtual fingerprint image (740) of FIG. 7) using the at least one NPU (e.g., the NPU (113) of FIG. 1A).

[0231] In an electronic device according to one embodiment (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3), the memory (e.g., the memory (1320) of FIG. 13) may store templates corresponding to each of a plurality of users (e.g., a first user (1210), a second user (1220), a third user (1230) of FIG. 12). When the instructions are individually or collectively executed by the at least one processor (e.g., the processor (1310) of FIG. 13), the electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3)) may specify a user of an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3)) based on the templates corresponding to the plurality of users (e.g., the first user (1210), the second user (1220), the third user (1230) of FIG. 12).

[0232] According to one embodiment, an operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) may include an operation of acquiring an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3), which is a fingerprint image input by a user, by using a sensor (e.g., a sensor (310) of FIG. 3, a sensor (1330) of FIG. 13). The operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3)) may include an operation of determining whether the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) matches some of the plurality of registered fingerprint images based on a plurality of fingerprint templates (e.g., a normal state fingerprint template (301) of FIG. 3, an abnormal state fingerprint template (302) of FIG. 3, a fake fingerprint template (303) of FIG. 3) corresponding to each of a plurality of registered fingerprint images. An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) may include an operation of generating at least one virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433), a virtual fingerprint image (740) of FIG. 7) based on an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) that matches a portion of the plurality of registered fingerprint images, when the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) matches a portion of the plurality of registered fingerprint images.An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) may include an operation of storing templates of the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) and the at least one virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7) in the plurality of fingerprint templates (e.g., a normal state fingerprint template (301) of FIG. 3, an abnormal state fingerprint template (302) of FIG. 3, a fake fingerprint template (303) of FIG. 3). An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) may include an operation of training a fingerprint generation artificial intelligence model (e.g., a fingerprint generation artificial intelligence model (370) of FIG. 3, a fingerprint generation artificial intelligence model (615, 616, 625, 635) of FIG. 6, a fingerprint generation artificial intelligence model (720) of FIG. 7) based on the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) and the at least one virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7).

[0233] An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) according to one embodiment may include an operation of training an ASP module (e.g., an ASP module (350) of FIG. 3, an ASP module (920) of FIG. 9) for determining whether an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) is forged based on an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images and at least one virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412), a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7)) to determine whether the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) is forged.

[0234] A method of operating an electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) according to one embodiment may include an operation of determining whether an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images is forged using the ASP (Anti-Spoofing-Protection) module. If the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of fingerprint images is determined to be a forged fingerprint image (e.g., a forged fingerprint image (403) of FIG. 4B), the method may include an operation of determining fingerprint authentication as failed. If the input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images is determined to be a non-forged fingerprint image, the method may include an operation of determining fingerprint authentication as successful.

[0235] An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) according to one embodiment may include an operation of comparing templates of a plurality of fingerprint templates (e.g., a normal state fingerprint template (301) of FIG. 3, an abnormal state fingerprint template (302) of FIG. 3, a fake fingerprint template (303) of FIG. 3) and an input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) to obtain a plurality of comparison values. The operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) may include an operation of determining whether the input fingerprint image (e.g., an input fingerprint image (315) of FIG. 3) matches some of the plurality of registered fingerprint images based on a result of comparing the plurality of comparison values ​​and a threshold value.

[0236] An operating method of an electronic device (e.g., the electronic device (100) of FIG. 1A, the electronic device (300) of FIG. 3) according to one embodiment may include an operation of classifying an input fingerprint image (e.g., the input fingerprint image (315) of FIG. 3) that matches some of the plurality of registered fingerprint images, based on the plurality of comparison values, into one of a forged fingerprint image that is a forged fingerprint image of a user's fingerprint (e.g., the forged fingerprint image (403) of FIG. 4B), an abnormal state image that includes a hindrance element that interferes with fingerprint authentication among the user's fingerprints (e.g., the abnormal state fingerprint image (402) of FIG. 4B), and a normal state image that does not include a hindrance element among the user's fingerprints (e.g., the normal state fingerprint image (401) of FIG. 4B).

[0237] A method of operating an electronic device (e.g., electronic device (100) of FIG. 1A, electronic device (300) of FIG. 3) according to one embodiment may include an operation of generating a virtual fingerprint image (e.g., virtual abnormal state fingerprint image (412), virtual abnormal state fingerprint image (422), virtual fake fingerprint image (413), virtual fake fingerprint image (433) of FIG. 6, virtual fingerprint image (740) of FIG. 7) corresponding to the classified input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) based on the classified input fingerprint image (e.g., input fingerprint image (315) of FIG. 3). An operating method of an electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3) may include an operation of storing a template of a generated virtual fingerprint image (e.g., a virtual abnormal state fingerprint image (412) of FIG. 6, a virtual abnormal state fingerprint image (422), a virtual fake fingerprint image (413), a virtual fake fingerprint image (433) of FIG. 6, a virtual fingerprint image (740) of FIG. 7) as a corresponding template among a template of a fake fingerprint image (e.g., a fake fingerprint image (403) of FIG. 4B), a template of an abnormal state image (e.g., an abnormal state fingerprint image (402) of FIG. 4B), and a template of a normal state image (e.g., a normal state fingerprint image (401) of FIG. 4B).

[0238] In an operating method of an electronic device (e.g., an electronic device (100) of FIG. 1A, an electronic device (300) of FIG. 3) according to one embodiment, the abnormal state image (e.g., an abnormal state fingerprint image (402) of FIG. 4B) is a fingerprint image including a wet state fingerprint image and a dry state fingerprint image. In an operating method of an electronic device (e.g., an electronic device (100) of FIG. 1, an electronic device (300) of FIG. 3)).

[0239] An operating method of an electronic device (e.g., electronic device (100) of FIG. 1A, electronic device (300) of FIG. 3) according to one embodiment may include an operation of determining fingerprint authentication of an input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) as failed when the input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) does not match some of the plurality of registered fingerprint images.

[0240] A method of operating an electronic device (e.g., electronic device (100) of FIG. 1A, electronic device (300) of FIG. 3) according to one embodiment may include an operation of generating at least one virtual fingerprint image (e.g., virtual abnormal state fingerprint image (412) of FIG. 6, virtual abnormal state fingerprint image (422), virtual fake fingerprint image (413), virtual fake fingerprint image (433), virtual fingerprint image (740) of FIG. 7) using at least one NPU (e.g., NPU (113) of FIG. 1).

[0241] A method of operating an electronic device (e.g., electronic device (100) of FIG. 1, electronic device (300) of FIG. 3) according to one embodiment may include an operation of specifying a user of an input fingerprint image (e.g., input fingerprint image (315) of FIG. 3) based on templates corresponding to a plurality of users (e.g., first user (1210), second user (1220), third user (1230) of FIG. 12).

[0242] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0243] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" each include all possible combinations of the items listed together in that phrase. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0244] The term "module" as used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0245] Various embodiments of the present document may be implemented as software including one or more instructions stored in a storage medium (e.g., memory (120)) readable by a machine (e.g., electronic device (100)). For example, a processor (e.g., processor (110)) of the machine (e.g., electronic device (100)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0246] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0247] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0248] Various embodiments of the present disclosure according to the claims and description of this specification may be realized in the form of hardware, software, or a combination of hardware and software.

[0249] The software may be stored on a non-permanent computer-readable storage medium. The non-permanent computer-readable storage medium stores one or more computer programs (software modules), and the one or more computer programs include computer-executable instructions that, when individually or collectively executed by at least one processor of an electronic device, cause the electronic device to perform the disclosed method.

[0250] The software may be stored in a form of volatile or non-volatile storage, such as, for example, a storage device such as a read-only memory (ROM), or a form of memory such as a random access memory (RAM), a memory chip, a device or an integrated circuit, whether erasable or rewritable, or an optical or magnetically readable medium such as, for example, a compact disc (CD), a digital versatile disc (DVD), a magnetic disk or a magnetic tape. The storage device and the storage medium are various embodiments of non-transitory machine-readable storage suitable for storing a computer program or a computer program including instructions that, when executed, implement various embodiments of the present disclosure. Accordingly, various embodiments provide a program comprising code for implementing an apparatus or method as claimed in any of the claims of this specification, and a non-transitory machine-readable storage storing such a program.

[0251] While the present disclosure has been illustrated and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. In electronic devices, sensor; At least one computer program including instructions and a memory storing a plurality of fingerprint templates corresponding to each of a plurality of registered fingerprint images; and At least one processor in communication with the sensor and the memory, The plurality of registered fingerprint images include a forged fingerprint image that is a forged fingerprint of the user, an abnormal fingerprint image that includes an obstacle that interferes with fingerprint authentication among the user's fingerprints, and a normal fingerprint image that is a fingerprint that has been successfully registered or matches the fingerprint that has been successfully registered. The above instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: An input fingerprint image, which is a fingerprint image input by a user, is acquired using the above sensor, Based on the plurality of fingerprint templates, it is determined whether the input fingerprint image matches some of the plurality of registered fingerprint images, If the input fingerprint image matches some of the plurality of registered fingerprint images, at least one virtual fingerprint image is generated based on the input fingerprint image that matches some of the plurality of fingerprint images, storing the templates of the input fingerprint image and the at least one virtual fingerprint image in the plurality of fingerprint templates, and An electronic device that trains a fingerprint generation artificial intelligence model based on the input fingerprint image and at least one virtual fingerprint image.

2. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, An electronic device that trains an anti-spoofing-protection (ASP) module to determine whether the input fingerprint image is forged based on an input fingerprint image that matches some of the plurality of registered fingerprint images and at least one virtual fingerprint image.

3. In the second paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, Using the above ASP (anti-spoofing-protection) module, determine whether an input fingerprint image that matches some of the multiple registered fingerprint images is forged, If an input fingerprint image that matches some of the above multiple fingerprint images is determined to be a fake fingerprint image, fingerprint authentication is determined to have failed, and An electronic device that determines fingerprint authentication as successful if it determines that an input fingerprint image that matches some of the plurality of registered fingerprint images is not a forged fingerprint image.

4. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, Comparing the plurality of fingerprint templates and the template of the input fingerprint image to obtain a plurality of comparison values, and An electronic device that determines whether the input fingerprint image matches some of the plurality of registered fingerprint images based on the results of comparing the plurality of comparison values ​​and threshold values.

5. In paragraph 4, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that classifies an input fingerprint image that matches some of the plurality of registered fingerprint images as one of the forged fingerprint image, the abnormal fingerprint image, or the normal fingerprint image based on the plurality of comparison values.

6. In the fifth paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, Generating a virtual fingerprint image corresponding to the classified input fingerprint image based on the classified input fingerprint image, and An electronic device that stores the template of the virtual fingerprint image generated above as a corresponding template among the template of the fake fingerprint image, the template of the abnormal fingerprint image, and the template of the normal fingerprint image.

7. In the 6th paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, If the input fingerprint image has a recognizable structural feature, generate at least one virtual fingerprint image corresponding to the input fingerprint image, An electronic device, wherein the template of the input fingerprint image and the template of the at least one virtual fingerprint image are stored in the plurality of fingerprint templates.

8. In the 7th paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, Receive another input fingerprint image for authentication, An electronic device, in response to receiving another input fingerprint image for the above authentication, to verify whether the other input fingerprint image matches a portion of the plurality of registered fingerprint images including the input fingerprint image and the at least one virtual fingerprint image.

9. In the fifth paragraph, the abnormal state fingerprint image is, An electronic device that contains a fingerprint image containing a wet fingerprint image and a dry fingerprint image.

10. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the electronic device, An electronic device that determines that fingerprint authentication of the input fingerprint image has failed if the input fingerprint image does not match some of the plurality of registered fingerprint images.

11. In the first paragraph, the processor, Contains at least one NPU (Neural Processing Unit), The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that generates at least one virtual fingerprint image using at least one NPU.

12. In the first paragraph, the memory, Save templates corresponding to each of multiple users, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to An electronic device for identifying a user of an input fingerprint image based on templates corresponding to the plurality of users.

13. In the method of operating an electronic device, An action of acquiring an input fingerprint image, which is a fingerprint image entered by a user, using a sensor. An operation of determining whether the input fingerprint image matches some of the plurality of registered fingerprint images based on a plurality of fingerprint templates corresponding to each of the plurality of registered fingerprint images; An operation of generating at least one virtual fingerprint image based on an input fingerprint image that matches some of the plurality of registered fingerprint images, when the input fingerprint image matches some of the plurality of registered fingerprint images; An operation of storing a template of the input fingerprint image and the at least one virtual fingerprint image in the plurality of fingerprint templates; and An operating method of an electronic device, comprising an operation of training a fingerprint generation artificial intelligence model based on the input fingerprint image and the at least one virtual fingerprint image.

14. In paragraph 13, An operating method of an electronic device, comprising an operation of training an anti-spoofing-protection (ASP) module to determine whether the input fingerprint image is forged based on an input fingerprint image that matches some of the plurality of registered fingerprint images and the at least one virtual fingerprint image.

15. In paragraph 14, An operation of determining whether an input fingerprint image matching some of the plurality of registered fingerprint images is forged using the above ASP (Anti-Spoofing-Protection) module; An operation for determining fingerprint authentication as a failure if an input fingerprint image matching some of the above multiple fingerprint images is determined to be a fake fingerprint image; and An operating method of an electronic device, comprising an action of determining fingerprint authentication as successful if an input fingerprint image matching some of the plurality of registered fingerprint images is determined to be a non-forged fingerprint image.

Images