Anti-spoofing method and device

Abstract

Disclosed are a fraud prevention method and apparatus. The fraud prevention method includes detecting first information about whether biometric information is fake based on a first output vector of a first neural network configured to detect whether biometric information is fake from input data; extracting an input embedding vector including a feature of biometric information of a user from input data including the biometric information; calculating a similarity value of the input embedding vector based on either or both of a real embedding vector and a registered embedding vector provided in advance and a fake embedding vector; calculating a total fake score based on the similarity value and a second output vector of the first neural network according to whether the first information is detected; and detecting second information about whether the biometric information is fake based on the total fake score.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2021-0042081, filed on March 31, 2021, with the Korean Intellectual Property Office, and Korean Patent Application No. 10-2021-0077373, filed on June 15, 2021, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to methods and devices for preventing spoofing, and more specifically, to methods for detecting whether biometric information has been forged. Background Technology

[0003] The importance of technologies for secure authentication of biometric information is increasing due to the development of smartphones and various mobile and wearable devices. For example, fingerprint recognition technology is widely used in this field due to its high level of convenience, security, and cost-effectiveness. In typical fingerprint recognition, user authentication or verification is performed by acquiring a user's fingerprint image via a sensor and comparing it with pre-registered fingerprint images. When a sophisticatedly crafted fake fingerprint pattern is input into the sensor, the fingerprint recognition device may be unable to distinguish it from a genuine fingerprint pattern. Therefore, the fingerprint recognition device may incorrectly identify a fake fingerprint pattern as a biometric fingerprint. For example, when a material with a fingerprint pattern etched onto it (such as rubber, silicone, gelatin, epoxy resin, and latex) comes into contact with the sensor, the fingerprint etched onto such a material can be identified as a human fingerprint. Distinguishing between spoofed biometric information and live biometric information is crucial for the security of biometric information. Summary of the Invention

[0004] This summary provides a brief overview of the options for the concepts further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, an anti-spoofing method includes: detecting first information about whether a user's biometric information has been forged based on a first output vector of a first neural network, the first neural network being configured to detect whether the biometric information has been forged from input data including the biometric information; extracting an input embedding vector of features including the biometric information from the input data; calculating a similarity value of the input embedding vector based on any one or both of a real embedding vector and a registered embedding vector, and a fake embedding vector, the real embedding vector and the registered embedding vector being provided in advance; calculating a total forgery score based on the similarity value and a second output vector of the first neural network, depending on whether the first information has been detected; and detecting second information about whether the biometric information has been forged based on the total forgery score.

[0006] The steps for detecting the first information may include: extracting a first output vector from an intermediate layer of a first neural network; and calculating an intermediate forgery score based on the first output vector.

[0007] The steps for detecting the first information may include: determining whether the intermediate forged score is within a threshold range for detecting the first information; and detecting the first information based on the intermediate forged score in response to determining that the intermediate forged score is within the threshold range.

[0008] The steps for calculating the total forgery score may include: in response to the absence of detection of the first information, calculating the total forgery score based on the second output vector and the similarity value.

[0009] The steps of calculating the total forgery score may include: in response to the absence of detection of the first information, calculating the final forgery score based on the second output vector output from the output layer of the first neural network; and calculating the total forgery score based on the final forgery score and the similarity value.

[0010] The steps for calculating the total forgery score may include: performing a weighted summation of the final forgery score and the similarity value; and calculating the total forgery score by applying a nonlinear function to the result of the weighted summation.

[0011] The steps for detecting the second information may include: detecting the second information by comparing the total forgery score with a first threshold used to detect the second information.

[0012] The method may further include: determining whether the embedding vector update condition of the input embedding vector is satisfied based on the total spoofing score; and using the input embedding vector to update any one or any combination of the registered embedding vector, the real embedding vector, and the spoof embedding vector based on the determination that the embedding vector update condition is satisfied.

[0013] The method may further include: generating a virtual registration embedding vector that has environmental characteristics different from those of the registration embedding vector but has the structural characteristics of the registration embedding vector. The step of calculating the similarity value may include: calculating the similarity value based on any one or any combination of the registration embedding vector, the real embedding vector, and the virtual registration embedding vector, as well as the spurious embedding vector.

[0014] Similarity values ​​can be calculated based on the posterior probability between the input embedding vector and any one or any combination of the true embedding vector, registered embedding vector, and spurious embedding vector. Posterior probabilities can also be calculated based on the distance or similarity between the input embedding vector and any one or any combination of the true embedding vector, registered embedding vector, and spurious embedding vector.

[0015] The steps for calculating similarity values ​​may include: determining a value for a first category corresponding to cases where biometric information has not been forged, based on a first distance between the real embedding vector and the input embedding vector and a third distance between the registered embedding vector and the input embedding vector; determining a value for a second category corresponding to cases where biometric information has been forged, based on a second distance between the fake embedding vector and the input embedding vector; and determining a confidence value for the input embedding vector based on the values ​​of the first category and the second category.

[0016] The step of extracting the input embedding vector may include: using a first neural network to extract the input embedding vector from biometric information. The first neural network may be trained based on live biometric information and deceptive biometric information.

[0017] In another general aspect, an anti-spoofing device includes: a processor configured to: detect first information regarding whether a user's biometric information has been forged based on a first output vector of a first neural network, the first neural network being configured to detect whether the biometric information has been forged from input data including the biometric information; extract an input embedding vector of features including the biometric information from the input data; calculate a similarity value of the input embedding vector based on either or both of a real embedding vector and a registered embedding vector, and a fake embedding vector, the real embedding vector and the registered embedding vector being provided in advance; calculate a total forgery score based on the similarity value and a second output vector of the first neural network, depending on whether the first information has been detected; and detect second information regarding whether the biometric information has been forged based on the total forgery score.

[0018] The processor can be configured to: extract a first output vector from an intermediate layer of a first neural network; and calculate an intermediate forgery score based on the first output vector.

[0019] The processor can be configured to: determine whether an intermediate forged score is within a threshold range for detecting first information; and, in response to determining that the intermediate forged score is within the threshold range, detect first information based on the intermediate forged score.

[0020] The processor can be configured to calculate a total forgery score based on a second output vector and a similarity value in response to the absence of first information.

[0021] The processor can be configured to: calculate a final forgery score based on a second output vector output from the output layer of a first neural network in response to the absence of first information; and calculate a total forgery score based on the final forgery score and a similarity value.

[0022] The processor can be configured to: perform a weighted summation of the final forgery score and similarity value; and calculate the total forgery score by applying a nonlinear function to the result of the weighted summation.

[0023] The processor can be configured to detect the second information by comparing the total forgery score with a first threshold used to detect the second information.

[0024] The processor can be configured to: determine whether the embedding vector update condition of the input embedding vector is satisfied based on the total spoofing score; and, based on the determination that the embedding vector update condition is satisfied, use the input embedding vector to update any one or any combination of the registered embedding vector, the real embedding vector, and the spoof embedding vector.

[0025] The processor can be configured to: generate a virtual registration embedding vector having environmental characteristics different from those of the registration embedding vector and having structural characteristics of the registration embedding vector; and calculate a similarity value based on any one or any combination of the registration embedding vector, the real embedding vector, and the virtual registration embedding vector, as well as the spurious embedding vector.

[0026] Similarity values ​​can be calculated based on the posterior probability between the input embedding vector and any one or any combination of the true embedding vector, registered embedding vector, and spurious embedding vector. Posterior probabilities can also be calculated based on the distance or similarity between the input embedding vector and any one or any combination of the true embedding vector, registered embedding vector, and spurious embedding vector.

[0027] The processor can be configured to: determine a value for a first category corresponding to the case where the biometric information has not been forged, based on a first distance between the real embedding vector and the input embedding vector and a third distance between the registered embedding vector and the input embedding vector; determine a value for a second category corresponding to the case where the biometric information has been forged, based on a second distance between the fake embedding vector and the input embedding vector; and determine a confidence value for the input embedding vector based on the values ​​of the first category and the second category.

[0028] The processor can be configured to extract input embedding vectors from biometric information using a first neural network. The first neural network can be trained based on live biometric information and deceptive biometric information.

[0029] In another general aspect, an apparatus includes: one or more processors configured to: extract an input embedding vector comprising features including biometric information included in input data; compute a similarity value of the input embedding vector based on one or both of a real embedding vector and a registered embedding vector and a spoof embedding vector; compute a total forgery score based on the similarity value and an output vector of a neural network trained using live biometric information and spoof biometric information; and determine whether the biometric information has been forged based on the total forgery score.

[0030] The one or more processors may be configured to: determine that the biometric information is live information when the total forgery score is greater than a first threshold; and determine that the biometric information is deceptive information when the total forgery score is less than or equal to the first threshold.

[0031] The one or more processors may be configured to determine a first threshold based on a rejection threshold and an acceptance threshold, wherein the rejection threshold corresponds to the maximum probability that the intermediate forgery score based on the output vector falls within the range where the biometric information is determined to be deceptive information, and the acceptance threshold corresponds to the minimum probability that the intermediate forgery score falls within the range where the biometric information is determined to be live information.

[0032] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0033] Figure 1 An example of a biometric information identification method is shown.

[0034] Figure 2 Examples of anti-spoofing methods are shown.

[0035] Figure 3A An example of the network structure and operation of a device used to detect whether biometric information has been forged is shown.

[0036] Figure 3B An example of a method for detecting first information about whether biometric information has been forged is shown.

[0037] Figure 3C An example of the first threshold range is shown.

[0038] Figure 4A , Figure 4B and Figure 4CAn example is shown where the similarity value of the input embedding vector is determined based on the estimated posterior probability.

[0039] Figure 5A and Figure 5B An example is shown of updating the initial model and the registration model based on the input embedding vector.

[0040] Figure 6A and Figure 6B This shows an example of registering biometric information using anti-spoofing methods.

[0041] Figure 7A , Figure 7B , Figure 7C and Figure 7D An example of generating a virtual embedding vector is shown.

[0042] Figure 8A An example of the relationship between the embedding vector derived from the embedding vectors of the attempt to register and the input embedding vector of the attempt to authenticate is shown in three dimensions.

[0043] Figure 8B This shows an example of a method for classifying input data.

[0044] Figure 9 An example of an anti-spoofing device is shown.

[0045] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated. Detailed Implementation

[0046] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order set forth herein, but may be changed as will become clear after understanding the disclosure of this application, except for operations that must occur in a specific order.

[0047] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be clear upon understanding the disclosure of this application.

[0048] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the features, quantities, operations, components, elements, and / or combinations thereof stated, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0049] Throughout the specification, when a component is described as "connected to" or "bonded to" another component, it may be directly "connected to" or directly "bonded to" another component, or there may be one or more other components in between. Conversely, when an element is described as "directly connected to" or "directly bonded to" another element, there may be no other elements in between.

[0050] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0051] Unless otherwise defined, all terms used herein (including technical or scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on the understanding of the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having the meaning consistent with their meaning in the context of the relevant field and in the disclosure of this application, and shall not be interpreted in an idealized or overly formalized sense.

[0052] Furthermore, in the description of the exemplary embodiments, descriptions will be omitted where a detailed description of a structure or function known therefrom after understanding the disclosure of this application would lead to a vague interpretation of the exemplary embodiments. Hereinafter, the examples will be described in detail with reference to the accompanying drawings, and the same reference numerals in the drawings always denote the same elements.

[0053] Examples can be implemented in various types of products (such as personal computers (PCs), laptops, tablets, smartphones, televisions (TVs), smart home appliances, smart vehicles, self-service terminals, and wearable devices). The examples will be described in detail below with reference to the accompanying drawings. In the drawings, the same reference numerals are used for the same elements.

[0054] Figure 1 An example of a biometric identification method is shown. (For ease of description) Figure 1 This assumes that the biometric information is a fingerprint. However, the example described here can be equally applied to various biometric information that can be identified in the form of an image (e.g., veins or irises).

[0055] Reference Figure 1 The fingerprint recognition device 100 may include a fingerprint sensor 110 configured to sense a user's fingerprint. The fingerprint recognition device 100 may acquire an input fingerprint image 115 representing the user's fingerprint through the fingerprint sensor 110.

[0056] In one example, fingerprint registration for fingerprint recognition can be performed. Registered fingerprint images 121, 122, and 123 can be pre-stored in a registered fingerprint database (DB) 120 via fingerprint registration processing. Optionally or additionally, for personal information protection, the registered fingerprint DB 120 may store features extracted from the registered fingerprint images 121, 122, and 123, rather than storing the registered fingerprint images 121, 122, and 123 unchanged. The registered fingerprint DB 120 may be stored in a memory (not shown) included in the fingerprint recognition device 100, or in an external device (such as a server (not shown) capable of communicating with the fingerprint recognition device 100).

[0057] For example, when fingerprint recognition device 100 receives an input fingerprint image 115 for authentication, fingerprint recognition device 100 can identify a user's fingerprint by comparing the fingerprint shown in input fingerprint image 115 (hereinafter referred to as the "input fingerprint") with the registered fingerprints shown in registered fingerprint images 121, 122, and 123. Fingerprint recognition device 100 can compare the features of the input fingerprint with the features of the registered fingerprints. Authentication of a fake fingerprint may succeed when a fake fingerprint is sensed in input fingerprint image 115 and when input fingerprint image 115 has a similar fingerprint pattern to one of the registered fingerprint images 121, 122, and 123. To prevent such spoofing attacks, processing may be needed to determine whether the input fingerprint in input fingerprint image 115 is a fake or genuine fingerprint.

[0058] If the identification system outputs a result indicating a mismatch in user information, the authentication process can be terminated immediately using the mismatch message. Conversely, if the system outputs a result indicating a match in user information, anti-spoofing procedures can be performed to determine if the corresponding biometric information has been forged or spoofed. The term "spoofing" as used herein can be interpreted to include, for example, the copying, forgery, and counterfeiting of biometric information. For final authentication to pass, the biometric information needs to be verified as living in the process of detecting whether it has been forged.

[0059] In one example, the fingerprint recognition device 100 may include an anti-spoofing device (not shown) to detect whether an input fingerprint is a fake fingerprint. Hereinafter, the anti-spoofing device may be referred to as a "fake fingerprint detection device".

[0060] Typical spoofing fingerprint detection systems use neural networks trained on a pre-provided database to distinguish between real and spoofed fingerprints. However, the performance of typical spoofing fingerprint detection systems can degrade under predetermined conditions where the neural network does not learn. For example, since typical spoofing fingerprint detection systems only use the dichotomous results of a general-trained network, regardless of each user's characteristics, their performance can be reduced due to differences in user biometrics (e.g., cracked fingerprints or relatively pale fingerprints). Furthermore, since the biometric information used in training the network is not used for authentication, performance can degrade in situations where the network's sensing capabilities are difficult to train. Additionally, typical spoofing detection systems may not respond quickly to errors based on the usage environment or due to time-varying factors (such as aging), and may not support features such as rapid decision-making in intermediate processing. Moreover, when new vulnerabilities related to spoofing fingerprint detection are discovered, the neural network must be retrained, making emergency response difficult.

[0061] A fake fingerprint detection device can determine whether an input fingerprint is fake based on a plurality of pre-provided unspecified real fingerprint features, a plurality of pre-provided unspecified fake fingerprint features, and / or the user's registered fingerprint features. Non-limiting examples of this will be further described below. Furthermore, the fake fingerprint detection device can first determine whether forgery has occurred or not, and then determine whether the remaining fingerprints have been forged. Additionally, the fake fingerprint detection device can use the input fingerprint to update the registered fingerprint database. Therefore, the anti-spoofing performance of the fake fingerprint detection device can be enhanced.

[0062] Figure 2 Examples of anti-spoofing methods are shown.

[0063] Reference Figure 2 Operations 210 to 250 can be referenced above. Figure 1 The described anti-spoofing device is implemented. The anti-spoofing device can be implemented by hardware modules, software modules, or various combinations thereof. Furthermore, although it can be implemented in the order and manner shown... Figure 2 However, without departing from the spirit and scope of the example, the order of some operations may be changed or some operations may be omitted, and they may be executed in parallel or simultaneously. Figure 2 The diagram shows several operations.

[0064] In operation 210, the anti-spoofing device can detect first information about whether biometric information has been forged based on a first output vector of a first neural network, the first neural network being configured to detect whether biometric information has been forged from input data including biometric information.

[0065] Anti-spoofing devices can receive input data (e.g., images) including biometric information sensed from the user by various sensors.

[0066] The sensor may include, for example, an ultrasonic fingerprint sensor, an optical fingerprint sensor, an electrostatic fingerprint sensor, a depth sensor, an iris sensor, or an image sensor, but is not limited thereto. One of the sensors may be used, or at least two of the sensors may be used. The biometric information sensed by the sensor may be, for example... Figure 1 Input fingerprint image 115, iris image or facial image.

[0067] A single image or multiple images may include a user's biometric information. If one image includes a user's biometric information, a single neural network can be used; if multiple images include a user's biometric information, a number of neural networks corresponding to the number of images can be used. For example, an anti-spoofing device can acquire 32-bit fingerprint images and 8-bit fingerprint images.

[0068] The anti-spoofing device can perform a first spoofing detection (i.e., it can first determine whether the presence or absence of biometric information that can be relatively clearly identified has been forged). The anti-spoofing device can perform a second spoofing detection (i.e., it can re-determine whether the presence or absence of biometric information that was not identified in the first spoofing detection has been forged).

[0069] The anti-spoofing device can extract a first output vector from an intermediate layer of a first neural network and can calculate an intermediate forgery score based on the first output vector. The term "forgery score" as used herein can mean a score that has been forged or altered. The anti-spoofing device can determine whether the intermediate forgery score is within a threshold range used to detect first information. When the intermediate forgery score is determined to be within the threshold range, the first information can be detected based on the intermediate forgery score. The following will refer to... Figures 3A to 3C A further example of detecting the first piece of information is described.

[0070] Figure 3A This diagram illustrates an example of the network structure and operation of a device used to detect whether biometric information has been forged. Figure 3B An example of a method for detecting first information about whether biometric information has been forged is shown.

[0071] Figure 3A and Figure 3BThe structure and operation of an anti-spoofing device 300, which may include a (1-1) classifier 340, a (1-2) classifier 350, and a second classifier 360, are shown. The (1-1) classifier 340 and the (1-2) classifier 350 may each include a shallow deep neural network (DNN).

[0072] Unlike classifiers in a typical DNN with an end-to-end structure, classifiers 340 (1-1) and 350 (1-2) can receive input data (e.g., images) 310 that includes biometric information, and can classify whether the biometric information has been forged based on a first output vector extracted from the intermediate layer of the DNN 330 during network inference.

[0073] The first output vector extracted from the intermediate layers of DNN 330 can be applied to classifiers 340 and 350 (1-1) and can be used to detect first information regarding whether biometric information has been forged. Classifiers 340 and 350 (1-1) can be classifiers trained to classify input images based on the output vector. For example, classifiers 340 and 350 (1-1) can include shallow DNNs requiring less computation than DNNs and can quickly detect first information without being slowed down by the relatively small overhead of early decisions (ED) in the intermediate layers.

[0074] The following describes an example of how the (1-1) classifier 340 and the (1-2) classifier 350 detect first information. The anti-spoofing device 300 can use the (1-1) classifier 340 to extract a first output vector (e.g., a (1-1) embedding vector) from the intermediate layers of the DNN 330. In operation 301, the anti-spoofing device 300 can compute a (1-1) intermediate forgery score based on the extracted (1-1) embedding vector. The anti-spoofing device 300 can detect first information based on the (1-1) intermediate forgery score. In operation 302, the anti-spoofing device 300 can use the trained (1-1) classifier 340 to determine whether the (1-1) intermediate forgery score is within a first threshold range used to determine the first information. The anti-spoofing device 300 can classify whether the (1-1) intermediate forgery score is within the range where biometric information is determined to be deceptive information or within the range where biometric information is determined to be live information.

[0075] For example, in operation 303, when the (1-1)th intermediate spoofing score is determined to be a score within a first threshold range, the anti-spoofing device 300 can detect the first information based on the (1-1)th intermediate spoofing score.

[0076] In operation 304, when the first information is not detected by the (1-1) intermediate spoofing score, the anti-spoofing device 300 may use the (1-2) classifier 350 to extract the (1-2) embedding vector from the intermediate layer after the intermediate layer of the DNN 330 (i.e., the intermediate layer from which the first output vector is extracted).

[0077] In operation 305, the anti-spoofing device 300 may calculate the (1-2) intermediate forgery score based on the (1-2) embedding vector.

[0078] In operation 306, the anti-spoofing device 300 may detect the first information based on the (1-2)th intermediate forgery score calculated in operation 305. In operation 305, the anti-spoofing device 300 may use a trained (1-2)th classifier 350 to determine whether the (1-2)th intermediate forgery score is within a first threshold range used to determine the first information. The anti-spoofing device 300 may classify the (1-2)th intermediate forgery score as either a score within the range where biometric information is determined to be deceptive information or a score within the range where biometric information is determined to be liveness information. For example, when the (1-2)th intermediate forgery score is determined to be within the first threshold range, the anti-spoofing device 300 may detect the first information based on the (1-2)th intermediate forgery score.

[0079] In one example, when the first information is detected by the (1-2) intermediate spoofing score, the anti-spoofing device 300 can output the classification result of the (1-2) classifier 350. In another example, when the first information is not detected by the (1-2) intermediate spoofing score, the anti-spoofing device 300 can detect the second information about whether the biometric information has been spoofed by applying the second output vector from the output layer of the DNN 330 to the second classifier 360.

[0080] When the first information is detected by the (1-1) classifier 340 and (1-2) classifier 350 performing early decision classification before obtaining the second output vector from the output layer, the anti-spoofing device 300 can immediately detect whether the biometric information has been forged, even without using the second output vector. Therefore, the amount of time required to determine whether the biometric information has been forged can be reduced.

[0081] When determining whether biometric information has been forged, a trade-off exists between the accuracy of the determination and the speed of detecting forgery. The anti-spoofing device 300 can perform a relatively rapid determination using a first (1-1) classifier 340 and a second (1-2) classifier 350. When the first (1-1) classifier 340 and the second (1-2) classifier 350 have a relatively high detection confidence level, the first information can be used immediately. When the first (1-1) classifier 340 and the second (1-2) classifier 350 have a relatively low detection confidence level, the second information can be determined from the second output vector by the second classifier 360.

[0082] Figure 3C An example of the first threshold range is shown.

[0083] Reference Figure 3C A graph 307 showing the probability distribution of the first forged score corresponding to the deceptive information and a graph 308 showing the probability distribution of the first forged score corresponding to the liveness information are shown.

[0084] In one example, if it can be clearly determined whether the first intermediate forgery score calculated based on the first output vector falls within the range where the biometric information is determined to be live or within the range where the biometric information is determined to be deceptive, then the anti-spoofing device 300 can immediately detect whether the biometric information has been forged based on the first intermediate forgery score. In another example, if it cannot be clearly determined whether the first intermediate forgery score falls within the range where the biometric information is determined to be live or within the range where the biometric information is determined to be deceptive, then the anti-spoofing device 300 cannot immediately determine whether the biometric information has been forged based on the first intermediate forgery score.

[0085] In one example, the first threshold range may correspond to a probability range that allows for a clear determination of whether biometric information has been forged based on a first intermediate forgery score. The first threshold range can be used to identify whether the presence or absence of forgery cannot be clearly determined by a first intermediate forgery score. If forgery is not immediately determined by the first intermediate forgery score, the anti-spoofing device 300 may determine whether forgery has occurred based on a second intermediate forgery score.

[0086] In graph 307, the rejection threshold 375 corresponds to the maximum probability (maximum (deception score)) of the first intermediate forgery score being determined to fall within the range where biometric information is clearly identified as deception information. Furthermore, in graph 308, the acceptance threshold 385 corresponds to the minimum probability (minimum (liveness score)) of the first intermediate forgery score being determined to fall within the range where biometric information is clearly identified as liveness information.

[0087] The first threshold range can be determined based on a rejection threshold 375 and an acceptance threshold 385. The rejection threshold 375 corresponds to the maximum probability (maximum (deception score)) that the first intermediate forgery score is determined to fall within the range where biometric information is determined to be deceptive information, and the acceptance threshold 385 corresponds to the minimum probability (minimum (liveness score)) that the first intermediate forgery score is determined to fall within the range where biometric information is determined to be live information. The first threshold range can correspond to a range outside of interval 380, which is greater than the rejection threshold 375 in curve 307 and less than the acceptance threshold 385 in curve 308. If the first intermediate forgery score is in interval 370 of curve 307 that is less than or equal to the rejection threshold 375 (e.g., corresponding to deception rejection SR), then the first intermediate forgery score can be determined to fall within the range where biometric information is clearly determined to be "deceptive information". Furthermore, if the first intermediate forgery score is in the interval 390 of the curve 308 that is greater than or equal to the acceptance threshold 385 (e.g., corresponding to live acceptance LA), then the first intermediate forgery score can be determined to fall within the range where biometric information is clearly identified as “live information”.

[0088] In one example, when the first intermediate spoofing score falls within the intervals 370 and 390, the anti-spoofing device 300 can determine that the first intermediate spoofing score falls within a first threshold range, and can immediately detect first information regarding whether the biometric information has been spoofed based on the first intermediate spoofing score. In another example, when the first intermediate spoofing score falls within the interval 380, the anti-spoofing device 300 can determine that the first intermediate spoofing score does not fall within the first threshold range, and can immediately detect second information regarding whether the biometric information has been spoofed based on a second output vector and a similarity value, which will be described below.

[0089] Return to reference Figure 2 In operation 220, the anti-spoofing device can extract an input embedding vector containing features including biometric information from the input data.

[0090] When receiving biometric information sensed by the user, the anti-spoofing device can extract an input embedding vector from the input data that includes the biometric information.

[0091] Anti-spoofing devices can use a neural network trained to extract features from input biometric information to extract an input embedding vector. The neural network can be trained based on multiple unspecified live biometric data and multiple unspecified spoofed biometric data. The embedding vector generated by the neural network can include feature information from the biometric data. This embedding vector can be referred to as a "feature vector."

[0092] In this example, the neural network may include a first neural network. (See reference...) Figure 3AThe anti-spoofing device 300 can extract the input embedding vector based on the (1-1)th output embedding vector.

[0093] In operation 230, the anti-spoofing device can calculate the similarity value of the input embedding vector based on any one or two of the pre-provided real embedding vector and registered embedding vector, as well as the spoofing embedding vector.

[0094] The anti-spoofing device can obtain the true embedding vectors of a pre-provided true model based on live biometric information and / or the registration embedding vectors of a registration model generated based on the biometric information of a pre-registered user. The true model may correspond to a set of true embedding vectors extracted from multiple unspecified live biometric information entries. These unspecified live biometric information entries may be the same as the live biometric information used to train the aforementioned neural network. The true model may include all of the multiple unspecified true embedding vectors, or it may include only the embedding vectors represented by clustering.

[0095] The registration model can be based on the biometric information of pre-registered users (e.g., Figure 1 A set of registration embedding vectors extracted from the registered fingerprint images 121, 122, and 123. The anti-spoofing device can store the registration model during fingerprint registration processing. Figure 1 The registered fingerprint is in DB 120.

[0096] Anti-spoofing devices can obtain spoofing embedding vectors of a pre-provided spoofing model based on spoofing biometric information. The spoofing model may correspond to a set of spoofing embedding vectors extracted from multiple unspecified spoofing biometric information. These unspecified spoofing biometric information may be the same as the spoofing biometric information used to train the aforementioned neural network. The spoofing model may include all of the multiple unspecified spoofing embedding vectors, or it may include only the embedding vectors represented by clustering.

[0097] Anti-spoofing devices can use real embedding vectors and / or registered embedding vectors, as well as spoofing embedding vectors, to determine the similarity value of input embedding vectors. The similarity value of input embedding vectors can be an indicator of how closely the input fingerprint image resembles a real fingerprint. As the similarity value increases, the probability that the input fingerprint image is a real fingerprint increases. The following will refer to... Figures 4A to 4C Further describe an example of determining similarity values.

[0098] In operation 240, the anti-spoofing device can calculate a total forgery score based on a similarity value and a second output vector of a first neural network, depending on whether the first information is detected. As described above, when the first information is not detected, the anti-spoofing device can calculate a total forgery score based on the second output vector and a similarity value.

[0099] In one example, when no first information is detected, the anti-spoofing device can calculate a final forgery score based on a second output vector from the output layer of a first neural network, and can calculate a total forgery score based on the final forgery score and a similarity value. For example, the anti-spoofing device can perform a weighted summation of the final forgery score and the similarity value, and can calculate the total forgery score by applying a predetermined nonlinear function to the result of the weighted summation.

[0100] In operation 250, the anti-spoofing device can detect second information regarding whether the biometric information has been forged based on the total forgery score. When the total forgery score is calculated, the anti-spoofing device can compare the total forgery score with a first threshold used to detect the second information, and can detect the second information regarding whether the biometric information has been forged.

[0101] For example, if the total forgery score is greater than a first threshold used to detect the second information, the anti-spoofing device can identify the corresponding biometric information as liveness information. If the total forgery score is less than or equal to the first threshold, the anti-spoofing device can identify the corresponding biometric information as spoofing information.

[0102] Although fake fingerprint detectors in typical fake fingerprint detection systems are designed based on a pre-provided training database, it would be impractical in terms of time and cost to use the training database to cover all real-world usage scenarios.

[0103] For example, situations where it's difficult to utilize the training database to cover the real-world usage environment can arise from changes in the terminal itself (hereinafter referred to as "terminal change situations"), such as a protective film being attached to a spoofing fingerprint detector or scratches on the spoofing fingerprint detector. Another situation where it's difficult to utilize the training database to cover the real-world usage environment can arise from fingerprints shrinking or deforming depending on the usage environment (e.g., low temperature, high temperature, or dry environment) (hereinafter referred to as "fingerprint variation situations"). In these situations, it may be difficult to utilize the training database to cover the real-world usage environment, thus reducing the performance of the spoofing fingerprint detector.

[0104] Therefore, anti-spoofing devices can determine whether the embedding vector update condition of the input embedding vector is met based on the total forgery score, and can use the input embedding vector to update any one or any combination of the registered embedding vector, the real embedding vector, and the spoofing embedding vector based on the determination that the embedding vector update condition is met. Anti-spoofing devices can reflect the actual usage environment by updating the anti-spoofing device based on input data including biometric information, thereby enhancing the performance of determining whether a fingerprint has been forged. The following will refer to... Figure 5A and Figure 5B Further describe an example of updating any or any combination of the registration embedding vector, the true embedding vector, and the spurious embedding vector using the embedding vector.

[0105] Figures 4A to 4C An example is shown where a similarity value for an input embedding vector is determined based on an estimated posterior probability. For instance, an anti-spoofing device may compute a similarity value based on the posterior probability between the input embedding vector and any one or any combination of the real embedding vector, the registered embedding vector, and the spoofing embedding vector, wherein the posterior probability may be computed based on the distance or similarity between the input embedding vector and any one or any combination of the real embedding vector, the registered embedding vector, and the spoofing embedding vector.

[0106] Reference Figure 4A The anti-spoofing device can determine the similarity value of the input embedding vector based on the real embedding vector of the real model 420 and the fake embedding vector of the fake model 430. The anti-spoofing device can use a first neural network (e.g., DNN) 410 to extract the input embedding vector corresponding to the input biometric information.

[0107] In one example, the anti-spoofing device can estimate a first posterior probability between the input embedding vector and a representative embedding vector of the real model 420, and estimate a second posterior probability between the input embedding vector and a representative set of embedding vectors of the fake model 430.

[0108] The anti-spoofing device can calculate a first probability ratio as the ratio of a first posterior probability to a second posterior probability, and can determine the first probability ratio as a similarity value of the input embedding vectors. For example, the first probability ratio can be calculated as shown in Expression 1 below.

[0109] Expression 1:

[0110]

[0111] In expression 1, x represents the input embedding vector, Θ real Nbest represents the set of embedding vectors of the real model 420 (e.g., a representative set of embedding vectors), and Nbest represents the "N" spurious embedding vectors among the spurious embedding vectors included in the spurious model 430 that are most similar to the input embedding vectors.

[0112] In another example, the anti-spoofing device can estimate a first posterior probability between the input embedding vector and a representative set of embedding vectors of the real model 420, and a second posterior probability between the input embedding vector and a representative set of embedding vectors of the fake model 430. In this example, the first probability ratio can be calculated as shown in Expression 2 below.

[0113] Expression 2:

[0114]

[0115] The numerator of Expression 2 can correspond to the average of the posterior probabilities between the input embedding vector and the N best true embedding vectors (Nbest) that are most similar to the input embedding vector among the true embedding vectors included in the true model 420.

[0116] When the first probability ratio, which is the similarity value of the input embedding vector, is greater than or equal to a predetermined threshold, the anti-spoofing device can determine that the input biometric information is not forged. When the first probability ratio is less than the threshold, the anti-spoofing device can determine that the input biometric information is forged.

[0117] Reference Figure 4B The anti-spoofing device can determine the similarity value of the input embedding vector based on the fake embedding vector of the fake model 430 and the registered embedding vector of the registered model 440.

[0118] In one example, the anti-spoofing device can estimate a third posterior probability between the input embedding vector and the representative embedding vector of the registered model 440, and a second posterior probability between the input embedding vector and the representative embedding vector set of the fake model 430.

[0119] In this example, the anti-spoofing device can calculate a second probability ratio as the ratio of the second posterior probability to the third posterior probability, and can determine the second probability ratio as a similarity value of the input embedding vectors. For example, the second probability ratio can be calculated as shown in Expression 3 below.

[0120] Expression 3:

[0121]

[0122] In expression 3, Θ enroll This represents the representative embedding vector of registration model 440.

[0123] In another example, the anti-spoofing device can estimate a third posterior probability between the input embedding vector and a representative set of embedding vectors of the registered model 440, and a second posterior probability between the input embedding vector and a representative set of embedding vectors of the fake model 430. In this example, the second probability ratio can be calculated as shown in Expression 4 below.

[0124] Expression 4:

[0125]

[0126] The numerator of Expression 4 can correspond to the average of the posterior probabilities between the input embedding vector and the Nbest of the N registered embedding vectors that are most similar to the input embedding vector among the registered embedding vectors included in the registered model 440.

[0127] When the second probability ratio, which is the similarity value of the input embedding vector, is greater than or equal to a predetermined threshold, the anti-spoofing device can determine that the input biometric information is not forged. When the second probability ratio is less than the threshold, the anti-spoofing device can determine that the input biometric information is forged.

[0128] To control the variability of values ​​or limit the range of values, anti-spoofing devices can normalize the probability ratio using a logarithmic or sigmoid function. For example, the normalized probability ratio can be expressed as shown in Expression 5 below.

[0129] Expression 5:

[0130]

[0131] When the normalized probability ratio of the similarity values ​​used as input embedding vectors is greater than or equal to a predetermined threshold, the anti-spoofing device can determine that the input biometric information is not forged. When the normalized probability ratio is less than the threshold, the anti-spoofing device can determine that the input biometric information is forged.

[0132] Reference Figure 4C The anti-spoofing device can determine the similarity value of the input embedding vector based on the real embedding vector of the real model 420, the fake embedding vector of the fake model 430, and the registered embedding vector of the registered model 440. (See above for reference.) Figure 4A and Figure 4B As described, the anti-spoofing device can calculate a first probability ratio and a second probability ratio, and can determine a similarity value of the input embedding vector based on the first probability ratio and the second probability ratio. For example, the anti-spoofing device can determine the similarity value of the input embedding vector as the result of a weighted sum of the first probability ratio and the second probability ratio. For example, the result of a weighted sum of the first probability ratio and the second probability ratio can be expressed as shown in Expression 6 below, where β can represent the weight of the weighted summation.

[0133] Expression 6:

[0134]

[0135] When the similarity value of the input embedding vector is greater than or equal to a predetermined threshold γ th When the similarity value is less than the threshold, the anti-spoofing device can determine that the input biometric information is not forged. For example, the anti-spoofing device can determine whether the input biometric information has been forged based on Expression 7 shown below.

[0136] Expression 7:

[0137]

[0138] Although not shown in the accompanying drawings, the anti-spoofing device can determine a first category value corresponding to the case where the input biometric information has not been forged, based on a first distance between the real embedding vector and the input embedding vector and a third distance between the registered embedding vector and the input embedding vector. Furthermore, the anti-spoofing device can determine a second category value corresponding to the case where the input biometric information has been forged, based on a second distance between the fake embedding vector and the input embedding vector.

[0139] Anti-spoofing devices can determine the similarity value of input embedding vectors based on values ​​from a first category and values ​​from a second category. The anti-spoofing device can calculate the similarity value of input embedding vectors by applying the values ​​from the first category and the second category to a softmax function.

[0140] In one example, the anti-spoofing device can also determine the similarity value of the input embedding vectors based on expressions 8 and 9 shown below.

[0141] Expression 8:

[0142]

[0143] In expression 8, dist fake This represents the distance between the input embedding vector and the spurious embedding vector, and can correspond to the value of the second category. Furthermore, dist... enroll / real It can be calculated using expression 9, and can correspond to the value of the first category.

[0144] Expression 9:

[0145] dist enroll / real =γ×min(dist enroll dist real )+(1-γ)×dist enroll

[0146] In expression 9, dist enroll Dist represents the distance between the input embedding vector and the registered embedding vector. real This represents the distance between the input embedding vector and the true embedding vector, and `min()` represents the minimum value. Furthermore, γ represents the weight of the weighted summation and can have a value of "0.5".

[0147] Figure 5A and Figure 5B An example is shown of updating the initial model and the registration model based on the input embedding vector.

[0148] Reference Figure 5AThe fake fingerprint detection device can determine whether the number of fingerprint data entries stored in the initial model or registration model has reached the storage limit. If the number of fingerprint data entries in the initial model or registration model has reached the storage limit, the fake fingerprint detection device can determine which fingerprint data should be removed from the corresponding model. Figure 5A The above-mentioned update method is mainly used when updating the initial model based on changes in the terminal, but is not limited to this.

[0149] Reference Figure 5B When the number of fingerprint data entries stored in the initial model or registration model does not reach the storage limit, the fake fingerprint detection device can add the input embedding vector to the corresponding model until the number of fingerprint data entries reaches the storage limit. Figure 5B The above-mentioned update method is mainly used when updating the registration model based on fingerprint changes, but is not limited to this.

[0150] An example of updating the initial model and the registration model can be represented as shown in Expression 10 below.

[0151] Expression 10:

[0152] θ t =θ t-1 ∪{d(x,γ th )}-{g(θ t-1 ,d(x,γ) th ))}

[0153] In expression 10, θ t θ represents the updated initial model or the updated registered model. t-1 This represents the initial model that has not been updated or the registered model that has not been updated, d(x, γ). th ) represents the input embedding vector with a first confidence value greater than or equal to the first threshold. When the number of fingerprint data stored in the initial model or registration model does not reach the storage limit, g(θ) t-1 ,d(x,γ) th )) represents the empty set; when the number of fingerprint data stored in the initial model or registration model reaches the storage limit, g(θ) t-1 ,d(x,γ) th ) represents candidate fingerprint data (i.e., fingerprint data that will be removed).

[0154] Figure 6A and Figure 6B This shows an example of registering biometric information using anti-spoofing methods.

[0155] Anti-spoofing devices can generate registration models based on biometric information registered for biometric authentication. In one example, a registration model can be generated assuming that all registered biometric information has not been forged. In another example, a registration model can be generated using both a genuine model and a fake model based on biometric information determined to be authentic.

[0156] Reference Figure 6A The anti-spoofing device can generate a registration model assuming that all registered biometric information has not been forged. The anti-spoofing device can use a first neural network 610 to extract registration embedding vectors corresponding to the registered biometric information, and can store the registration embedding vectors in the registration model 620.

[0157] Reference Figure 6B The anti-spoofing device can use a real model 630 and a fake model 640 to generate a registration model based on biometric information determined to be authentic. The anti-spoofing device can use a first neural network 610 to extract a registration embedding vector corresponding to the registered biometric information. It can estimate the posterior probability between the registration embedding vector and the real embedding vector of the real model 630, and the posterior probability between the registration embedding vector and the fake embedding vector of the fake model 640. It can determine the confidence value of the registration embedding vector based on the probability ratio between the estimated posterior probabilities. The anti-spoofing device can store registration embedding vectors with confidence values ​​greater than or equal to a predetermined threshold in the registration model 620.

[0158] Anti-spoofing devices can receive input data including live biometric information under normal conditions and can determine whether the biometric information has been forged. In one example, because the input embedding vector corresponding to a real fingerprint image representing a normal fingerprint is similar to the registered embedding vector corresponding to a registered fingerprint image rather than the spoofing embedding vector corresponding to a spoofing fingerprint image, the anti-spoofing device can identify the input fingerprint included in the real fingerprint image as a real fingerprint rather than a spoofing fingerprint.

[0159] In another example, the anti-spoofing device may receive a genuine fingerprint image representing a dried fingerprint as an input fingerprint image and determine whether forgery has occurred. Because the input embedding vector corresponding to the genuine fingerprint image representing a dried fingerprint resembles a spoofing embedding vector rather than a registered embedding vector, the anti-spoofing device may incorrectly identify an input fingerprint included in the genuine fingerprint image representing a dried fingerprint as a spoofing fingerprint.

[0160] As mentioned above, since fingerprint registration is performed only once under predetermined conditions, fake fingerprint detection devices may incorrectly identify a person's genuine fingerprint as a fake fingerprint in environments different from the registration time. In the following text, reference will be made to... Figures 7A to 7DThis section describes an example of spoofing fingerprint identification based on a virtual registered fingerprint embedding vector generated from a registered fingerprint image. The anti-spoofing device can input a registered fingerprint image of a normal fingerprint into an artificial neural network (ANN) capable of altering the fingerprint's conditions to, for example, dry, oily, or low-temperature conditions. This generates a virtual registered image that retains structural characteristics while incorporating the altered environmental properties. This virtual registered image can then be used to determine whether the fingerprint has been forged. The following will refer to… Figures 7A to 7D An example of generating a virtual registration fingerprint embedding vector is further described.

[0161] Figures 7A to 7D An example of generating a virtual embedding vector is shown.

[0162] Reference Figure 7A The anti-spoofing device can generate virtual registered fingerprint images 701 and 702 by inputting the registered fingerprint image 700 into an image generator 710 constructed using an ANN.

[0163] The anti-spoofing device can input a virtual registration fingerprint image 701 representing a dry fingerprint into an embedding vector extractor 715 to generate a dry-condition virtual registration fingerprint embedding vector set including one or more virtual registration fingerprint embedding vectors corresponding to the dry fingerprint. Similarly, the anti-spoofing device can input a virtual registration fingerprint image 702 representing an oily fingerprint into an embedding vector extractor 715 to generate an oily-condition virtual registration fingerprint embedding vector set including one or more virtual registration fingerprint embedding vectors corresponding to the oily fingerprint.

[0164] Anti-spoofing devices can construct virtual registration fingerprint embedding vector models based on the generated virtual registration fingerprint embedding vector set under dry conditions and the generated virtual registration fingerprint embedding vector set under oily conditions.

[0165] The anti-spoofing device can input the registered fingerprint image 700 into the embedding vector extractor 715 to generate a registered fingerprint embedding vector, and can construct a registered fingerprint embedding vector model 722 based on the registered fingerprint embedding vector. The virtual registered fingerprint embedding vector model 721 and the registered fingerprint embedding vector model 722 can be stored in the registered fingerprint DB.

[0166] Reference Figure 7B The anti-spoofing device can also generate a virtual fake fingerprint image 703 by inputting the registered fingerprint image 700 into the image generator 710. The anti-spoofing device can determine whether spoofing has occurred based on the virtual fake fingerprint image 703, which maintains the structural characteristics of the registered fingerprint image 700, thereby enhancing the performance of determining whether the input fingerprint image has been forged.

[0167] The anti-spoofing device can input a virtual fake fingerprint image 703 into an embedding vector extractor 715, generate a virtual fake fingerprint embedding vector set including one or more virtual fake fingerprint embedding vectors, and construct a virtual fake fingerprint embedding vector model 723 based on the virtual fake fingerprint embedding vector set.

[0168] Reference Figure 7C The anti-spoofing device can directly generate virtual registered fingerprint embedding vectors from the registered fingerprint image 700 instead of generating a virtual registered fingerprint image. The anti-spoofing device can input the registered fingerprint image 700 into the embedding vector generator 711 and generate multiple sets of virtual registered fingerprint embedding vectors with different environmental characteristics. The anti-spoofing device can construct a virtual registered fingerprint embedding vector model 721 based on the multiple generated sets of virtual registered fingerprint embedding vectors (e.g., a set of virtual registered fingerprint embedding vectors for dry conditions and a set of virtual registered fingerprint embedding vectors for oily conditions).

[0169] Furthermore, the anti-spoofing device can input the registered fingerprint image 700 into the embedding vector generator 711 to generate a set of virtual spoofed fingerprint embedding vectors, including one or more virtual spoofed fingerprint embedding vectors, and can construct a virtual spoofed fingerprint embedding vector model 723 based on the set of virtual spoofed fingerprint embedding vectors. Additionally, the anti-spoofing device can input the registered fingerprint image 700 into the fifth ANN 740 to generate registered fingerprint embedding vectors, and can construct a registered fingerprint embedding vector model 722 based on the registered fingerprint embedding vectors. (Refer to the following...) Figure 7D Further examples of the operation of image generator 710 and embedding vector generator 711 are described. In one embodiment, the anti-spoofing device may calculate a similarity value based on any one or any combination of registration embedding vector, real embedding vector, and virtual registration fingerprint embedding vector, as well as spoofing embedding vector.

[0170] Figure 7D An example of the operation of image generator 710 and embedding vector generator 711 is shown.

[0171] Reference Figure 7D The image generator 710 and the embedding vector generator 711 can operate based on the same principle. For ease of description, the operation of the image generator 710 is described first.

[0172] The image generator 710 can be implemented by an ANN, and the ANN can be trained such that the virtual registered fingerprint image can have environmental characteristics different from those of the registered fingerprint image 750 while maintaining the structural characteristics of the registered fingerprint image 750.

[0173] An ANN may include a G generator 760, an F generator 765, a D discriminator 770 (e.g., a real / fake discriminator), and a C discriminator 775 (e.g., a liveness / spoofing anti-spoofing (AS) classifier). An ANN can generate desired output data even without paired training data.

[0174] The G generator 760 can convert the registered fingerprint image 750 into a virtual registered fingerprint image 751, and the F generator 765 can recover the registered fingerprint image from the virtual registered fingerprint image 751. The ANN can determine the model parameters that minimize the difference between the recovered registered fingerprint image 752 and the original registered fingerprint image 750 during the training operation.

[0175] The D discriminator 770 can be used to allow the virtual registration fingerprint image 751 to have desired environmental characteristics. The D discriminator 770 can be trained such that the virtual registration fingerprint image 751 can have predetermined environmental characteristics based on a fingerprint image 753 with predetermined environmental characteristics. For example, in order to train the virtual registration fingerprint image 751 to represent a dried fingerprint, the ANN can determine model parameters that minimize the difference between the fingerprint image representing the dried fingerprint and the virtual registration fingerprint image 751.

[0176] The C-discriminator 775 can be used to allow the virtual registered fingerprint image 751 to have characteristics of a real fingerprint image, rather than characteristics of a fake fingerprint image. For example, a real fingerprint image representing a dry fingerprint and a fake fingerprint image representing a fingerprint forged using wood glue can have similar characteristics, and a real fingerprint image representing an oily fingerprint and a fake fingerprint image representing a fingerprint forged using gelatin can have similar characteristics. Therefore, the C-discriminator 775 can determine model parameters that minimize the difference between the virtual registered fingerprint image 751 and the real fingerprint image.

[0177] An ANN can be trained to determine model parameters that minimize the loss function. The loss function can be used as an indicator of the optimal model parameters during the training process of the ANN. Model parameters can be parameters determined through training and may include the weights of synaptic connections or the biases of neurons. Furthermore, hyperparameters can be parameters that need to be set before learning using a machine learning algorithm and may include the learning rate, number of repetitions, minimum batch size, or initialization function. The loss function can be expressed as shown in Expression 11 below.

[0178] Expression 11:

[0179] Loss = Loss Gwv (G,D,I E ,I v )+Loss GAN (F,D,I v ,IE′ )+λ i Loss cycle (G,F)+λ2Loss AS (G,F)

[0180] In expression 11, I E It is the registered fingerprint image 750, I V It is a virtual registration fingerprint image 751, I E' G is the recovered registered fingerprint image 752, F is the function that converts the registered fingerprint image 750 into a virtual registered fingerprint image 751, D is the function that converts the virtual registered fingerprint image 751 into the recovered registered fingerprint image 752, and λ1 and λ2 can represent coefficients. Furthermore, in expression 11, Loss... GAN (F, D, I) E I V ) is the generative adversarial network (GAN) loss function associated with the image generated by the G generator 760. GAN (F, D, I) V I E ) is the GAN loss function associated with the image generated by the F generator 765, Loss cycle (G, F) is the cycle consistency loss function associated with the registered fingerprint image 750 and the recovered registered fingerprint image 752, and Loss AS (G, F) is the loss function of the C discriminator 775 associated with the virtual registered fingerprint image 751 and the real fingerprint image.

[0181] Figure 8A An example of the relationship between the embedding vector derived from the embedding vectors of the attempt to register and the input embedding vector of the attempt to authenticate is shown in three dimensions.

[0182] Figure 8A The diagram shows an input embedding vector 810, a registered embedding vector 820 in the embedding cluster, and an embedding vector 830 (hereinafter referred to as the "adaptive embedding vector") that is adapted to the input embedding vector 810 and determined by a combination of the registered embedding vector 820.

[0183] For example, multiple registered embedding vectors 820 included in the embedding cluster based on registration data can be assumed to be sparsely arranged. In this example, the similarity between the input embedding vector 810 in the embedding cluster and the registered embedding vector that is closest to the input embedding vector 810, or the similarity between the input embedding vector 810 and statistically representative embedding vectors (e.g., the average of the registered embedding vectors 820 included in the embedding cluster), can indicate a result that differs from the actual similarity.

[0184] In the similarity between the input embedding vector 810 and the closest registered embedding vector, the blind spot may be caused by the sparse distribution of registered embedding vectors in the embedding space. Furthermore, the similarity between the input embedding vector 810 and statistically representative embedding vectors can dilute the meaning of individual registered embedding vectors in the embedding cluster.

[0185] In one example, by determining the adaptive embedding vector 830 as the input embedding vector 810 through a combination of multiple registered embedding vectors 820, blind spots in the embedding space can be eliminated, and the dilution of the meaning of the individual embedding vectors can be prevented. In this example, the adaptive embedding vector 830 may correspond to the embedding vector most similar to the input embedding vector 810 among the combinations of registered embedding vectors 820 in the embedding cluster. Furthermore, the combination of embedding vectors similar to the input embedding vector 810 among the combinations of registered embedding vectors 820 may be referred to as a "combined embedding vector". The combined embedding vector may correspond to one or more candidates of the adaptive embedding vector 830. For example, the final determined combined embedding vector among the combined embedding vectors may be the adaptive embedding vector 830. If a combined embedding vector is provided, the combined embedding vector may be the adaptive embedding vector 830.

[0186] In one example, the physical sparsity in the embedding cluster can be addressed using combinations and / or interpolation of the registered embedding vectors 820 included in the embedding cluster. In this example, various examples can be provided based on the combination schemes of the registered embedding vectors 820. For instance, if the combined embedding vector obtained by any combination f of the registered embedding vectors 820 is... This indicates that the optimal combination f to be obtained will be... O It can be represented as shown in expression 12 below.

[0187] Expression 12:

[0188]

[0189] In expression 12, x i This represents the input embedding vector 810, and This represents the registered embedding vector 820 included in the embedding cluster. The combined embedding vector can be obtained from... Let S(x, y) represent the similarity between embedding vectors x and y. Let L(S(·, ·)) represent the loss function related to similarity, where argmin f Let f represent the combination that minimizes the loss function.

[0190] In one example, the search can be performed by finding the "input embedding vector x" as shown in Expression 12. i210 and the combined embedding vectors in embedding clustering The loss function L and the combination function f are defined to appropriately combine the adaptive embedding vector 830 to obtain the appropriate similarity between the input embedding vector 810 and the registered embedding vector 820 included in the embedding cluster.

[0191] Figure 8B This shows an example of a method for classifying input data.

[0192] Figure 8B The process by which the classification device measures the similarity between the input embedding vector 810 and each registered embedding vector included in the embedding cluster 840 is illustrated.

[0193] The classification device can use, for example, a DNN to extract an input embedding vector 810 from the input data. The classification device can use the input embedding vector 810 as an independent variable for an optimization algorithm 850 that optimizes similarity measurement and combination schemes. In this example, since the optimization algorithm 850 combines embedding vectors that are adaptive to the input embedding vector, the optimization algorithm 850 can also be referred to as an "adaptive algorithm".

[0194] The classification device can use a DNN to convert the registration data included in the embedding cluster 840 into registration embedding vectors. The registration embedding vectors can also be used as independent variables for the optimization algorithm 850 and the combination scheme.

[0195] The classification device can apply optimization algorithm 850 to the registered embedding vectors in the input embedding vector 810 and embedding clustering 840, and can obtain the optimal combination scheme in operation 860 based on the result of applying optimization algorithm 850. The classification device can generate combined embedding vectors, which can prevent errors in similarity calculation by adaptively clustering to a combination of embedding vectors of the input embedding vector.

[0196] In operation 870, the classification device can measure the similarity between the input embedding vector 810 and the combined embedding vector determined according to the obtained optimal combination scheme. The classification device can use the measured similarity without alteration, or it can convert the measured similarity into the form of a similarity score 880 and use the similarity score 880.

[0197] In one example, a DNN may include a large number of artificial neurons connected by linear edges. These artificial neurons may be referred to as nodes. Artificial neurons are interconnected by edges with connection weights. These connection weights may be predetermined values ​​for the edges and may be referred to as "synaptic weights" or "connection strengths." A DNN may include, for example, an input layer, hidden layers, and an output layer. Each of the input, hidden, and output layers may include multiple nodes. Nodes included in the input layer may be referred to as "input nodes," nodes included in the hidden layers may be referred to as "hidden nodes," and nodes included in the output layer may be referred to as "output nodes."

[0198] The input layer receives input for training or recognition and can send this input to the hidden layers. The output layer generates the DNN's output based on the signals received from the hidden layers. The hidden layers can be located between the input and output layers and can modify the training input from the training data received via the input layers into relatively predictable values. Input nodes included in the input layer and hidden nodes included in the hidden layers can be connected to each other via weighted connections. Hidden nodes included in the hidden layers and output nodes included in the output layers can be connected to each other via weighted connections.

[0199] The classification device can input the output of a previous hidden node, which is included in the previous hidden layer, into the corresponding hidden layer via connection lines with connection weights. The classification device can generate the output of the hidden nodes included in the hidden layer based on the activation function and the values ​​obtained by applying the connection weights to the output of the previous hidden node. In one example, the output can be sent to the next hidden node when the result of the activation function exceeds a threshold of the current hidden node. In this example, the current hidden node can remain inactive instead of sending a signal to the next hidden node until a predetermined threshold activation strength is reached through the input vector.

[0200] A classification device can be trained using a DNN through supervised learning. The classification device can be implemented, for example, through hardware modules or a combination thereof. Supervised learning can be a scheme where training data, along with the corresponding training outputs, is fed into the DNN, and the connection weights of the connections are updated so that output data corresponding to the training outputs can be output. Training data can be data that includes pairs of training inputs and training outputs.

[0201] Figure 9 An example of an anti-spoofing device is shown.

[0202] Reference Figure 9 The anti-spoofing device 900 includes a processor 910. The anti-spoofing device 900 may also include a memory 930, a communication interface 950, and a sensor 970. The processor 910, memory 930, communication interface 950, and sensor 970 can communicate with each other via a communication bus 905.

[0203] The processor 910 can extract the input embedding vector corresponding to the input biometric information, obtain the false embedding vector of the pre-provided fake model, and obtain the real embedding vector of the pre-provided real model and / or the registration embedding vector of the registered model. It can determine the confidence value of the input embedding vector based on any one or both of the real embedding vector and the registration embedding vector and the false embedding vector, and determine whether the input biometric information has been forged.

[0204] The processor 910 can extract an input embedding vector containing features of the biometric information from input data including the user's biometric information. It can detect first information regarding whether the biometric information has been forged based on a first output vector of a first neural network configured to detect whether the biometric information has been forged from the input data. Furthermore, it can calculate a similarity value of the input embedding vector based on any one or both of a pre-provided true embedding vector and a registered embedding vector, as well as a spoof embedding vector. Additionally, the processor 910 can calculate a total forgery score based on the similarity value and a second output vector of the first neural network, depending on whether the first information has been detected, and can detect second information regarding whether the biometric information has been forged based on the total forgery score.

[0205] The memory 930 may include a database (DB) configured to store a registration model, a real model, and / or a spoof model. The memory 930 may store pre-training parameters of a neural network. The sensor 970 may be a means for sensing a user's biometric information and may include, for example, a fingerprint sensor for sensing the user's fingerprint.

[0206] The processor 910 can estimate the posterior probability between the input embedding vector and any one or any combination of the true embedding vector, the spurious embedding vector, and the registered embedding vector, and can determine the confidence value of the input embedding vector based on the posterior probability.

[0207] Processor 910 can execute the above reference. Figures 2 to 8B The described method or algorithm. Processor 910 can execute programs and control anti-spoofing device 900. The program code to be executed by processor 910 can be stored in memory 930. Anti-spoofing device 900 can be connected to an external device (e.g., a PC or network) via an input / output device (not shown) to exchange data with the external device. Anti-spoofing device 900 can be installed on various computing devices and / or systems (such as smartphones, tablets, laptops, desktop computers, TVs, wearable devices, security systems, or smart home systems).

[0208] Regarding Figure 1 , Figure 3A , Figures 4A to 4C , Figure 6A, Figure 6B , Figures 7A to 7D and Figure 8B The described anti-spoofing devices, as well as other means, devices, units, modules, and components, are implemented by or represent hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented by one or more processing elements, such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" are used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component, or two or more hardware components, may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0209] For example Figure 2 and Figure 3BThe methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., one or more processors or a computer), which is implemented to execute instructions or software as described above to perform the operations performed by the methods described in this application. For example, a single operation, or two or more operations, may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation, or two or more operations.

[0210] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0211] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed in, or may be recorded, stored, or fixed on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), flash memory, card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers, enabling one or more processors or computers to execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0212] While this disclosure includes specific examples, it will be apparent to those skilled in the art that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example should be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

[0213] Therefore, the scope of this disclosure is not limited by the specific embodiments, but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be interpreted as included in the disclosure.

Claims

1. A method for preventing deception, comprising: An intermediate forgery score is calculated based on a first output vector extracted from an intermediate layer of a first neural network, which is configured to detect whether biometric information has been forged from input data including biometric information. The first piece of information used to detect whether a user's biometric information has been forged is based on intermediate forgery scores. Extract input embedding vectors containing features including biometric information from the input data; The similarity value of the input embedding vector is calculated based on one or both of the real embedding vector and the registered embedding vector, as well as the spurious embedding vector. The real embedding vector and the registered embedding vector are provided in advance. When the first information is not detected, the final forgery score is calculated based on the second output vector output from the output layer of the first neural network, and the total forgery score is calculated based on the final forgery score and the similarity value. as well as Based on the total forgery score, a second piece of information is detected regarding whether the biometric information has been forged.

2. The anti-deception method according to claim 1, wherein, The steps for detecting the first information include: Determine whether the intermediate forged score is within the threshold range used to detect the first information; and In response to determining that the intermediate forged score is within a threshold range, the first information is detected based on the intermediate forged score.

3. The anti-deception method according to claim 2, wherein, The threshold range includes the range within which biometric information is identified as live information and the range within which biometric information is identified as deceptive information.

4. The anti-deception method according to claim 1, wherein, The steps to calculate the total fraud score include: Perform a weighted summation of the final forgery score and similarity value; and The total fraud score is calculated by applying a nonlinear function to the weighted summation result.

5. The anti-deception method according to claim 1, wherein, The steps for detecting the second information include: detecting the second information by comparing the total forgery score with a first threshold used to detect the second information.

6. The anti-spoofing method according to claim 5 further includes: Determine whether the embedding vector update condition of the input embedding vector is met based on the total forgery score; as well as Based on the determination that the embedding vector update condition is met, the input embedding vector is used to update any one or any combination of the registered embedding vector, the true embedding vector, and the spurious embedding vector.

7. The anti-spoofing method according to claim 1 further includes: Generate virtual registration embedding vectors that have environmental characteristics different from those of the registration embedding vectors but possess the structural characteristics of the registration embedding vectors. The steps for calculating the similarity value include: calculating the similarity value based on any one or any combination of the registered embedding vector, the real embedding vector, and the virtual registered embedding vector, as well as the spurious embedding vector.

8. The anti-spoofing method according to claim 1 further includes: Similarity values ​​are calculated based on the posterior probability between the input embedding vector and any one or any combination of the true embedding vector, the registered embedding vector, and the spurious embedding vector. The posterior probability is calculated based on the distance or similarity between the input embedding vector and any one or any combination of the true embedding vector, the registered embedding vector, and the spurious embedding vector.

9. The anti-deception method according to claim 1, wherein, The steps for calculating similarity values ​​include: Based on the first distance between the real embedding vector and the input embedding vector and the third distance between the registered embedding vector and the input embedding vector, the value of the first category corresponding to the case where the biometric information has not been forged is determined; Based on the second distance between the fake embedding vector and the input embedding vector, a value for the second category corresponding to the case of forged biometric information is determined; and The similarity value of the input embedding vector is determined based on the values ​​of the first category and the values ​​of the second category.

10. The anti-spoofing method according to any one of claims 1 to 9, wherein, The steps for extracting the input embedding vector include: using a first neural network to extract the input embedding vector from biometric information, and The first neural network was trained based on both live biological feature information and deceptive biological feature information.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 10.

12. An anti-spoofing device, comprising: The processor is configured as follows: An intermediate forgery score is calculated based on a first output vector extracted from an intermediate layer of a first neural network, which is configured to detect whether biometric information has been forged from input data including biometric information. The first piece of information used to detect whether a user's biometric information has been forged is based on intermediate forgery scores. Extract input embedding vectors containing features including biometric information from the input data; The similarity value of the input embedding vector is calculated based on one or both of the real embedding vector and the registered embedding vector, as well as the spurious embedding vector. The real embedding vector and the registered embedding vector are provided in advance. When the first information is not detected, the final forgery score is calculated based on the second output vector output from the output layer of the first neural network, and the total forgery score is calculated based on the final forgery score and the similarity value. as well as Based on the total forgery score, a second piece of information is detected regarding whether the biometric information has been forged.

13. The anti-spoofing device according to claim 12, wherein, The processor is configured as follows: Determine whether the intermediate forged score is within the threshold range used to detect the first information; and In response to determining that the intermediate forged score is within a threshold range, the first information is detected based on the intermediate forged score.

14. The anti-spoofing device according to claim 12, wherein, The processor is configured as follows: Perform a weighted summation of the final forgery score and similarity value; and The total fraud score is calculated by applying a nonlinear function to the weighted summation result.

15. The anti-spoofing device according to claim 12, wherein, The processor is configured to detect the second information by comparing the total forgery score with a first threshold used to detect the second information.

16. The anti-spoofing device according to claim 15, wherein, The processor is configured as follows: Determine whether the embedding vector update condition of the input embedding vector is met based on the total forgery score; as well as Based on the determination that the embedding vector update condition is met, the input embedding vector is used to update any one or any combination of the registered embedding vector, the true embedding vector, and the spurious embedding vector.

17. The anti-spoofing device according to claim 12, wherein, The processor is configured as follows: Generate a virtual registration embedding vector that has environmental characteristics different from those of the registration embedding vector but has the structural characteristics of the registration embedding vector; as well as The similarity value is calculated based on any one or any combination of the registered embedding vector, the real embedding vector, and the virtual registered embedding vector, as well as the spurious embedding vector.

18. The anti-spoofing device according to claim 12, wherein, The processor is configured as follows: Similarity values ​​are calculated based on the posterior probability between the input embedding vector and any one or any combination of the true embedding vector, the registered embedding vector, and the spurious embedding vector. The posterior probability is calculated based on the distance or similarity between the input embedding vector and any one or any combination of the true embedding vector, the registered embedding vector, and the spurious embedding vector.

19. The anti-spoofing device according to claim 12, wherein, The processor is configured as follows: Based on the first distance between the real embedding vector and the input embedding vector and the third distance between the registered embedding vector and the input embedding vector, the value of the first category corresponding to the case where the biometric information has not been forged is determined; Based on the second distance between the fake embedding vector and the input embedding vector, the value of the second category corresponding to the case of forged biometric information is determined. as well as The similarity value of the input embedding vector is determined based on the values ​​of the first category and the values ​​of the second category.

20. The anti-spoofing device according to any one of claims 12 to 19, wherein, The processor is configured to: extract an input embedding vector from biometric information using a first neural network, and The first neural network was trained based on both live biological feature information and deceptive biological feature information.

21. An anti-spoofing device, comprising: One or more processors are configured as follows: The intermediate forgery score is calculated based on the first output vector extracted from the intermediate layer of the neural network, which is trained using live biometric information and deceptive biometric information. The first piece of information used to detect whether a user's biometric information has been forged is based on intermediate forgery scores. Extract the input embedding vector that includes the biometric information contained in the input data; The similarity value of the input embedding vector is calculated based on one or both of the real embedding vector and the registered embedding vector, as well as the spurious embedding vector. When the first information is not detected, the final forgery score is calculated based on the second output vector output from the output layer of the neural network, and the total forgery score is calculated based on the final forgery score and the similarity value. as well as The total forgery score is used to determine whether biometric information has been forged.

22. The anti-spoofing device according to claim 21, wherein, The one or more processors are configured to: If the total forgery score exceeds the first threshold, the biometric information is determined to be from a living individual; and If the total forgery score is less than or equal to the first threshold, the biometric information is determined to be deceptive.

23. The anti-spoofing device according to claim 22, wherein, The one or more processors are configured to: The first threshold is determined based on the rejection threshold and the acceptance threshold. The rejection threshold corresponds to the maximum probability that the intermediate forgery score based on the first output vector falls within the range where the biometric information is determined to be deceptive information. The acceptance threshold corresponds to the minimum probability that the intermediate forgery score falls within the range where the biometric information is determined to be live information.

Images