Adaptive personalization for anti-spoofing protection in biometric authentication systems

Abstract

Some aspects of the present disclosure provide techniques and apparatus for biometric authentication using an improved anti-spoofing protection model using online data. The method generally includes receiving a biometric data input of a user. Features of the received biometric data input are extracted through a first machine learning model. Using the features of the received biometric data input and a second machine learning model, it is determined whether the received biometric data input of the user is authentic or inauthentic. It is determined whether to add the extracted features of the received biometric data input labeled with an indication of whether the received biometric data input is authentic or inauthentic to a fine-tuning dataset. The second machine learning model is adjusted based on the fine-tuning dataset.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS)

[0001] This application claims priority to U.S. patent application Ser. No. 18 / 155,408, entitled "Adaptive Personalization for Anti-Spoofing Protection in Biometric Authentication Systems," filed on January 17, 2023, which claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 267,985, entitled "Adaptive Personalization for Anti-Spoofing Protection in Biometric Authentication Systems," filed on February 14, 2022 and assigned to the assignee of this application, the contents of each of which are incorporated herein by reference in their entirety.

[0002] introduction Aspects of the present disclosure relate to using artificial neural networks to protect against biometric credential spoofing in biometric authentication systems. [Background technology]

[0003]

[0003] In various computing systems, such as smartphones, tablet computers, etc., users may authenticate and gain access to these computing systems using various techniques, either alone (e.g., single-factor authentication) or in combination with each other (e.g., multi-factor authentication). One authentication technique involves the use of biometric data to authenticate a user. Biometric data generally includes information derived from a user's physical characteristics, such as fingerprint data, iris scan data, face scan data, etc.

[0004]

[0004] In a biometric authentication system, a user typically enrolls in an authentication service (e.g., performed locally on the device or remotely on a separate computing device) by providing one or more scans of relevant biometric features (e.g., body parts) to the authentication service, which may be used as a reference data source. For example, in a biometric authentication system in which a fingerprint is used to authenticate a user, multiple fingerprint scans may be provided to account for differences in how the user holds the device, to account for differences between different regions of the finger, and to account for different fingers that may be used in authenticating the user. In another example, in a biometric authentication system in which a user's face is used for authentication, multiple images of the user's face may be provided to account for different angles or perspectives that may be used in capturing an image of the user's face for authentication. When a user attempts to access a device, the user may scan or otherwise capture an image of a relevant body part, and the captured image (or a representation thereof) may be compared against a reference (e.g., a reference image or a representation thereof). If the captured image sufficiently matches the reference image, the user may be granted access to the device or application. If not, access to the device or application may be denied because a poor match may indicate that an unauthorized or unknown user is attempting to access the device or application.

[0005]

[0005] Although biometric authentication systems add an additional layer of security to access control systems over passwords or passcodes, techniques exist to circumvent these biometric authentication systems. For example, in a fingerprint-based biometric authentication system, a fingerprint may be authenticated based on the similarity between ridges and valleys captured in a query image and between ridges and valleys captured in one or more enrollment images (e.g., through ultrasonic sensors, optical sensors, etc.). In another example, in an image-based facial recognition system, facial recognition may be achieved based on parts of a user's face that may be replicated in other images. Because the general techniques by which these biometric authentication systems authenticate users are known, it may be possible to attack these authentication systems and use copies of a user's biometric data to gain unauthorized access to protected resources. These types of attacks are sometimes referred to as "spoofing" attacks. Summary of the Invention

[0006]

[0006] Some aspects provide a method for biometric authentication using an improved anti-spoofing protection model using online data. The method generally includes receiving a biometric data input of a user. Features of the received biometric data input are extracted through a first machine learning model. Using the features of the received biometric data input and a second machine learning model, it is determined whether the received biometric data input of the user is authentic or inauthentic. It is determined whether to add the extracted features of the received biometric data input labeled with an indication of whether the received biometric data input is authentic or inauthentic to a fine-tuning dataset. The second machine learning model is trained based on the fine-tuning dataset.

[0007]

[0007] Another aspect provides a processing system configured to perform the aforementioned methods as well as methods described herein; a non-transitory computer readable medium comprising instructions which, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods as well as methods described herein; a computer program product embodied on a computer readable storage medium comprising code for performing the aforementioned methods as well as methods further described herein; and the processing system comprising means for performing the aforementioned methods as well as methods further described herein.

[0008] The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. [Brief description of the drawings]

[0009]

[0009] The accompanying drawings illustrate several aspects of the present disclosure and therefore should not be considered as limiting the scope of the present disclosure. [Figure 1]

[0010] 1 illustrates an exemplary biometric authentication pipeline. [Diagram 2]

[0011] 1 illustrates an exemplary anti-spoofing protection system in a biometric authentication pipeline. [Diagram 3]

[0012] 1 illustrates the use of current and past biometric authentication data inputs in a biometric authentication pipeline, according to an aspect of the present disclosure. [Figure 4]

[0013] 1 illustrates a biometric authentication system having anti-spoofing protection based on online adaptive personalization, according to an aspect of the present disclosure. [Diagram 5]

[0014] 1 illustrates an example operation of authenticating biometric data based on a fine-tuning data set generated from captured biometric data and adjusting an anti-spoofing protection model for biometric authentication according to aspects of the present disclosure. [Figure 6]

[0015] 1 illustrates an example thresholding technique for adding captured biometric data to a fine-tuning data set for tuning an anti-spoofing protection model, according to an aspect of the present disclosure. [Figure 7]

[0016] 1 illustrates an example adjustment of labels for captured biometric data based on labels assigned to other captured biometric data, according to aspects of the present disclosure. [Figure 8]

[0017] 1 illustrates an example weighting of captured biometric data in a fine-tuning data set for tuning an anti-spoofing protection model, according to an aspect of the present disclosure. [Figure 9]

[0018] 1 illustrates an example implementation of a processing system in which biometric authentication and anti-spoofing protection in a biometric authentication pipeline may be performed, according to aspects of the present disclosure.

[0010]

[0019] For ease of understanding, wherever possible, like reference numerals have been used to designate like elements common to the figures, and it is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further detail. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011]

[0020] Aspects of the present disclosure provide techniques for anti-spoofing protection for biometric authentication systems and methods.

[0012]

[0021] In many biometric security systems, an image (or sample) of a user's biometric features (e.g., a fingerprint image obtained from an image scan or ultrasonic sensor configured to generate an image based on reflections from fingerprint ridges and valleys, facial structure derived from a face scan, iris structure derived from an iris scan, etc.) is captured for use in authenticating the user. The acceptable similarity between the captured image and a reference image may be adjusted to satisfy false acceptance rate (FAR) and false rejection rate (FRR) metrics. The FAR may represent the rate at which the biometric security system erroneously grants access to a system or application (e.g., to a user other than the user(s) associated with the reference image(s) in the biometric security system), and the FRR may represent the rate at which the biometric security system erroneously blocks access to a system or application. In general, a false acceptance may constitute a security breach, while a false rejection may be a nuisance (e.g., by delaying access to the system). Because biometric security systems are frequently used to grant or deny access to potentially sensitive information or systems, and because false accepts are generally dangerous, biometric security systems may typically be configured to minimize the FAR as close to zero as possible, at the expense of an increased FRR.

[0013]

[0022] In some cases, biometric security systems may be fooled (or "spoofed") into accepting spoofed biometric authentication information, which may enable unauthorized access to protected resources within a computing system and other security breaches. For example, in some fingerprint authentication systems, fake fingers created with fingerprints lifted from another location may be used to gain unauthorized access to protected computing resources. These fake fingers may be easily created, for example, using 3D printing or other additive manufacturing processes, gelatin molding, or other processes. In other cases, an image or model of a user's face may be used to gain unauthorized access to protected computing resources protected by a facial recognition system. Because fake biometric data sources can be easily created, biometric authentication systems generally include anti-spoofing protection systems that attempt to distinguish biometric data from real or fake sources.

[0014] Exemplary Biometric Data Authentication Pipeline

[0023] 1 illustrates an example biometric authentication pipeline 100 in accordance with certain aspects of the disclosure. Although the biometric authentication pipeline 100 is illustrated as a fingerprint authentication pipeline, it should be appreciated that the biometric authentication pipeline 100 may also or alternatively be used in capturing and authenticating other biometric data, such as face scans, iris scans, and other types of biometric data. Similarly, while various aspects refer to capturing an image (e.g., by a sensor), it should be appreciated that other types of samples may be captured for authentication (in addition to or as an alternative to an image).

[0015]

[0024] As shown, biometric data, such as (but not limited to) an image (or sample) of a fingerprint, is captured by a sensor 110 and provided to a comparator 120, which determines whether the biometric data captured by the sensor 110 corresponds to one of a plurality of known sets of biometric data (e.g., whether the captured image of a fingerprint corresponds to a known fingerprint). The sensor 110 may be, for example, an imaging sensor, a scanner, an ultrasonic sensor, or other sensor capable of generating image data from a scan of a user biometric. To determine whether the biometric data captured by the sensor 110 corresponds to one of a plurality of known sets of biometric data, the comparator 120 may compare the captured biometric data (or features derived therefrom) to samples (or features derived therefrom) in an enrollment sample set captured when a user enrolls one or more biometric data sources (e.g., a finger) for use in authenticating the user. Typically, the enrollment image set includes multiple images for each biometric data source enrolled in the fingerprint authentication system. However, for security purposes, the actual enrollment image may be stored in a protected area in memory (not shown), or a representation of the enrollment image may be stored in place of the actual enrollment image to protect against extraction and malicious use of the enrollment image.

[0016]

[0025] In general, the comparator 120 may identify unique physical features in the captured biometric data and attempt to match these unique physical features with similar physical features in one of the enrollment samples (e.g., an enrollment image). For example, in a fingerprint authentication system, the comparator 120 may identify ridge and valley patterns in the fingerprint and / or fingerprint minutiae, such as ridge / valley bifurcations or terminations, to attempt to match the captured fingerprint with the enrollment image. In some cases, the comparator 120 may apply various transformations to the captured biometric data to attempt to align features in the captured biometric data with similar features in one or more of the images in the enrollment image set. These transformations may include, for example, applying a rotational transformation (i.e., rotating) to the captured biometric data, shifting laterally (i.e., translating) the captured biometric data, scaling the captured biometric data to a defined resolution, combining the captured biometric data with one or more of the enrollment images in the enrollment image set to create a composite image, etc. If the comparator 120 determines that the captured biometric data does not match any of the images in the enrollment image set, the comparator 120 can determine that the captured biometric data is not from an enrolled user and can deny access to the protected computing resource.

[0017]

[0026] Otherwise, if the comparator 120 determines that the captured biometric data matches at least one of the images in the enrollment image set, the anti-spoofing protection engine 130 may determine whether the captured biometric data is from a genuine source or a fake source. If the anti-spoofing protection engine 130 determines that the captured biometric data is from a genuine source, the anti-spoofing protection engine 130 may allow access to the protected computing resource. Otherwise, the anti-spoofing protection engine 130 may refuse to allow access to the protected computing resource. Various techniques may be used to determine whether the captured biometric data is from a genuine source or a fake source. For example, in a fingerprint authentication system, surface conductivity may be used to determine whether a fingerprint image is from a genuine finger or a fake finger. Because human skin has certain known conductivity characteristics, an image captured from a source that does not have these conductivity characteristics may be determined to be sourced from a fake finger. However, because these techniques are typically performed without reference to an enrollment image set and / or a captured fingerprint image, anti-spoofing protection systems can be defeated through the use of various materials or other technological means that replicate known anatomical characteristics of genuine biometric data sources that might otherwise be used to prevent spoofing attacks.

[0018]

[0027] 1 illustrates a biometric authentication pipeline in which a comparison is performed before determining whether the captured biometric data (e.g., a captured image of a fingerprint) is from a genuine or fake source, it should be recognized by one of ordinary skill in the art that these operations may be performed in any order or simultaneously. That is, within the biometric authentication pipeline, the anti-spoofing protection engine 130 may determine whether the captured biometric data is from a genuine or fake source before the comparator 120 determines whether a match exists between the biometric data captured by the sensor 110 and one or more images in the enrollment image set.

[0019] Exemplary Anti-Spoofing Protection System in a Fingerprint Authentication Pipeline

[0028] FIG. 2 illustrates an exemplary anti-spoofing protection system 200 in a biometric authentication pipeline, such as (but not limited to) a fingerprint authentication pipeline.

[0020]

[0029] In the anti-spoofing protection system 200, a sample 202 captured by a sensor (e.g., an ultrasonic sensor, an optical sensor, etc.) may be provided as an input to an anti-spoofing protection model 204. The anti-spoofing protection model 204 may be generally trained based on a predefined training data set to determine whether the captured sample 202 is from a real finger or a fake finger (e.g., to make a live or spoof decision that may be used in a fingerprint authentication pipeline to determine whether to allow a user access to a protected computing resource). However, the anti-spoofing protection model 204 may be relatively inaccurate because the training data set used to train the anti-spoofing protection model 204 may not take into account natural variations between users that may change the characteristics of the sample 202 captured for different users. For example, users may have different skin characteristics, such as dry skin, oily skin, etc., that may affect the data captured in the sample 202. A user with dry skin may, for example, cause the generation of a sample 202 with lower visual acuity than a user with oily skin. In addition, the anti-spoofing protection model 204 may not account for differences between sensors and / or surface coatings for sensors used to capture the sample 202. For example, sensors may have different levels of acuity or may be placed under cover glasses of different thickness, refractive index, or other properties that may change (or distort) the captured sample 202 relative to other sensors used to capture other samples. Furthermore, different instances of sensors of the same model may have different characteristics due to manufacturing variations (e.g., in alignment, sensor thickness, glass cover thickness, etc.) and calibration differences resulting therefrom. Still further, some users may cover sensors used to capture the sample 202 with a protective film or otherwise obstruct the sensor (e.g., from dirt, grime, etc.) that may affect the image captured by the sensor.

[0021]

[0030] In general, anti-spoofing protection models determine independently for each query whether the query is from a genuine or fake biometric data source. These anti-spoofing protection models may not consider contextual information such as (but not limited to) information about the current user, information about the device, or a history of attempts to access a protected computing resource using biometric authentication. Thus, in real-life deployments, biometric data samples generally have temporal correlations that can be used to inform predictions of whether biometric data captured for use in an attempt to access a protected computing resource is from a genuine or fake source, but the anti-spoofing protection models may not learn from previous misclassifications of biometric authentication attempts.

[0022]

[0031] For example, it can be observed that consecutive samples, especially those close to each other in time, tend to be similar. That is, for a set of n consecutive samples, the conditions under which these samples are captured are likely to be similar. Thus, in an anti-spoofing context, each of these n samples is likely to be all from a genuine source or all from a fake source. Similarly, with respect to the fidelity of the captured biometric data, the conditions at the sensor that captured the biometric data and the biometric data source itself are likely to remain the same or similar. Because past information may have some correlation with the current information used by the biometric authentication system and the anti-spoofing protection model, aspects of the present disclosure leverage this correlation to improve the accuracy of the anti-spoofing protection model and customize the anti-spoofing protection model for a particular device and user.

[0023]

[0032] FIG. 3 illustrates the use of current and past biometric authentication data inputs in a biometric authentication pipeline, according to an embodiment of the present disclosure.

[0024]

[0033] In this example, past authentication attempts 310, 312, 314, and 316, as well as a current attempt 318, may be input into an anti-spoofing protection model 320. One or more of the past authentication attempts 310, 312, 314, and 316 may include historical information that may have some correlation with the current attempt 318. For example, if the past authentication attempts 310, 312, 314, and 316 are close in time to the current attempt 318, the conditions at the sensor(s) used to capture the biometric data and the conditions of the biometric data source (e.g., dry skin, oily skin, etc.) may be assumed to be similar across the past authentication attempts 310, 312, 314, and 316, as well as the current attempt 318. Furthermore, it may be assumed that the same biometric data source is used in each of the past authentication attempts 310, 312, 314, and 316, as well as the current attempt 318. Thus, the anti-spoofing protection model 320 can generate predictions with improved accuracy by considering the similarities between the data used in past and current authentication attempts.

[0025]

[0034] In various aspects, these assumptions may be context-specific: for example, these assumptions may be valid for biometric authentication on a mobile device used by a single user, but may not be valid for a public biometric scanner that captures a variety of biometric data from multiple biometric data sources over a short period of time.

[0026] An exemplary online adaptive personalization of anti-spoofing protection models in biometric authentication systems

[0035] Further improvement in the accuracy of anti-spoofing protection models can be achieved through on-device (or online) adaptive personalization of such models, as shown in FIG.

[0027]

[0036] FIG. 4 illustrates an anti-spoofing protection pipeline 400. In the anti-spoofing protection pipeline 400, a sample 410 captured by a biometric data capture device (e.g., an ultrasonic sensor, a light sensor, a camera, etc.) may be provided as an input to an anti-spoofing protection model 420. The anti-spoofing protection model 420 may be generally trained based on a predefined training data set to determine whether the captured sample 410 is from a genuine source or a fake source. For example, in a fingerprint authentication system, the anti-spoofing protection model 420 may make a decision whether the source of the sample 410 is a live source (e.g., a user's finger) or a spoof source (e.g., a replica of the user's finger). A prediction 430 generated by the anti-spoofing protection model 420 may then be used to determine whether to allow the user access to a protected computing resource. In general, when the prediction 430 indicates that the source of the sample 410 is likely to be a live source, the biometric authentication system may allow access to a protected computing resource if the sample 410 matches an enrolled sample. In contrast, when the prediction 430 indicates that the source of the sample 410 is likely to be a spoof or inauthentic source, the biometric authentication system may block access to a protected computing resource regardless of whether the sample 410 matches an enrolled sample.

[0028]

[0037] The anti-spoofing protection models 420 may include a first model that extracts features from a captured sample 410 and a second model that generates predictions 430 from the features extracted from the sample 410. The first model may include, for example, convolutional neural networks (CNNs), transformer neural networks, recurrent neural networks (RNNs), or any of a variety of other suitable artificial neural networks or other machine learning models that may be used to extract features from a sample or a representation thereof. The second model may include a variety of probabilistic or predictive models that may predict whether the sample 410 is from an authentic or inauthentic (biometric data) source.

[0029]

[0038] To personalize the anti-spoofing protection model 420, the online adaptive personalization module 440 may use the predictions 430 generated by the anti-spoofing protection model 420 for the samples 410 to generate a fine-tuning dataset D for adjusting (e.g., retraining) the anti-spoofing protection model 420. In some aspects, as described in more detail below, the fine-tuning dataset D may be initialized as a null set, and the samples may be added to the fine-tuning dataset D.

[0030]

[0039] In some aspects, the prediction 430 may be a prediction score or other score between a defined lower limit value and a defined upper limit value. The lower limit value may be associated with a classification of the sample as obtained from an inauthentic source, and the upper limit value may be associated with a classification of the sample as obtained from an authentic source. Values ​​above the threshold level may be associated with an authentic source classification, and in a labeling stage 442, the sample 410 may be labeled with an indication that the sample 410 is from an authentic source. On the other hand, values ​​below the threshold level may be associated with an inauthentic source classification, and in a labeling stage 442, the sample 410 may be labeled with an indication that the sample 410 is from an inauthentic source (e.g., a replica of a user's finger, an image or a three-dimensional model of a user's face, etc.). In other aspects, only one of the authentic sample or inauthentic source may be so labeled.

[0031]

[0040] In a fine-tuning dataset generation stage 444, it may be determined whether to add the labeled samples generated in the labeling stage 442 to a fine-tuning dataset 446 for use in retraining and refining the anti-spoofing protection model 420. In some aspects, each captured sample may be added to the fine-tuning dataset 446 for use in retraining and refining the anti-spoofing protection model 420. However, adding each captured sample to the fine-tuning dataset 446 may introduce samples into the fine-tuning dataset 446 whose classification may be inaccurate or uncertain. For example, assuming a range of predicted scores between 0 and 1, adding samples with scores near the middle (e.g., within a threshold range from 0.5) to the fine-tuning dataset 446 may add samples with labels (or classifications) that may actually be somewhat uncertain, and thus retraining and refining the anti-spoofing protection model 420 based on such data may adversely affect the accuracy of predictions made by the anti-spoofing protection model 420.

[0032]

[0041] Thus, in some aspects, the fine-tuning dataset generation stage 444 can ensure that the fine-tuning dataset 446 includes data on which classification can be relied upon with a certain degree of confidence. To do so, the prediction 430 can be compared to at least one threshold score, such as a first threshold score and a second threshold score. The first threshold score can be, for example, a maximum score for samples classified as samples from an inauthentic source, and the second threshold score can be a minimum score for samples classified as samples from an authentic source. As shown in example 610 of FIG. 6 and described in more detail below, if the prediction 430 is below the first threshold score or above the second threshold score, the labeled sample 410 can be added to the fine-tuning dataset 446. Otherwise, if the prediction 430 is between the first and second threshold scores, the prediction 430 can be considered sufficiently uncertain that the sample 410 may not be a good sample to add to the fine-tuning dataset 446.

[0033]

[0042] In some aspects, the fine-tuning dataset generation stage 444 can use smoothing techniques to improve the consistency of the labels associated with samples in the fine-tuning dataset 446. For example, the smoothing technique can be implemented within a sliding time window (e.g., as described in more detail below with respect to FIG. 7). For example, the label for a sample at time t over a sliding time window of duration W may be

[0034]

number

[0035] may be applied according to the following formula:

[0036]

number

[0037] where i represents the i-th sample in a time window centered at time t. The duration of W may be selected to enable the anti-spoofing protection model 420 to respond to rapid transitions between genuine access attempts and spoofing attacks.

[0038]

[0043] In a model adjustment stage 448, the anti-spoofing protection model 420 may be retrained and refined based on the fine-tuning dataset 446. In some aspects, the anti-spoofing protection model 420 may be retrained and refined periodically (e.g., after m samples are added to the fine-tuning dataset, after some defined amount of time, upon system reboot, after running one or more applications some defined number of times, etc.).

[0039]

[0044] In some aspects, if the anti-spoofing protection model 420 is a deep learning model (e.g., a deep neural network or other neural network), the retraining and refinement of the anti-spoofing protection model 420 may be performed in some aspects as several iterations of mini-batch gradient descent that seeks to optimize cross-entropy as an objective function, where a mini-batch comprises data sampled from the fine-tuning dataset 446. The cross-entropy loss that is optimized during the mini-batch gradient descent may be represented by the following equation:

[0040]

number

[0041] where y i corresponds to the prediction generated by the anti-spoofing protection model 420,

[0042]

number

[0043] corresponds to the label assigned to sample i in the fine-tuning dataset 446. Depending on the type of anti-spoofing protection model 420 (e.g., whether the anti-spoofing protection model 420 is a support vector machine, a random tree, etc.), other update techniques may be used in some cases.

[0044]

[0045] In some aspects, as described in further detail below, the anti-spoofing protection model 420 may be retrained, for example, by weighting the data in the fine-tuning data set 446 differently based on various characteristics of each sample in the fine-tuning data set 446.

[0045]

[0046] For example, if the fine-tuning data set 446 includes a pre-training data set of data from inauthentic biometric data sources of different known subjects, sensors, and / or types used in a spoofing attack, and a set of samples captured during operation of the biometric authentication system (also referred to as "online data"), different weights may be applied to the pre-training data set and the set of online data. For example, over time, the weights applied to the pre-training data set may be decreased and the weights applied to the set of online data may be increased to increasingly tune the resulting model to the characteristics of the biometric sensors of the device itself and the characteristics of the user who uses the biometric authentication system to gain access to protected computing resources. The use of the pre-training data set and the set of online data may be used to prevent overfitting problems that may arise from retraining and refining the anti-spoofing protection model 420 based on an unbalanced set of online data that may probabilistically contain significantly more data from authentic biometric sources than inauthentic biometric sources.

[0046]

[0047] In another example, the set of online data may be weighted in time. In general, older samples in the set of online data may be considered less relevant to the user than newer samples in the set of online data, since it may be assumed that the conditions under which the older samples were captured may be different from the conditions under which the newer samples were captured and therefore may not represent the current conditions of the sensor(s) used to capture the biometric data or the source of the biometric data. Thus, it may be assumed that the most recent samples in the set of online data have characteristics that are most similar to the incoming samples used in biometric authentication than the older samples. The older samples may be assigned progressively lower weights to de-emphasize these older samples, for example, when retraining and refining the anti-spoofing protection model 420 in the model adjustment stage 448.

[0047]

[0048] In some aspects, a threshold age may be established to weight (or prune such) samples in the fine-tuning data set 446. Samples of the online data older than the threshold age may be assigned a zero weight (or otherwise pruned) in the model adjustment stage 448, which may effectively remove these samples from consideration in retraining and refining the anti-spoofing protection model 420. Samples newer than the threshold age may be considered in retraining and refining the anti-spoofing protection model 420, and in some aspects may be weighted differentially in the model adjustment stage 448 such that the newest samples are assigned the highest weights and the oldest samples that are still newer than the threshold age are assigned the lowest weights.

[0048]

[0049] In some aspects, the data in the fine-tuning data set 446 may be an unbalanced data set that includes a significantly greater number of genuine biometric data inputs than ingenuine biometric data inputs. To avoid a situation where an unrepresentative data set is used to tune the anti-spoofing protection model 420, the samples in the fine-tuning data set 446 selected to tune the anti-spoofing protection model 420 may reflect the distribution of genuine and ingenuine biometric data inputs identified in a real-world deployment of the anti-spoofing protection model 420.

[0049]

[0050] In some aspects, various techniques may be used to normalize the anti-spoofing protection model 420 and avoid a situation where the anti-spoofing protection model 420 overfits the fine-tuning dataset 446 (e.g., the anti-spoofing protection model 420 fits the fine-tuning dataset 446 but provides insufficient inference accuracy for data outside the fine-tuning dataset 446). To do so, the anti-spoofing protection model 420 may be periodically reset to an initial state. For example, weights in the anti-spoofing protection model may be reset to weights established when the anti-spoofing protection model was initially trained based on a pre-training dataset of data from different known targets, sensors, and / or types of inauthentic biometric data sources used in spoofing attacks. In another example, parameter updates may be constrained by a limited learning rate or through the use of various optimization constraints. Furthermore, only a portion of the anti-spoofing protection model may be updated during the model adjustment stage 448.

[0050]

[0051] In some aspects, the anti-spoofing protection model 420 includes a feature extractor φ that extracts features from the incoming samples 410. f and a classifier φ that generates predictions 430 c The feature f of the tth sample x can be expressed as t is the formula f t =φ f (xt ) and the classification y of sample x can be represented by the formula y t =φ c (f t )=φ c (φ f (x t )) can be expressed as f t is the input sample x t (e.g., sample 410). During the model training stage 448, in some embodiments, φ f may remain static, and φ c may be retrained based on the fine-tuning dataset 446. c Since φ may represent only a part of the neural network (e.g., the final layer of the neural network), c Retraining and refining x may be a computationally inexpensive process compared to training the entire anti-spoofing protection model 420. Furthermore, the data in the fine-tuning dataset 446 may be t Input x, not itself t The extracted features of f t , the size of the fine-tuned data set 446 can be minimized and the privacy of sensitive input data that can be used to generate a data source for a spoofing attack can be maintained.

[0051] Exemplary method for online adaptive personalization of anti-spoofing protection models in biometric authentication systems

[0052] FIG. 5 illustrates example operations 500 that may be performed to authenticate biometric data based on a fine-tuning data set generated from captured biometric data and adjust an anti-spoofing protection model for biometric authentication (e.g., as shown in FIG. 4 and described above) in accordance with some aspects of the disclosure.

[0052]

[0053] As shown, operation 500 begins at block 510, where a biometric data input (e.g., sample 410 shown in FIG. 4) is received for a user to authenticate the user. The biometric data input may include, for example, but not limited to, an image of a fingerprint, an image of the user's face, an image of the user's iris, etc. In some aspects, the biometric data input may include two-dimensional data or three-dimensional data (e.g., with depth) that characterizes a biometric data source to be used in authenticating the user and controlling access to protected computing resources. In some aspects, the received image may be an image in a binary color space where a first color represents a surface and a second color represents a transition between different surfaces. For example, the first color may represent a valley in a fingerprint and the second color may represent a transition from the valley to the ridge in a fingerprint. In some aspects, the received image may be an image in a low bit depth monochrome color space where a first color represents a first type of characteristic in the biometric data input, a second color represents a second type of characteristic in the biometric data input, and colors between the first color and the second color represent transitions between the first type of characteristic and the second type of characteristic. In yet another example, the biometric data input may include other data that can be used in determining whether the biometric data is from an authentic or inauthentic source. The biometric data input may include (without limitation) video, thermal data, depth maps, and / or other information that can be used to authenticate a user and determine whether the user's biometric data input is from an authentic or inauthentic source.

[0053]

[0054] At block 520, features of the received biometric data input are extracted through a first machine learning model. The first machine learning model may include, for example, convolutional neural networks (CNNs), transformer neural networks, recurrent neural networks (RNNs), or any of a variety of other suitable artificial neural networks or other machine learning models that may be used to extract features from images or representations thereof. Features may be extracted for the received images and images in the enrollment image set using neural networks using different weights or using the same weights. In some aspects, features may be extracted a priori for images in the enrollment image set (e.g., when a user enrolls a biometric data source, such as a finger, face, or iris, for use in biometric authentication). In other aspects, features of images in the enrollment image set may be extracted based on non-image representations of images in the enrollment image set as the user attempts to authenticate through the biometric authentication pipeline.

[0054]

[0055] In block 530, using the extracted features of the received biometric data input and the second machine learning model, it is determined whether the received biometric data input of the user is authentic or inauthentic (e.g., whether the input is sourced from a real finger, face, iris, etc., or whether the input is sourced from a replica of a finger, face, iris, etc.). This determination may be based on a prediction score generated by the second machine learning model, such as, for example, the prediction 430 generated by the anti-spoofing protection model 420 shown in FIG. 4. In some aspects, the inauthentic input may also include synthetic images of the biometric data source captured from different data sources, and / or synthetically generated and improved biometric data inputs, or biometric data inputs (e.g., from a collection of fingerprints) designed to match many users of the biometric authentication system. In some aspects, the system may determine whether the received biometric data input of the user is authentic or inauthentic using various types of neural networks that can use various features extracted from the biometric data input and other contextual information to determine whether the received biometric data input is authentic or inauthentic. In general, the decision may be made based on a predicted score or other score generated by the second machine learning model. If the predicted score or other score exceeds a threshold, the received biometric data input may be deemed authentic. If not, the received data input may be deemed inauthentic.

[0055]

[0056] In some aspects, the extracted features of the received biometric data input may include (but are not limited to) features from video, thermal data, depth maps, or other information that may be used in determining whether the received biometric data input is from an authentic or inauthentic source. For example, the features extracted from the video input may indicate the degree or amount of movement in the biometric data input. The degree of subject movement across frames in the received biometric data input may be a data point that indicates the biometric data input is from an authentic source, and the lack of subject movement across frames in the received biometric data input may be a data point that indicates the biometric data input is from an authentic source. In another example, the features extracted from the received biometric data input may correspond to thermal data captured for the biometric data source, with certain temperature ranges corresponding to biometric data sources that are more likely to be authentic and other temperature ranges corresponding to biometric data sources that are less likely to be authentic. In yet another aspect, if the received biometric data input includes data from a depth map, extracted features of the depth data from the depth map may be used in determining whether the received biometric data input is authentic or inauthentic based on the assumption that depth data that differs significantly from the depth data included in the data in the enrollment data set may correspond to biometric data input received from an inauthentic source.

[0056]

[0057] In block 540, it is determined whether to add the extracted features of the received biometric data input (which in some aspects may be labeled with an indication of whether the received biometric data input is authentic or inauthentic) to a fine-tuning dataset (e.g., fine-tuning dataset 446 shown in FIG. 4). In some aspects, the biometric data input may be added to the fine-tuning dataset regardless of the predicted score or other score generated for the biometric data input. In some aspects (such as in example 610 shown in FIG. 6 and described below), if the predicted score of the biometric data input is deemed strong enough to have a high confidence that the received biometric data input is labeled as authentic or inauthentic, the biometric data input may be added to the fine-tuning dataset. A first threshold score (e.g., first threshold 612 shown in FIG. 6) corresponding to a maximum predicted score of an inauthentic biometric input and a second threshold score (e.g., second threshold 614 shown in FIG. 6) corresponding to a minimum predicted score of an authentic biometric input may be established. If the prediction score of the received biometric data input is less than the first threshold score or greater than the second threshold score, the received biometric data input may be added to the fine-tuning data set. Otherwise, the prediction of the received biometric data input may be deemed not to be strong enough to justify adding the received biometric data input to the fine-tuning data set.

[0057]

[0058] At block 550, the second machine learning model is adjusted based on the fine-tuning data set. As described, adjusting the machine learning model may include retraining one or more layers in the neural network based on the fine-tuning data set with data from the fine-tuning data set weighted to prevent overfitting and weight recent biometric data inputs more heavily than older biometric data inputs. The adjusted model may then be used in future predictions of whether a received biometric data input is authentic or inauthentic.

[0058] Exemplary Generation and Weighting of Fine-Tuning Data Sets for Tuning Anti-Spoofing Protection Models

[0059] 6 illustrates an example thresholding technique for adding captured biometric data to a fine-tuning data set that adjusts an anti-spoofing protection model, according to an aspect of the present disclosure. These thresholding techniques may be used, for example, to generate the fine-tuning data set 446 illustrated in FIG. 4, as described above with respect to block 540 illustrated in FIG. 5.

[0059]

[0060] As shown in example 600, a single threshold value (t spoof ) 602 may be established. As shown, the predicted scores generated by the anti-spoofing protection model may be scaled to a single threshold t spoof If it is less than 602, the received biometric data input may be labeled with a genuine label and added to the fine-tuning dataset. Otherwise, the predicted score generated by the anti-spoofing protection model may be adjusted to a single threshold t spoof 602, the received biometric data input may then be labeled with an inauthentic label and added to the fine-tuning data set.

[0060]

[0061] As explained above, regardless of the strength of the prediction score associated with each received biometric data input, adding each received biometric data input to the fine-tuning dataset may result in a fine-tuning dataset that includes samples of the biometric data input, where confidence in the accuracy of the labels associated with these samples may be low. To improve the quality of the data in the fine-tuning dataset, as shown in example 610, two thresholds 612, 614 may be established to determine whether to add a received biometric data input to the fine-tuning dataset. The thresholds 612(t live ) may be, for example, the maximum predicted score of a received biometric data input that is classified as a genuine input that may be added to the fine-tuning data set, and the threshold 614(t spoof ) may be the minimum predicted score for a received biometric data input that will be classified as an inauthentic input that may be added to the fine-tuning data set. If the received biometric data input has a score between threshold 612 and threshold 614, then confidence that the received biometric data input will be correctly classified may be insufficient to justify adding the received biometric data input to the fine-tuning data set.

[0061]

[0062] In some aspects, the thresholds 602, 612, and 614 may be optimized for a calibration data set according to a target false positive rate and a target false negative rate. To do so, an anti-spoofing protection model, such as the anti-spoofing protection model 420 shown in FIG. 4, may be trained using biometric data inputs that have scores according to a first set of thresholds. If the anti-spoofing protection model produces a false positive rate or false negative rate that exceeds the target false positive rate or false negative rate, the thresholds may be adjusted to include biometric data inputs with stronger predictive scores that indicate a higher likelihood that the biometric data input is authentic or inauthentic.

[0062]

[0063] 7 illustrates an example alignment of labels for captured biometric data based on labels assigned to other captured biometric data, according to an embodiment of the present disclosure. These alignment techniques may be used, for example, to generate or correct the fine-tuning data set 446 shown in FIG. 4, as described above with respect to block 540 shown in FIG. 5.

[0063]

[0064] As shown, in example 700, several inputs 702, 704, 706, 708, and 710 may be received. Inputs 702, 704, 708, and 710 may initially be classified as genuine biometric data inputs, and input 706 may be classified as an inauthentic biometric data input. However, because an inauthentic biometric data source is unlikely to be used to generate a biometric data input that is close in time to a biometric data input generated using an authentic data source (e.g., corresponding to inputs 702, 704, 706, 708, and 710), contextual information associated with timing and sequence information of inputs 702, 704, 708, and 710 may indicate that input 706 is in fact a genuine biometric input. Thus, as shown, in example 750, the classification of input 706 may be changed such that label 712 of input 706 corresponds to a genuine classification rather than an inauthentic classification.

[0064]

[0065] Various techniques may be used to correct the classifications assigned to the biometric data entries in the fine-tuning data set. As shown in Figure 7, one technique for correcting the classifications assigned to the biometric data entries may include using information about successive samples to determine an appropriate classification for the biometric data entries in the fine-tuning data set.

[0065]

[0066] In another example, time windowing can be used to determine an appropriate classification of a biometric data input within the time window. In yet another example, an appropriate classification of a biometric data input can be determined and generated based on classifications of other biometric data inputs with similar features. In this example, a set of biometric data inputs similar to a target biometric data input can be identified based on the distance between the target biometric data input and the other biometric data inputs in the feature space. The set of biometric data inputs used to correct the classification assigned to the target biometric data input can be biometric data inputs in the fine-tuning data set that are less than a threshold distance away from the target biometric data input.

[0066]

[0067] Correcting the label assigned to a biometric data input can be based on various selection techniques. In one example, a majority voting scheme can be used to select the correct label for a group of biometric data inputs. As shown in FIG. 7, for example, it can be seen that four samples correspond to predictions of genuine biometric data inputs and one sample (input 706) corresponds to predictions of inauthentic biometric data inputs. Since the majority of the samples in example 700 are predicted to be genuine biometric data inputs, a majority voting scheme can cause the label assigned to input 706 to be changed from an inauthentic label to a genuine label (e.g., as shown in example 750).

[0067]

[0068] In another example, a weighted average can be used to correct the labels assigned to samples in the fine-tuning dataset. To correct the labels for samples in the fine-tuning dataset, a weight can be assigned to each biometric data input in the group of inputs based on, for example, its temporal proximity to the sample to be corrected, the order in which the sample is located in the fine-tuning dataset relative to the sample to be corrected, feature space information, etc. As an example, weights can be applied such that samples closer to each other in time have higher weights. For example, n When correcting the label 712 assigned to the input 710 at time t n+2 The weights assigned to the inputs 708 at time t n+1 , may be greater than the weight assigned to input 706 in , and so on. The weighted average score may be used to determine the correct classification of each biometric data input in the group. Of course, it should be recognized that these are only a few examples of techniques that may be used to correct the labels assigned to biometric data inputs in the fine-tuning data set, and that other interpolation techniques may also or alternatively be used.

[0068]

[0069] 8 illustrates an example weighting of captured biometric data in a fine-tuning data set for adjusting an anti-spoofing protection model, according to an embodiment of the present disclosure. These weighting techniques may be used, for example, to generate the fine-tuning data set 446 illustrated in FIG. 4, as described above with respect to block 540 illustrated in FIG. 5.

[0069]

[0070] As shown, in the example 800, indexes t0 to t n+2A set of samples with ages greater than a threshold age may be present in the fine-tuning dataset. Samples older than a threshold age may be excluded from use in the fine-tuning dataset. For example, it can be seen that samples 802 and 804 corresponding to samples captured at times t0 and t1 may be excluded from the fine-tuning dataset (e.g., deleted, assigned a zero weight, etc.) because these samples may be the oldest samples in the fine-tuning dataset and may have at most an attenuated level of correspondence or relevance to the current biometric data input processed through the anti-spoofing protection model, or may be assigned a weight lower than the weights assigned to other samples in the fine-tuning dataset. On the other hand, samples 802 and 804 corresponding to samples captured at times t0 and t1 may be excluded from the fine-tuning dataset (e.g., deleted, assigned a zero weight, etc.) because these samples may be the oldest samples in the fine-tuning dataset and may have at most an attenuated level of correspondence or relevance to the current biometric data input processed through the anti-spoofing protection model, or may be assigned a weight lower than the weights assigned to other samples in the fine-tuning dataset. n , t n+1 , and t n+2 Samples 812, 814, and 816 corresponding to samples captured at may be included in the fine-tuned dataset. These samples may be assigned weights corresponding to their relative freshness in the fine-tuned dataset (e.g., sample 812 may be assigned the highest weight and sample 816 may be assigned the lowest weight, since sample 816 is the oldest sample and sample 812 is the newest sample).

[0070]

[0071] By differentially weighting samples in a fine-tuning dataset used to retrain and refine an anti-spoofing protection model, aspects of the present disclosure may thus enable the anti-spoofing protection model to adapt to evolving biometric sources and environmental conditions over time, which may improve the accuracy of determining whether a biometric data input is captured from an authentic or non-authentic source.

[0071] Exemplary Processing System for Fingerprint Authentication Using Machine Learning-Based Anti-Spoofing Protection

[0072] FIG. 9 depicts an example processing system 900 that authenticates biometric data and adjusts an anti-spoofing protection model for biometric authentication based on a fine-tuning data set generated from the captured biometric data, e.g., as described herein with respect to FIGS. 4 and 5.

[0072]

[0073] The processing system 900 includes a central processing unit (CPU) 902, which in some examples may be a multi-core CPU. Instructions executed by the CPU 902 may be loaded, for example, from a program memory associated with the CPU 902 or from a partition in memory 924.

[0073]

[0074] The processing system 900 also includes additional processing components tailored to specific functions, such as a graphics processing unit (GPU) 904, a digital signal processor (DSP) 906, a neural processing unit (NPU) 908, a multimedia processing unit 910, and wireless connectivity components 912.

[0074]

[0075] An NPU, such as the NPU 908, is generally a dedicated circuit configured to perform control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU may alternatively be referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graph processing unit.

[0075]

[0076] NPUs, such as NPU 908, are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, an NPU may be part of a dedicated neural network accelerator.

[0076]

[0077] NPUs can be optimized for training or inference, or in some cases can be configured to balance performance between both. For NPUs capable of performing both training and inference, the two tasks can still generally be performed independently.

[0077]

[0078] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters, such as weights and biases, to improve model performance. In general, optimization based on mispredictions involves backpropagating through layers of the model to determine gradients to reduce prediction errors.

[0078]

[0079] NPUs designed to accelerate inference are generally configured to operate on complete models, and therefore may be configured to input new data and rapidly process this new data through models that have already been trained to produce model outputs (e.g., inferences).

[0079]

[0080] In one implementation, the NPU 908 is part of one or more of the CPU 902, the GPU 904, and / or the DSP 906.

[0080]

[0081] In some examples, the wireless connectivity component 912 may include sub-components for, for example, third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. The wireless connectivity component 912 is further connected to one or more antennas 914.

[0081]

[0082] The processing system 900 may also include one or more sensor processing units 916 associated with any type of biometric sensor (e.g., imaging sensors used to capture images of a biometric data source, ultrasonic sensors, depth sensors used to generate a three-dimensional map of biometric features, etc.), one or more image signal processors (ISPs) 918 associated with any type of image sensor, and / or a navigation processor 920, which may include satellite-based positioning system components (e.g., GPS or GLONASS), as well as inertial positioning system components.

[0082]

[0083] The processing system 900 may also include one or more input and / or output devices 922, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, a speaker, a microphone, etc.

[0083]

[0084] In some examples, one or more of the processors of processing system 900 may be based on the ARM or RISC-V instruction set.

[0084]

[0085] Processing system 900 also includes memory 924, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, memory 924 includes computer-executable components capable of being executed by one or more of the aforementioned processors of processing system 900.

[0085]

[0086] In particular, in this example, memory 924 includes a biometric data input receiving component 924A, an image feature extraction component 924B, a biometric data input authenticity determination component 924C, a fine-tuning dataset addition component 924D, and a model adjustment component 924E. The illustrated components, and other components not illustrated, may be configured to perform various aspects of the methods described herein.

[0086]

[0087] In general, the processing system 900 and / or its components may be configured to perform the methods described herein.

[0087]

[0088] In particular, in other aspects elements of processing system 900 may be omitted, such as when processing system 900 is a server computer, etc. For example, multimedia processing unit 910, wireless connectivity component 912, ISP 918, and / or navigation processor 920 may be omitted in other aspects. Additionally, elements of processing system 900, such as training models and using models to generate inferences, such as user authentication predictions, may be distributed.

[0088] Example clauses

[0089] Details of the implementation of various aspects of the disclosure are set forth in the following numbered clauses.

[0089]

[0090] Clause 1: A method comprising: receiving a biometric data input of a user; extracting features of the received biometric data input through a first machine learning model; determining whether the received biometric data input of the user is authentic or inauthentic using the extracted features of the received biometric data input and a second machine learning model; determining whether to add the extracted features of the received biometric data input to a fine-tuning dataset; and tuning the second machine learning model based on the fine-tuning dataset.

[0090]

[0091] Clause 2: The method of clause 1, wherein determining whether to add a feature of the received biometric data input to the fine-tuning dataset includes determining whether to add the feature and a label associated with the feature based on whether the received biometric data input of the user is authentic or inauthentic.

[0091]

[0092] Clause 3: The method of clause 2, wherein determining whether to add the feature and the label associated with the feature based on whether the user's received biometric data input is authentic or inauthentic includes one of adding the feature and the label associated with the feature to both the authentic and inauthentic received biometric data input, adding the feature and the label associated with the feature only when the user's received biometric data input is authentic, or adding the feature and the label associated with the feature only when the user's received biometric data input is inauthentic.

[0092]

[0093] Clause 4: A method according to any one of clauses 1 to 3, wherein determining whether the user's received biometric data input is authentic or inauthentic includes generating a predictive score corresponding to the likelihood that the user's received biometric data input is from a genuine biometric data source.

[0093]

[0094] Clause 5: The method of clause 4, wherein determining whether to add the extracted feature of the received biometric data input to the fine-tuning dataset includes determining that the predicted score is greater than a first threshold or less than a second threshold, and adding the extracted feature of the received biometric data input to the fine-tuning dataset based on determining that the predicted score is greater than the first threshold or less than the second threshold.

[0094]

[0095] Clause 6: The method of clause 5, wherein the first threshold comprises a threshold of biometric data input that is more likely to correspond to data from a genuine biometric source, and the second threshold comprises a threshold of biometric data input that is more likely to correspond to data from an inauthentic biometric source.

[0095]

[0096] Clause 7: The method of any one of clauses 4 to 6, wherein determining whether to add an extracted feature of the received biometric data input to the fine-tuning dataset comprises: adding an extracted feature labeled with an indication that the feature corresponds to data from a genuine biometric source based on determining that the prediction score exceeds a threshold; and adding an extracted feature labeled with an indication that the feature corresponds to data from a non-genuine biometric source based on determining that the prediction score is below the threshold.

[0096]

[0097] Clause 8: The method of any one of clauses 1 to 7, further comprising determining that a label assigned to the extracted feature of the received biometric data input is different from other biometric data inputs received within a threshold time of the received biometric data input, and modifying the label assigned to the extracted feature of the received biometric data input based on the label assigned to the other biometric data input.

[0097]

[0098] Clause 9: The method of any one of clauses 1 to 8, further comprising determining that a label assigned to the extracted feature of the received biometric data input is different from a label assigned to another biometric data input having a similar feature to the extracted feature, and modifying the label assigned to the extracted feature of the received biometric data input based on the label assigned to the other biometric data input having the similar feature.

[0098]

[0099] Clause 10: The method of any one of clauses 1 to 9, wherein adjusting the second machine learning model based on the fine-tuning dataset includes applying weights to the fine-tuning dataset proportional to the temporal age of each sample in the fine-tuning dataset.

[0099]

[0100] Clause 11: The method of clause 10, wherein applying weights to the fine-tuning dataset includes assigning a zero weight to samples in the fine-tuning dataset that are older than a threshold age.

[0100]

[0101] Clause 12: The method of any one of clauses 1 to 11, wherein the fine-tuning dataset includes a pre-training dataset and an online training dataset, and determining whether to add extracted features of the received biometric data input to the fine-tuning dataset includes determining whether to add extracted features of the biometric data input to the online training dataset.

[0101]

[0102] Clause 13: The method of clause 12, wherein adjusting the second machine learning model based on the fine-tuning dataset includes adjusting the second machine learning model based on first weights assigned to the pre-training dataset and second weights assigned to the online training dataset.

[0102]

[0103] Clause 14: A processing system comprising a memory containing computer-executable instructions and one or more processors configured to execute the computer-executable instructions to cause the processing system to perform a method according to any of clauses 1 to 13.

[0103]

[0104] Clause 15: A processing system comprising means for carrying out a method according to any of clauses 1 to 13.

[0104]

[0105] Clause 16: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method according to any of clauses 1 to 13.

[0105]

[0106] Clause 15: A computer program product embodied on a computer-readable storage medium comprising code for performing the method according to any of clauses 1 to 11.

[0106] Additional Considerations

[0107] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples described herein are not intended to limit the scope, applicability, or aspects described in the claims. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, changes may be made in the function and arrangement of the elements described without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects described herein. Furthermore, the scope of the disclosure is intended to encompass such apparatus or methods that are implemented using other structures, functions, or structures and functions in addition to or other than the various aspects of the disclosure described herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0107]

[0108] As used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

[0108]

[0109] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. By way of example, "at least one of a, b, or c" is intended to encompass a, b, c, ab, ac, bc, and abc, as well as any combination having multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other permutation of a, b, and c).

[0109]

[0110] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, database, or another data structure), ascertaining, and the like. "Determining" may also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. "Determining" may also include resolving, selecting, choosing, establishing, and the like.

[0110]

[0111] The methods disclosed herein include one or more steps or actions for achieving the method. The steps and / or actions of those methods may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order of specific steps and / or actions and / or the use of those steps and / or actions may be modified without departing from the scope of the claims. Furthermore, various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions. Those means may include various hardware and / or software component(s) including, but not limited to, circuits, application specific integrated circuits (ASICs), or processors, and / or various hardware and / or software module(s). In general, when operations are illustrated in figures, those operations may have corresponding equivalent means-plus-function components that are similarly numbered.

[0111]

[0112] The following claims are not intended to be limited to the embodiments set forth herein, but are to be accorded the full scope consistent with the language of the claims. Within the claims, reference to an element in the singular is not intended to mean "only one" unless expressly stated as such, but rather "one or more." Unless expressly stated otherwise, the term "several" refers to one or more. No element of a claim is to be construed under the provisions of 35 U.S.C. 112(f) unless the element is expressly recited using the phrase "means for" or, in the case of a method claim, unless the element is recited using the phrase "step for." All structural and functional equivalents of the elements of the various embodiments described throughout this disclosure that are known or later become known to those of skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Furthermore, nothing disclosed herein is intended to be made public, regardless of whether such disclosure is expressly recited in the claims.

Claims

1. 1. A processor-implemented method, comprising: receiving a user biometric data input from a sensor; extracting features of the received biometric data input through a first machine learning model; and using the features of the received biometric data input and a second machine learning model to determine whether the received biometric data input of the user is authentic or inauthentic; determining whether to add the feature of the received biometric data input to a fine-tuning dataset; tuning the second machine learning model based on the fine-tuning dataset; A method comprising:

2. 2. The method of claim 1, wherein determining whether to add the feature of the received biometric data input to the fine-tuning data set comprises determining whether to add the feature and a label associated with the feature based on whether the received biometric data input of the user is authentic or inauthentic.

3. determining whether to add the feature and a label associated with the feature based on whether the received biometric data input of the user is authentic or inauthentic; adding said features and labels associated with said features to both authentic and inauthentic received biometric data inputs; adding said feature and said label associated with said feature only if said received biometric data input of said user is authentic; or adding said feature and said label associated with said feature only if said received biometric data input of said user is not authentic; The method of claim 2 , comprising one of:

4. 2. The method of claim 1, wherein determining whether the received biometric data input of the user is authentic or inauthentic comprises generating a predictive score corresponding to the likelihood that the received biometric data input of the user is from a genuine biometric data source.

5. determining whether to add the feature of the received biometric data input to the fine-tuning data set; determining that the predicted score is above a first threshold or below a second threshold; adding the feature of the received biometric data input to the fine-tuning data set based on the determining that the prediction score is above the first threshold or below the second threshold; The method of claim 4, comprising:

6. 6. The method of claim 5, wherein the first threshold comprises a threshold of biometric data input that is likely to correspond to data from a genuine biometric source, and the second threshold comprises a threshold of biometric data input that is likely to correspond to data from an inauthentic biometric source.

7. determining whether to add the feature of the received biometric data input to the fine-tuning data set; adding the feature labeled with an indication that the feature corresponds to data from an authentic biometric source based on determining that the prediction score exceeds a threshold; adding the feature labeled with an indication that the feature corresponds to data from an inauthentic biometric source based on determining that the prediction score is less than the threshold; and The method of claim 4, comprising:

8. determining that a label assigned to the feature of the received biometric data input is different from another biometric data input received within a threshold time from the received biometric data input; modifying the labels assigned to the characteristics of the received biometric data input based on labels assigned to the other biometric data inputs; The method of claim 1 further comprising:

9. determining that a label assigned to the feature of the received biometric data input is different from labels assigned to other biometric data inputs having similar features; modifying the labels assigned to the characteristics of the received biometric data input based on the labels assigned to the other biometric data inputs having the similar characteristics; The method of claim 1 further comprising:

10. 2. The method of claim 1, wherein adjusting the second machine learning model based on the fine-tuning dataset comprises applying weights to the fine-tuning dataset proportional to the temporal age of each sample in the fine-tuning dataset.

11. The method of claim 10 , wherein applying the weights to the fine-tuning dataset comprises assigning a zero weight to samples in the fine-tuning dataset that are older than a threshold age.

12. the fine-tuning dataset comprises a pre-training dataset and an online training dataset, and determining whether to add the features of the received biometric data input to the fine-tuning dataset comprises determining whether to add the features of the biometric data input to the online training dataset; 2. The method of claim 1 , optionally wherein adjusting the second machine learning model based on the fine-tuning dataset comprises adjusting the second machine learning model based on first weights assigned to the pre-training dataset and second weights assigned to the online training dataset.

13. a memory comprising computer-executable instructions; a processor configured to execute the computer-executable instructions; and A system comprising: The processor may include: receiving a user's biometric data input; extracting features of the received biometric data input through a first machine learning model; determining, using the features of the received biometric data input and a second machine learning model; determining whether to add the features of the received biometric data input labeled with an indication of whether the received biometric data input is authentic or inauthentic to a fine-tuning data set; tuning the second machine learning model based on the fine-tuning dataset; configured to execute the computer-executable instructions for system.

14. The system of claim 13, further configured to perform a method according to any one of claims 2 to 12.

15. 13. A computer readable medium having stored thereon instructions that, when executed by a processor, perform operations including carrying out the method of any one of claims 1 to 12.

Images