A fingerprint live detection data blinding method, system and electronic equipment

Abstract

The application discloses a kind of fingerprint living body detection data blind filling method, system and electronic equipment, the method includes: positioning detection blind area sample, extract portrait feature and determine blind area portrait interval, the portrait feature includes ridge line coverage, average brightness and gray standard deviation;Generation adversarial network of supporting sensor native resolution is constructed, and false body fingerprint image is synthesized on the native resolution;Filtering is carried out to synthesized image according to the blind area portrait interval, and the synthesized sample that hits blind area is screened;The synthesized sample that hits blind area is used as attack class sample, and directional blind filling training is carried out to training set according to preset sampling density;Living body detection is carried out using the model after blind filling training, and when new continuous missed sample is detected, blind area portrait is iteratively updated and blind filling is repeated.The method realizes directional data blind filling to the blind area of real data coverage insufficient high imitation false body, and improves the detection rate of blind area false body of living body detection model.

Description

Technical Field

[0001] This invention relates to the field of biometric identification and information security technology, and in particular to a method, system and electronic device for filling in missing fingerprint liveness detection data. Background Technology

[0002] Fingerprint liveness detection (PAD) is a key security technology in biometric identification systems used to distinguish real fingers from fingerprints forged using prosthetic materials such as gelatin, silicone, and wood glue. The performance of fingerprint liveness detection models is highly dependent on the diversity and coverage of prosthetic samples in the training data: the richer the attack presentation types included in the training set, and the closer the materials and processes are to real-world attack scenarios, the stronger the model's generalization ability to unknown attacks. However, in actual acquisition processes, obtaining prosthetic fingerprint samples faces bottlenecks such as long acquisition cycles, high annotation costs, and limited technological conditions, resulting in a quantity of high-quality prosthetic samples that is far from sufficient to cover the attack patterns that may occur in the real world.

[0003] Of particular concern is that under specific acquisition conditions, such as low-light acquisition in low-light environments, full contact between the finger and sensor, or the use of high-density ridged gelatin-like materials, the texture statistical features of the images differ significantly from those of mainstream prostheses. These high-fidelity prosthesis samples are highly similar to real fingers in appearance, making conventional augmentation and feature engineering difficult to distinguish them effectively. Because the acquisition cost of such samples is extremely high and their occurrence probability under natural conditions is relatively low, usable real high-fidelity prosthesis data is extremely scarce. This "long tail" phenomenon in data distribution results in significant detection blind spots for existing liveness detection models under these conditions—areas where the model incorrectly identifies prosthesis samples as real fingers and consistently misses detection.

[0004] Using Generative Adversarial Networks (GANs) to synthesize spurious fingerprint samples to expand the training set is a feasible technical approach that has received widespread attention in recent years. However, existing methods have at least two shortcomings: First, in unconditional or weakly conditional free generation mode, the image distribution of the generated images is mainly determined by the mainstream distribution of the training data, and the proportion of images hitting the aforementioned detection blind spots is extremely low, contributing negligibly to the coverage of blind spot data; Second, mainstream GAN architectures (such as the StyleGAN series) default to generating images at power-of-two square resolutions (such as 256×256, 512×512), while the native acquisition resolution of real fingerprint sensors may be a non-power-of-two square resolution (such as 160×160, 320×320, etc.) or a non-square rectangular resolution (such as 192×256, 144×208, etc.). If padding or scaling is performed after synthesis to match the sensor resolution, the geometric proportions and periodic characteristics of the fingerprint ridges will inevitably be destroyed, causing the synthesized spur to deviate from the real sensor acquisition characteristics in terms of texture statistics, thus reducing the effectiveness of the synthesized samples in model training.

[0005] Therefore, there is an urgent need for a method to fill in the blind spots of fingerprint liveness detection models with targeted data. This method should be able to accurately quantify the fingerprint profile features of the blind spots, synthesize spoofed fingerprint images while maintaining the original resolution of the sensor, and efficiently inject the synthesized samples that hit the blind spots into the training process, thereby systematically improving the liveness detection model's ability to detect highly realistic spoofed fingerprints in the blind spots. Summary of the Invention

[0006] To address the aforementioned deficiencies in existing technologies, this invention provides a fingerprint liveness detection data blind spot method, system, and electronic device to solve the technical problems in existing technologies where the scarcity of real, high-fidelity fake body data leads to detection blind spots in liveness detection models, and where existing GAN synthesis methods cannot specifically produce samples that hit the blind spots at the sensor's native resolution.

[0007] In a first aspect, embodiments of this application provide a method for filling in missing fingerprint liveness detection data, comprising the following steps: Locate the blind zone samples of the fingerprint liveness detection model to form a blind zone sample set, extract the image features of each sample in the blind zone sample set, and determine the blind zone image interval based on the image features; A generative adversarial network supporting the sensor's native resolution is constructed, and the generative adversarial network is used to synthesize a spurious fingerprint image at the sensor's native resolution; Based on the blind zone image interval, the synthesized spurious fingerprint image is subjected to image filtering to select synthetic samples that meet the image filtering conditions as synthetic samples that hit the blind zone; the image filtering conditions are determined based on the blind zone image interval. The synthetic samples that hit the blind zone are used as attack samples and injected into the training set of the liveness detection model according to a preset sampling density to perform targeted blind spot training on the liveness detection model. After completing the targeted blind spot training, the trained liveness detection model is used to perform liveness detection on the spurious fingerprint test samples. When a new continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot. The sensor's native resolution includes a non-power-of-2 square resolution or a non-square rectangular resolution; the image features include ridge coverage, average brightness, and grayscale standard deviation; the detection blind zone sample is a spurious fingerprint sample that is missed as a real finger by both the target liveness detection model and at least one reference liveness detection model on the validation set or blind zone diagnostic set under the same evaluation protocol and fixed decision threshold, and whose missed detection is stably reproduced in multiple training rounds and / or multiple independent training instances.

[0008] Preferably, the blind zone samples of the positioning fingerprint liveness detection model form a blind zone sample set, the image features of each sample in the blind zone sample set are extracted, and the blind zone image interval is determined based on the image features, including: On a validation set or blind zone diagnostic set, a set of fingerprint liveness detection models are evaluated according to an evaluation protocol with a fixed decision threshold and a single forward inference. The set of fingerprint liveness detection models includes the target liveness detection model to be deployed and at least one reference liveness detection model trained for diagnostic comparison. The fake fingerprint samples that are simultaneously missed as real fingers by the set of fingerprint liveness detection models and stably reproduce the missed detection in multiple training rounds and / or multiple independent training instances are identified as the detection blind zone samples. For each sample in the detection blind zone sample set, calculate its ridge coverage, average brightness, and grayscale standard deviation. The range of values ​​for the sample set of the detection blind zone in the dimensions of ridge coverage, average brightness, and grayscale standard deviation is taken, and the Cartesian product of the three dimension ranges is taken as the blind zone image range.

[0009] Preferably, the step of constructing a generative adversarial network that supports the sensor's native resolution, and using the generative adversarial network to synthesize a spurious fingerprint image at the sensor's native resolution, includes: Using StyleGAN2 as the generator architecture, when the target output resolution of its resolution chain is the native resolution of the sensor, the type of the native resolution of the sensor is determined. If the sensor's native resolution is a non-power-of-2 square resolution, the starting base resolution of the resolution chain is extended from a fixed value to a selectable odd or non-power-of-2 value, so that the resolution chain reaches the sensor's native resolution after being upsampled by 2 times at each level. If the sensor's native resolution is a non-square rectangular resolution, the resolution chain is started with a preset height and width base resolution, and each synthesis level carries an axis-by-axis upsampling factor. The height direction is upsampled by 2 times at each level, and the width direction is anisotropically amplified through selective axis-by-axis resampling, so that the resolution chain finally reaches the sensor's native resolution.

[0010] Preferably, the selection of the initial substrate resolution is used to ensure the generator's ability to model the global structure of the fingerprint.

[0011] Preferably, the composite fingerprint image obtained by image filtering is performed based on the blind zone image interval, and composite samples that meet the image filtering conditions are used as composite samples that hit the blind zone; the image filtering conditions are determined based on the blind zone image interval and include: Calculate the ridge coverage, average brightness, and grayscale standard deviation for each synthetic spur fingerprint image; Determine whether the synthesized spurious fingerprint image meets preset conditions, the preset conditions being: the ridge coverage is not lower than the lower bound of the coverage of the blind area image interval, the average brightness falls into the brightness range of the blind area image interval, and the grayscale standard deviation is not lower than the lower bound of the standard deviation of the blind area image interval. The synthetic spurious fingerprint image that meets the preset conditions is used as the synthetic sample for the hit blind zone.

[0012] Preferably, the step of using the synthetic samples that hit the blind zone as attack samples and injecting them into the training set of the liveness detection model according to a preset sampling density to perform targeted blind spot training on the liveness detection model includes: The sampling budget for attack samples in the training set is divided into multiple independent sampling segments, including real low-coverage attack segments, blind zone synthetic attack segments, and other real attack segments. Each segment is sampled independently according to the set budget ratio. A main classifier and a source classifier are placed side-by-side after the shared backbone network of the liveness detection model. The source classifier is connected to the shared backbone network via a gradient inversion layer. Training is performed using the sum of the main task classification loss and the source classifier classification loss as the total loss. The gradient inversion layer is used to invert the gradient of the source classifier classification loss during backpropagation, and the inverted gradient is backpropagated to the shared backbone network. The source classifier is discarded after training is completed. Preferably, the preset sampling density ensures that the density of the synthetic samples that hit the blind zone in the training batch is higher than the density when they are randomly mixed into the training set.

[0013] Preferably, after completing the targeted blind spot training, the trained liveness detection model is used to perform liveness detection on the spurious fingerprint test samples. When a new, continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot filling, including: After each round of targeted blind patching training, the current liveness detection model is evaluated on the validation set or blind zone diagnosis set to locate the set of spurious fingerprint samples that are still being missed. If there are still samples that are continuously missed, the profile features of the sample set are recalculated and the value range is taken to obtain the updated blind zone profile interval. Based on the updated blind zone profile interval, the steps of synthesizing spurious fingerprint images, profile filtering and targeted blind spot training are re-executed. Repeat the above process until no new continuously missed samples are detected.

[0014] Secondly, embodiments of this application provide a fingerprint liveness detection data blinding system, comprising: The blind zone localization and image quantization module is used to locate the detection blind zone samples of the fingerprint liveness detection model, form a detection blind zone sample set, extract the image features of each sample in the detection blind zone sample set, and determine the blind zone image interval based on the image features. A native resolution synthesis module is used to construct a generative adversarial network that supports the native resolution of the sensor, and to synthesize a spurious fingerprint image at the native resolution of the sensor using the generative adversarial network. The image filtering module is used to perform image filtering on the synthesized spurious fingerprint image based on the blind zone image interval, and select the synthesized samples that meet the image filtering conditions as the synthesized samples that hit the blind zone; the image filtering conditions are determined based on the blind zone image interval. The targeted injection training module is used to inject the synthetic samples that hit the blind zone as attack samples into the training set of the liveness detection model according to a preset sampling density, and to perform targeted blind-filling training on the liveness detection model. The iterative blind spot filling module is used to perform liveness detection on the spurious fingerprint test samples using the trained liveness detection model after the targeted blind spot filling training is completed. When a new continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot filling. The sensor's native resolution includes a non-power-of-2 square resolution or a non-square rectangular resolution; the image features include ridge coverage, average brightness, and grayscale standard deviation; the detection blind zone sample is a spurious fingerprint sample that is missed as a real finger by both the target liveness detection model and at least one reference liveness detection model on the validation set or blind zone diagnostic set under the same evaluation protocol and fixed decision threshold, and whose missed detection is stably reproduced in multiple training rounds and / or multiple independent training instances.

[0015] Thirdly, embodiments of this application provide an electronic device, including: at least one memory for storing a computer program; and at least one processor for executing the computer program stored in the memory to implement the method described in any one of the first aspects.

[0016] Compared with the solutions of the prior art, the present invention has at least the following beneficial technical effects: (1) By quantifying the boundary of the detection blind zone in the multi-dimensional image feature space, the detection blind zone is transformed from fuzzy empirical judgment to precise quantitative description. By introducing three complementary image attributes, namely ridge coverage, average brightness and gray standard deviation, as blind zone image features, the fingerprint image statistical characteristics of the blind zone prosthesis can be effectively characterized from three complementary dimensions: texture density, illumination conditions and contrast, providing clear and operable generation constraints for subsequent targeted synthesis; (2) By modifying the resolution chain structure of the StyleGAN2 generator, it can directly synthesize spurious fingerprint images at non-power-of-2 square resolutions or non-square rectangular resolutions, avoiding the traditional secondary processing flow of first synthesizing and then padding or scaling. This helps to maintain the geometric proportions and ridge periods of the synthesized fingerprint image consistent with the characteristics of the real sensor acquisition, significantly reduces texture distortion introduced by resolution transformation, and significantly improves the fidelity of the synthesized spur in terms of texture statistics to the characteristics of the real sensor acquisition; (3) By using a blind zone image filtering mechanism, synthetic samples that hit the blind zone are efficiently screened from a large number of fake fingerprint images generated by GAN. Based on image filtering (rejection sampling), image distance loss-guided generation is further adopted, which can make the generated samples fall directly or nearly directly into the blind zone image interval, significantly improving the hit efficiency and greatly reducing the amount of data discarded after generation; (4) By using a segmented balanced sampling strategy, the sampling budget of attack-type samples in the training set is divided into multiple independent sampling segments, which increases the sampling density of synthetic samples in the blind zone by at least one order of magnitude compared to generalized augmentation. This ensures that a small number of high-quality synthetic samples in the blind zone can generate sufficient gradient contributions during model training, avoiding the problem of being submerged in a large number of non-blind zone samples due to the small sample size, and achieving accurate targeted data blinding. (5) By introducing a source discriminator and a gradient reversal layer after the shared backbone network of the liveness detection model, source invariance adversarial training is performed, which enables the model to learn feature representations that are invariant to synthetic and real data sources. This effectively suppresses the bias effect of domain differences between synthetic and real data on the classifier boundary, so that the model can use synthetic samples to fill blind spots without reducing the risk of losing the ability to discriminate real samples due to synthetic artifacts or statistical bias. (6) Targeted blind spot training is conducted using an iterative closed-loop approach: After each round of blind spot training, the detection blind spots of the model are re-evaluated, the blind spot image interval is updated, and the generation direction and injection strategy of the generator are adjusted accordingly. This closed-loop iterative mechanism allows the detection blind spots to gradually shrink as training progresses, and the iteration ends when the termination condition is met. This realizes a continuous optimization process from discovering blind spots to filling in blind spots to discovering new blind spots, ultimately strengthening the decision boundary of the liveness detection model in the high-fidelity prosthetic region.

[0017] In summary, this invention constructs a complete fingerprint liveness detection data blind spot filling technology solution through blind spot quantization, native resolution-oriented synthesis, image filtering, high-density targeted injection, and iterative blind spot filling. This method transforms the detection blind spot from an implicit concept into an explicit, quantifiable image constraint, achieving high-fidelity, high-hit-rate spoof fingerprint synthesis at the sensor's native resolution. Furthermore, segmented sampling and source invariance adversarial training ensure the effective injection of synthesized samples, ultimately achieving data-level blind spot filling for highly realistic spoofs in blind spots with insufficient real data coverage. This data-driven approach significantly improves the detection rate and overall robustness of the liveness detection model against spoofs in blind spots. Attached Figure Description

[0018] Exemplary embodiments of the present invention can be more fully understood by referring to the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the present invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 A flowchart of a fingerprint liveness detection data blinding method provided as an exemplary embodiment of this application; Figure 2 A schematic diagram of resolution chain modification under a non-power-of-2 square resolution provided as an exemplary embodiment of this application; Figure 3 A schematic diagram of an anisotropic resolution chain under non-square rectangular resolution provided for an exemplary embodiment of this application; Figure 4 A schematic diagram of a synthetic spurious fingerprint sample with native resolution provided as an exemplary embodiment of this application; Figure 5 A flowchart of a source invariance anti-biasing mechanism provided for an exemplary embodiment of this application; Figure 6 A schematic diagram comparing the effects of the present method and the prior art, provided as an exemplary embodiment of this application; Figure 7 A schematic diagram of a fingerprint liveness detection data completion system provided as an exemplary embodiment of this application; Figure 8 A schematic diagram of an electronic device provided for an exemplary embodiment of this application; Figure 9 This is a schematic diagram of a computer-readable medium provided for an exemplary embodiment of this application. Detailed Implementation

[0020] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0021] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0022] For ease of explanation, this application adopts the following conventions: spurious fingerprint samples are collectively referred to as attack samples, serving as negative samples for liveness detection; real finger samples are referred to as real samples.

[0023] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] Please refer to Figure 1 , Figure 1 This application discloses a fingerprint liveness detection data blinding method in some embodiments. The method implemented in this disclosure is executed by a processor in an electronic device, and the method includes the following steps: S110, locate the detection blind zone samples of the fingerprint liveness detection model, form a detection blind zone sample set, extract the image features of each sample in the detection blind zone sample set, and determine the blind zone image interval based on the image features. Specifically, a set of fingerprint liveness detection models is first evaluated on a validation set or blind zone diagnostic set according to an evaluation protocol with a fixed decision threshold and single forward inference. This set of models preferably includes the target liveness detection model Mtarget to be deployed, and one or more reference liveness detection models Mref1, Mref2, ..., Mrefk trained for diagnostic comparison. The reference models can employ different backbone network architectures than the target model (e.g., the target model uses MobileNetV3, while the reference models use ResNet-18 and EfficientNet-B0 respectively), or different training hyperparameter configurations (e.g., different learning rate strategies, different data augmentation combinations). The aim is to provide diverse model perspectives to distinguish between accidental missed detections and systemic blind zones.

[0025] Specifically, for each sample image in the set of samples in the detection blind zone... Calculate the following three dimensions of portrait features respectively: (1) Average brightness Average brightness This reflects the overall grayscale / response level of the acquired image (the corresponding illumination level for optical sensors, and the corresponding signal response intensity for capacitive sensors, etc.), and is a key feature distinguishing low-brightness high-fidelity fakes from normal samples. Let... For sample images In pixel coordinates The gray value at point (8-bit grayscale image, value from 0 to 255) is the average gray value of the entire image. The calculation formula is: (1); in, The summation symbol indicates the summation of the sample images. Sum all pixels, , Sample images The height and width. The average brightness is calculated directly on the original grayscale image (or an image that has only undergone uniform grayscale calibration), without preprocessing to remove the global illumination component, in order to retain the overall brightness / response information as image features; based on large-scale Gaussian filtering ( The pre-processing for estimating illumination components can only be used for the calculation of other features such as ridge segmentation, which will be discussed later, and should not be applied to average brightness.

[0026] The sample image can be processed before calculation. Illumination normalization preprocessing is performed, specifically by using a large-scale Gaussian filter ( Estimate the illumination component, then subtract the illumination component from the original image to eliminate the interference of uneven lighting. For samples under low-light conditions, its The value was significantly lower than that of the sample under normal lighting conditions.

[0027] (2) Standard deviation of gray level Specifically, grayscale standard deviation Reflects the sample image The overall pixel grayscale fluctuation (calculated across all pixels) is used to characterize the grayscale contrast level of the sample image. Sample image of The calculation formula is: (2); For the blind zone of the full-contact high-imitation prosthesis targeted in this embodiment, the texture imprint is complete, the grayscale fluctuation is at a high level, and the grayscale standard deviation is usually high (belonging to the strong contrast type, which is consistent with the screening direction of only setting the lower bound in formula (8) below).

[0028] (3) Ridge coverage Ridge coverage This reflects the density of texture information in a fingerprint image, specifically the proportion of foreground pixels containing ridge textures to the total number of pixels in the image. For high-quality gelatin-based fingerprint prostheses, due to the excellent imprint transfer properties of the material, ridge coverage is typically high. Specifically, for sample images... Texture intensity thresholding / binarization is performed to obtain a foreground region mask containing fingerprint ridge texture (foreground refers to the effective fingerprint region containing ridge texture, not just the ridge pixels). The ridge coverage rate referred to in this application refers to the proportion of the effective foreground region containing ridge texture to the image, not just the proportion of fingerprint ridge pixels. Specifically, the ridge coverage rate... The calculation formula is: (3); in, The number of pixels within the foreground area mask; ridge coverage when fully pressed. The value is close to 0.9. In a preferred embodiment, the determination of the effective fingerprint texture foreground region can be achieved by preprocessing the fingerprint image through filtering, and then generating a foreground region mask using methods such as local adaptive thresholding, texture intensity, or orientation consistency. In another embodiment, a fingerprint foreground segmentation network based on deep learning can be used to output a pixel-level effective texture foreground probability map, and a foreground region mask can be obtained based on a preset probability threshold.

[0029] After calculating the 3D profile features of all samples in the set, the range of values ​​for each dimension of the set is taken (the minimum-maximum range or the 5%~95% quantile range can be used to remove outliers). The Cartesian product of the three dimension ranges is used as the blind zone profile range. : (4); in, Represents Cartesian product rather than numerical multiplication, blind spot image interval It is a box-shaped region in the three-dimensional feature space; , These are the minimum and maximum values ​​(or lower and upper quantiles) of the sample set in the detection blind zone along the ridge coverage dimension. , These are the minimum and maximum values ​​(or lower and upper quantiles) in the average brightness dimension, respectively. , These are the minimum and maximum values ​​(or lower and upper quantiles) in the grayscale standard deviation dimension, respectively.

[0030] Specifically, blind spot image intervals This refers to the quantized geometric representation of the detection blind zone in the 3D image feature space. If any sample's 3D image features fall within... If the sample falls within the detection blind zone in terms of image features, then it is considered to fall within that zone. In practical applications, to increase the diversity of synthetic samples and avoid overly rigid boundaries, a relaxation factor can be appropriately extended outwards for each dimension's interval. (like =0.1, indicating expansion to both the minimum and maximum ends by 10%, for example, ... Expand to [ , and Then, they are scaled according to their respective value ranges.

[0031] In an alternative embodiment, the determination of blind zone image intervals can also employ a probability density estimation-based method: kernel density estimation (KDE) is performed on the three-dimensional features of the set, and regions with densities exceeding a set threshold are selected as blind zone image intervals. This method can capture nonlinear, non-convex blind zone morphologies, but its implementation complexity is high. In engineering practice, the Cartesian product form of interval representation is more practical due to its simplicity and interpretability.

[0032] The two interval construction schemes are applicable to different scenarios: the three-dimensional Cartesian product interval calculation logic is simple and has low computational cost, making it suitable for lightweight offline training scenarios such as mobile terminals and edge devices; the kernel density estimation (KDE) scheme can more flexibly characterize the boundaries of irregular and non-convex blind zones, but its computational complexity is relatively high, making it suitable for offline training environments with sufficient computing power.

[0033] For example, for a typical "dark-light full-contact high-fidelity" blind area, the typical value of its image feature is: ridge line coverage. average brightness grayscale standard deviation .

[0034] S120, Construct a generative adversarial network that supports the sensor's native resolution, and use the generative adversarial network to synthesize a spurious fingerprint image at the sensor's native resolution; This embodiment uses StyleGAN2 as the generator architecture. In low-sample scenarios, the StyleGAN2-ADA variant (adaptive discriminator augmentation to prevent discriminator overfitting) is preferred, and both have the same resolution chain structure. In scenarios where real-world samples of prosthetic fingerprints are scarce, adaptive discriminative augmentation can expand the discriminant input distribution without modifying the generator's resolution chain, avoiding rapid overfitting of the discriminator to a small number of real prosthetics and ensuring stable convergence of native resolution synthesis training.

[0035] Specifically, the StyleGAN2 generator consists of two parts: a mapping network and a synthesis network. The mapping network and the synthesis network will be explained in detail below: (1) Mapping network Input hidden variables (Typically a 512-dimensional standard normal distribution vector) is mapped to intermediate latent variables: (5); Among them, mapping network It is an MLP structure with latent variables. .

[0036] (2) Synthetic Network by As input, the final image is synthesized through a series of progressive resolution levels. A truncation coefficient is introduced during the inference generation stage. To control the diversity of generated images, specifically, the inference generation stage includes the following process: Let... The average style center of the latent space (by randomly sampling a large number of samples on the training set). And calculate the corresponding (obtained by averaging), then the actual truncated latent variables used. for: (6); In this embodiment, to preserve the natural defect features in the synthetic prosthetic fingerprint image (such as white spots, clumps, uneven pressure, etc., which can serve as effective clues for liveness detection), this embodiment... =1.0 (without truncation) to preserve real material defects such as white spots, lumps, and uneven pressing, which are effective clues for liveness detection.

[0037] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the resolution chain modification under a non-power-of-2 square resolution (taking 160×160 as an example) in this embodiment. The starting base resolution of the resolution chain is extended from a fixed value of 4 to a selectable non-power-of-2 value, so that the resolution chain accurately reaches the target resolution after being upsampled by 2 times at each stage.

[0038] For a target resolution of 160×160, if the initial base resolution is set to base=5, the resolution chain is 5→10→20→40→80→160 (a total of 6 levels), reaching the target resolution through progressively doubling upsampling at each level. The selection of the base resolution must satisfy two conditions: firstly, the base should not be too fine (e.g., an excessively fine initial base resolution like 8×5, while providing good local texture modeling capabilities, will impair the coherence of the global fingerprint structure, such as ridge orientation, singularities, and whorls); secondly, the base should ensure that the final resolution after integer-level doubling upsampling is exactly equal to the target resolution.

[0039] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the anisotropic resolution chain under a non-rectangular resolution (taking 128×80 as an example) in this embodiment. The resolution chain is started with preset height and width bases respectively. Each synthesis level carries an axis-by-axis upsampling factor. The height direction is fixed at 2 times upsampling per level, and the width direction is anisotropically amplified through selective axis-by-axis resampling, finally reaching the target rectangular resolution.

[0040] Specifically, the recursive formula for the resolution chain is: (7); in, , The first The height and width of the feature map output from each synthesis layer This is the upsampling factor in the width direction, with a value of 1 (keeping the width unchanged) or 2 (double upsampling in the width direction). This can be selected as needed at each level. The resolution chain can eventually reach any rectangular target resolution. Taking a target resolution of 128×80 and an initial base of 4×5 as an example, the resolution chain is: 4×5→8×5→16×10→32×20→64×40→128×80. At the implementation level, anisotropic upsampling is achieved by performing convolutional resampling in both the height and width directions: isotropic paths correspond to the stepwise 2x upsampling convolution operation of the original StyleGAN2, while anisotropic paths perform upsampling in the width direction... Values ​​can be selectively inserted or the upsampling operation can be skipped.

[0041] Please refer to Figure 4 , Figure 4 This example demonstrates a gelatin-based fingerprint sample directly synthesized at a non-power-2 native resolution of 160×160. Thanks to the modified native resolution generation chain, the ridge direction and ridge period of the synthesized image are consistent with the characteristics of real sensor acquisition, without exhibiting the obvious geometric distortion and texture blurring caused by traditional methods of first synthesizing and then scaling and edge patching. At the same time, it completely preserves the natural defects of real prosthetic materials such as white spots, lumps, and uneven pressure. These defect features can serve as effective discrimination clues for liveness detection, providing a high-fidelity synthetic data foundation for subsequent targeted blind spot training.

[0042] S130, Based on the blind zone image interval, the synthesized spurious fingerprint image is subjected to image filtering to select the synthesized samples that meet the image filtering conditions as the synthesized samples that hit the blind zone; the image filtering conditions are determined based on the blind zone image interval. For each synthetic prosthetic fingerprint image Calculate its ridge coverage according to the method described in step S110. Average brightness and grayscale standard deviation The core criteria for image filtering are: (8); in, Corresponding to the blind spot image interval Lower bound of the mid-ridge coverage dimension , These are the lower and upper limits of the average brightness, respectively. This represents the lower bound of the grayscale standard deviation dimension. In practical applications, a margin can be appropriately relaxed for each lower bound and interval to increase the diversity of synthetic samples and training robustness.

[0043] It should be noted that only a lower bound is set for ridge coverage and grayscale standard deviation, but no upper bound is set. This is because the detection blind zone characteristics targeted in this embodiment are full contact and strong contrast types, and samples with feature values ​​higher than the upper bound still belong to the target blind zone type. However, both upper and lower bounds are set for average brightness, so that dark light conditions can be precisely defined.

[0044] The image filtering in this embodiment is essentially a rejection sampling strategy, which applies to each randomly generated latent variable. It determines whether the corresponding generated image meets the image quality criteria; if not, it discards the image and resamples a new one. This process is repeated until a sufficient number of synthetic samples that hit the blind zone are collected. Here, the hit rate refers to the proportion of image features hitting the blind zone in the generated candidate samples: under the indiscriminate generalized synthesis settings of existing technologies, this proportion is only about 1%; this method trains the generator with prosthetic domain data at the sensor's native resolution and preserves the complete distribution, resulting in a blind zone hit rate of approximately 0.10 (about 10%) for generated candidates; image filtering (rejection sampling) selects hit samples from the candidates according to this acceptance rate to form the injection set, where the hit rate is close to 100%. The improvement in the candidate hit rate stems from the generator settings (in-domain training and native resolution), while filtering itself does not change the candidate distribution.

[0045] Specifically, lightweight engineering deployment can directly adopt rejection sampling, without the need for gradient optimization and with lower inference overhead; for production line training scenarios with large blind zone sample requirements and sufficient computing power, the image distance loss-guided generation is given priority, which greatly improves the sample hit rate.

[0046] As a preferred implementation, to further improve the hit rate, a targeted generation technique based on image distance loss can be employed. (Define image distance loss) for: (9); in, , and These represent the blind spot image intervals. Target intervals in the dimensions of ridge coverage, average brightness, and grayscale standard deviation; distance function That is, when the eigenvalue Located in the interval The distance within the interval is 0; otherwise, the distance is the portion exceeding the interval. This is achieved by analyzing latent variables. Perform gradient descent optimization: (10); in, The optimized latent vectors, In order to make Take the minimum value This optimization formula shrinks the generator's distribution towards the blind image region. If the image feature calculation involves non-differentiable operations (such as binarized thresholding), a soft thresholding function (such as the Sigmoid approximation) can be used instead of a hard threshold to ensure that the gradient can be backpropagated to the latent variables throughout the computation graph. After adopting directional generation technology, It can shrink the generated distribution to the blind area image interval, significantly improve the hit rate, and eliminate the need to generate and discard a large number of images.

[0047] As an alternative implementation, blind zone image features, blind zone image intervals, or their encoding results can be used as conditional information input to a generative adversarial network (i.e., image conditional generation). This allows the generative network to directly generate a prosthetic fingerprint image whose image features satisfy the image selection conditions under the stated conditional constraints. For example, a target image vector (ct, Bt, σt) can be determined or sampled from the target blind zone image interval, mapped into a conditional embedding via a conditional encoding network, and input into a mapping network along with latent variables, or used to modulate the synthesis layer in the generative network. Image conditional generation can be combined with or used in place of the aforementioned image filtering and image distance loss-guided generation.

[0048] S140, Synthetic samples that hit the blind zone are used as attack samples and injected into the training set of the liveness detection model according to the preset sampling density to perform targeted blindness filling training on the liveness detection model; At the data loading level, a segmented balanced sampling strategy is designed. The sampling budget for attack samples in the training set is divided into three independent sampling segments: a real low-coverage attack segment (selecting samples with the lowest ridge coverage from real prosthetic samples sorted by portrait features), a blind zone synthesis attack segment (composed of synthetic samples that hit the blind zone selected in step S130), and the remaining real attack segment (containing regular real prosthetic samples). The proportions of the three sampling budgets are as follows: (11); This ratio is the preferred value in this embodiment and can be flexibly adjusted according to the scarcity of blind zone samples and the total number of samples. That is, in each training batch, attack class samples are allocated proportionally... Each attack sample is drawn independently from three sampling segments. The probability of each attack sample being sampled is proportional to the budget ratio of the sampling segment to which that sample belongs. The ratio is proportional to the total number of samples in that segment. The actual collected samples still dominate (approximately 35%), and the fixed density of the blind zone synthetic segment (approximately 15%) is about an order of magnitude higher than the natural occurrence density of blind zone samples under the general data augmentation method, ensuring that a small number of high-quality blind zone synthetic samples can generate sufficient gradient contributions during model training.

[0049] At the model architecture level, to prevent the model from using generator traces of synthetic samples as a shortcut for discrimination (otherwise it will fail when encountering real attacks without such traces during deployment), a source invariance adversarial bias removal mechanism is introduced, see [link to relevant documentation]. Figure 5 , Figure 5 A flowchart of the source invariance-resistant debiasing mechanism is presented. This is applied to the shared backbone network of the liveness detection model. After (such as ResNet-18, MobileNetV3, or EfficientNet), set up two output heads side by side: a main classification head (responsible for the live / pseudo binary classification task, outputting the live probability) and a source discriminant head. (Responsible for distinguishing whether the input sample's data source is real collected data or synthetically generated data). The source discriminator is connected to the shared backbone network through a Gradient Reversal Layer (GRL). Specifically, the behavior of the GRL is defined as follows: (12); That is, GRL behaves as an identity transformation during forward propagation (the input signal passes through without change), and during backward propagation, the gradient is multiplied by a negative coefficient - ( >0, where E is the identity matrix), thus reversing the gradient direction of the source discrimination loss and backpropagating it to the backbone network. Gradient reversal balance coefficient A value of 0.1 to 1 is acceptable; if the synthetic sample neighborhood is severely offset, the value can be increased. When weakened, it can be reduced Balance the training weights between the main classification task and the source discrimination adversarial task.

[0050] At this point, the model's total loss function is: (13); in, For live / pseudonym binary classification, the cross-entropy loss is used. For real living body / prosthetic labeling, For the backbone network to input Extracted features; The cross-entropy source classification loss of the source discriminant head. Labels indicating the source of real data (real / synthetic). (·) represents the gradient inversion layer; These are the balancing weights for the source discrimination loss. Through GRL's backward gradient flipping, the backbone network, while maximizing the main classification accuracy, is guided to learn feature representations invariant to the sample source (real / synthetic). That is, the features extracted by the backbone network should be so indistinguishable from real or synthetic samples by the source discriminator. This mechanism effectively mitigates the bias effect of the domain gap between synthetic and real data on the classification boundary. The source discriminator is only used during training and discarded after training, requiring no additional computational overhead during model deployment and inference.

[0051] S150: After completing the targeted blind spot training, the trained liveness detection model is used to perform liveness detection on the spurious fingerprint test samples. When a new continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the blind spot of fingerprint liveness detection data. This step establishes an iterative closed-loop optimization mechanism for the detector in the loop. Specifically: After each round of targeted blind-filling training, the trained liveness detection model is obtained. On the validation set or blind spot diagnostic set... The evaluation was conducted to identify the set of spurious fingerprint samples that continued to be missed. New persistently missed detection samples are determined according to the same stability reproducibility criteria as in step S110 (this can be combined with the reference model for determination; in a lightweight implementation, it can also be determined solely based on the stability of the current liveness detection model). If not empty (i.e., there are still samples that are missed), then use For the new set of blind zone samples, recalculate the profile features and select the value range according to the steps described above to obtain the updated blind zone profile interval: = (14); Based on the updated blind spot profile interval Return to step S120 and re-perform the prosthetic fingerprint synthesis, image filtering, and targeted blind spot training. Repeat the above iterative loop of evaluation, localization, image update, synthesis, filtering, and injection training until no new continuously missed samples are detected (i.e., If the value is empty, it is assumed that the detection blind zone has been fully covered, and the iteration terminates. The maximum number of iterations is also set. This serves as an upper limit for loop termination, preventing infinite iteration under the limits of the dataset. The number of cycles can be preset to 3 to 10 based on the production line quality standards.

[0052] To verify the effectiveness of the technical solution of this application, this embodiment conducts a comparative experiment on a dataset collected by a capacitive fingerprint sensor. The dataset corresponds to a sensor native resolution of 160×160 pixels (not a power of 2 square resolution), covering real finger samples and multi-material prosthetic fingerprint samples.

[0053] It should be noted that blind spot localization and profiling statistics are performed on diagnostic data; the batch samples of the blind spots that are located are only used for evaluation and profiling statistics and are never used for training. The improvement of their detection rate is used for targeted verification of the batches that are located; the false rejection rate of real fingers is evaluated on real finger samples that are unrelated to blind spot localization.

[0054] Please refer to Figure 6 , Figure 6 This diagram illustrates the comparison between the effectiveness of this method and existing technologies, including two sub-figures: (a) comparison of blind zone hit efficiency and (b) comparison of blind zone detection effectiveness.

[0055] in, Figure 6 (a) shows the blind zone hit efficiency comparison, used to verify the blind zone coverage efficiency of the synthetic data: Under the indiscriminate general synthesis setting of the existing technology, the proportion of blind zone images in the generated candidate samples is only about 1%; the blind zone hit rate of the generator (trained with native resolution and pseudo-domain data) of this method is about 10% in the generated candidates. The image filtering (rejection sampling) selects the hit samples from the candidates according to this acceptance rate to form the injection set. The candidate hit rate is about 10 times higher than the existing scheme. If further combined with targeted generation optimization, the hit efficiency can be further improved.

[0056] Figure 6 (b) shows the comparison of blind zone detection performance, used to verify the performance improvement after blinding: Before blinding, the single-shot detection rate of the target blind zone prosthesis was approximately 89%, failing to reach the target detection threshold of 95%; after one round of targeted blinding using this method, the single-shot detection rate of the blind zone prosthesis increased to approximately 96%, successfully exceeding the target detection threshold of 95%; at the same time, the false rejection rate of real fingers remained basically unchanged, and the detection rate of other conventional batches of prosthesis samples showed no significant fluctuation. This result verifies that this method can efficiently and accurately address the shortcomings of blind zone detection without compromising conventional detection performance.

[0057] It should be noted that this embodiment utilizes a four-stage technique—quantitative blind spot profiling, native resolution synthesis, image filtering, and density-enhanced targeted injection training—to achieve targeted data filling for high-fidelity spoofing in blind spots where real data coverage is insufficient. The thresholds and hyperparameters involved in each of the above steps (such as...) 、[ , ]、 , , (etc.) can be adjusted according to specific application scenarios and dataset characteristics, and do not constitute a limitation of the present invention. In addition, the method in this embodiment is not only applicable to gelatin prosthetic materials, but also to other prosthetic materials such as printed materials and white glue, as well as non-square aspect ratios of different sensor models, and can be used to perform targeted blind filling according to the same process.

[0058] Please refer to Figure 7 , Figure 7 This embodiment of the present application illustrates a fingerprint liveness detection data completion system 20. This system corresponds one-to-one with the method steps in the aforementioned method embodiments, and provides further explanation of the method implementation. System 20 includes the following functional modules: The blind zone localization and profiling quantization module 210 is used to locate the blind zone samples of the fingerprint liveness detection model, extract the profiling features of the blind zone samples, and determine the blind zone profiling interval based on the profiling features. This module further includes: a model evaluation submodule, used to evaluate a set of fingerprint liveness detection models on a validation set or blind zone diagnosis set according to an evaluation protocol with a fixed decision threshold and single forward inference; a blind zone identification submodule, used to identify spurious fingerprint samples that are simultaneously missed as genuine fingers by the model and stably reproduce the missed detection as blind zone samples; and a feature calculation submodule, used to calculate the ridge coverage rate for each sample in the blind zone sample set. Average brightness (B) and grayscale standard deviation The interval construction submodule is used to extract the Cartesian product of the value range of the 3D features as the blind spot image interval. .

[0059] The native resolution synthesis module 220 is used to construct a generative adversarial network (GAN) that supports the sensor's native resolution, and uses the GAN to synthesize spurious fingerprint images at the sensor's native resolution. This module further includes: a generator construction submodule, which uses StyleGAN2 as the generator architecture and modifies the resolution chain to support non-power-of-two square resolutions and non-square rectangular resolutions; an ADA (Adaptive Discriminator Augmentation) training submodule, used to apply adaptive data augmentation to the discriminator to stabilize the generator training under limited real spurious samples; and a truncation control submodule, which truncates parameters during the inference generation phase. Set to 1.0.

[0060] The image filtering module 230 is used to filter the synthesized spurious fingerprint images based on the blind zone image interval and select the synthesized samples that hit the blind zone. The module further includes: a feature extraction submodule, used to calculate the image features of each synthesized spurious fingerprint image; a condition judgment submodule, used to determine whether the three-dimensional features of the synthesized image meet the image filtering condition of formula (8); and a guided generation submodule, which performs gradient descent optimization (formula (10)) on the latent variables based on the image distance loss function (formula (9)).

[0061] The targeted injection training module 240 is used to inject synthetic samples that hit the blind zone as attack samples into the training set of the liveness detection model according to a preset sampling density, and to perform targeted blind spot training on the liveness detection model. This module further includes: a segmented sampling submodule, which divides the sampling budget of attack samples in the training set into multiple independent sampling segments according to the ratio of formula (11); and a source invariance adversarial training submodule, which sets the main classification head and the source discriminant head accessed through the gradient inversion layer in parallel after the shared backbone network of the liveness detection model, and performs source invariance adversarial training according to formulas (12) and (13).

[0062] The iterative blinding module 250 is used to perform liveness detection on the spurious fingerprint test sample after the directional blinding training is completed. When a new continuously missed sample is detected, the blind area image interval is updated according to formula (14) and a new round of data blinding is triggered.

[0063] It should be noted that the above functional modules are only logical functional divisions, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system. This system embodiment and the above method embodiment are based on the same inventive concept, and their specific implementation process and technical effects can be referred to the corresponding descriptions in the method embodiments.

[0064] Some embodiments of this application disclose specific application scenarios of the methods described in the above embodiments, so as to more specifically illustrate the specific workflow and beneficial effects of the methods in this invention and the liveness detection model trained by the methods when deployed in multiple scenarios, especially in three typical applications: smart terminal fingerprint unlocking, financial payment verification, and access control and attendance authentication.

[0065] In smartphone under-display fingerprint unlocking scenarios, the built-in optical fingerprint sensor captures the user's fingerprint image, and a liveness detection model trained using this method distinguishes between real fingers and spoofing attacks in real time. For spoofing attacks in blind spots, such as those occurring in low light or with full contact with highly realistic imitation membranes, this method pre-fills in targeted data, significantly improving the detection rate of spoofing attacks in these blind spots without increasing the false rejection rate of real fingers. This maintains the false rejection performance of real fingers, preventing information security risks from highly realistic spoofing attacks that compromise fingerprint unlocking. Furthermore, the model trained based on the sensor's native resolution requires no scaling or edge-padding preprocessing of the captured image; detection can be completed directly from the input. The single-frame inference time is comparable to conventional models, without affecting unlocking response speed.

[0066] In financial payment fingerprint verification scenarios, the fingerprint module of the payment terminal needs to meet high-security liveness detection requirements, demanding extremely high generalization ability against various unknown spoofs. Through the iterative closed-loop blind spot filling mechanism of this method, newly emerging spoofed samples can be continuously collected after deployment, automatically updating the blind spot profile and selectively supplementing synthetic data, iteratively optimizing the model's detection boundaries. This allows for rapid patching of security vulnerabilities without the need to re-collect a large number of real spoof samples, significantly reducing the data collection cost of security iterations. Simultaneously, the source-invariant adversarial training mechanism ensures that synthetic data blind spot filling does not introduce new discrimination shortcuts, and the model's performance remains stable and reliable under real attack scenarios.

[0067] In fingerprint authentication scenarios for access control and attendance, devices often face complex usage conditions with varying lighting environments and pressure levels. Low-light, full-contact spoofing attacks are more likely to bypass detection. This method accurately characterizes detection blind spots under different environments using 3D portrait features and expands the spoofing training samples under corresponding conditions. This allows the model to maintain stable liveness detection accuracy even in complex usage environments, reducing access control security risks caused by false access.

[0068] Please refer to Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device 30 includes a processor 300, a memory 301, a bus 302, and a communication interface 303. The processor 300, the communication interface 303, and the memory 301 are connected via the bus 302. The memory 301 stores a computer program that can run on the processor 300. When the processor 300 runs the computer program, it executes the method described above in this application. This electronic device can be a server for model training, an industrial control computer for production line calibration, or any device with fingerprint liveness detection requirements, such as a smartphone with fingerprint recognition function, an access control terminal, or a payment verification device.

[0069] The memory 301 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 303 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0070] Bus 302 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs. After receiving an execution instruction, the processor 300 executes the program. The methods disclosed in any of the foregoing embodiments of this application can be applied to the processor 300, or implemented by the processor 300.

[0071] The processor 300 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 300 or by instructions in software form. The processor 300 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 301. The processor 300 reads the information in memory 301 and, in conjunction with its hardware, completes the steps of the above method.

[0072] The electronic device provided in this application embodiment and the method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented. When the electronic device is used as a training end, it can efficiently complete blind spot image localization, native resolution prosthesis synthesis and targeted blind spot training, and output a highly robust liveness detection model; when used as a terminal deployment end, it can load the blind spot optimized model to achieve high-precision fingerprint liveness detection.

[0073] This application also provides a computer-readable medium corresponding to the method provided in the foregoing embodiments. Please refer to... Figure 9 The computer-readable storage medium shown is an optical disc 40, on which a computer program (i.e., a program product) is stored, which, when run by a processor, performs the aforementioned method.

[0074] The computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. In a specific application scenario, the computer-readable storage medium can be built into a conference terminal or mobile terminal as part of the firmware; or it can be deployed as a standalone software product on a model training device through download or physical distribution. This allows a conventional fingerprint liveness detection training platform to obtain the blind spot detection data completion function based on blind spot image-oriented synthesis as described in this invention after loading and executing the computer program on the storage medium. This improves the model's ability to detect high-fidelity fakes in blind spots without significantly modifying the original training framework.

[0075] The computer-readable storage medium provided in the above embodiments of this application and the method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0076] This embodiment also provides a computer program product, which includes a computer program. When executed by a processor, the computer program implements the fingerprint liveness detection data blinding method as described in any of the above embodiments.

[0077] The computer program product may include any combination of one or more program modules, such as: a blind spot image quantization module for locating and detecting blind spots and calculating the feature range of three-dimensional images; a native resolution synthesis module for modifying the GAN resolution chain and synthesizing native resolution prosthetic images; an image filtering module for filtering blind spot images and generating synthetic samples that hit the blind spots; a targeted injection training module for segmented sampling and source invariance adversarial training; and an iterative blind spot filling module for iteratively updating the blind spot images and iteratively optimizing them based on model evaluation results.

[0078] This computer program product can be deployed on a cloud training server to provide targeted blind spot optimization services for the liveness detection model to terminal device manufacturers in the form of a service; it can also be packaged as a software development kit (SDK) for third-party fingerprint algorithm manufacturers to integrate and call, so that the training platform integrating the SDK can obtain the blind spot targeted data blind spot capability of this invention without changing the original training process, and quickly improve the detection performance of the liveness detection model for rare high-fidelity fakes.

[0079] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0080] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application.

Claims

1. A method for filling in blind spots in fingerprint liveness detection data, characterized in that, Includes the following steps: Locate the blind zone samples of the fingerprint liveness detection model to form a blind zone sample set, extract the image features of each sample in the blind zone sample set, and determine the blind zone image interval based on the image features; A generative adversarial network supporting the sensor's native resolution is constructed, and the generative adversarial network is used to synthesize a spurious fingerprint image at the sensor's native resolution; Based on the blind zone image interval, the synthesized spurious fingerprint image is subjected to image filtering to select synthetic samples that meet the image filtering conditions as synthetic samples that hit the blind zone; the image filtering conditions are determined based on the blind zone image interval. The synthetic samples that hit the blind zone are used as attack samples and injected into the training set of the liveness detection model according to a preset sampling density to perform targeted blind spot training on the liveness detection model. After completing the targeted blind spot training, the trained liveness detection model is used to perform liveness detection on the spurious fingerprint test samples. When a new continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot. The sensor's native resolution includes a non-power-of-2 square resolution or a non-square rectangular resolution; the image features include ridge coverage, average brightness, and grayscale standard deviation; the detection blind zone sample is a spurious fingerprint sample that is missed as a real finger by both the target liveness detection model and at least one reference liveness detection model on the validation set or blind zone diagnostic set under the same evaluation protocol and fixed decision threshold, and whose missed detection is stably reproduced in multiple training rounds and / or multiple independent training instances.

2. The method according to claim 1, characterized in that, The detection blind zone samples of the fingerprint liveness detection model are used to form a detection blind zone sample set. Image features of each sample within the detection blind zone sample set are extracted, and the blind zone image interval is determined based on these image features, including: On a validation set or blind zone diagnostic set, a set of fingerprint liveness detection models are evaluated according to an evaluation protocol with a fixed decision threshold and a single forward inference. The set of fingerprint liveness detection models includes the target liveness detection model to be deployed and at least one reference liveness detection model trained for diagnostic comparison. The fake fingerprint samples that are simultaneously missed as real fingers by the set of fingerprint liveness detection models and stably reproduce the missed detection in multiple training rounds and / or multiple independent training instances are identified as the detection blind zone samples. For each sample in the detection blind zone sample set, calculate its ridge coverage, average brightness, and grayscale standard deviation. The range of values ​​for the sample set of the detection blind zone in the dimensions of ridge coverage, average brightness, and grayscale standard deviation is taken, and the Cartesian product of the three dimension ranges is taken as the blind zone image range.

3. The method according to claim 1, characterized in that, The construction of a generative adversarial network supporting the sensor's native resolution, and the use of the generative adversarial network to synthesize a spurious fingerprint image at the sensor's native resolution, includes: Using StyleGAN2 as the generator architecture, when the target output resolution of its resolution chain is the native resolution of the sensor, the type of the native resolution of the sensor is determined. If the sensor's native resolution is a non-power-of-2 square resolution, the starting base resolution of the resolution chain is extended from a fixed value to a selectable odd or non-power-of-2 value, so that the resolution chain reaches the sensor's native resolution after being upsampled by 2 times at each level; if the sensor's native resolution is a non-square rectangular resolution, the resolution chain is started with a preset height and width base resolution, each synthesis level carries an axis-by-axis upsampling factor, the height direction is upsampled by 2 times at each level, and the width direction is anisotropically amplified through selective axis-by-axis resampling, so that the resolution chain finally reaches the sensor's native resolution; The selection of the initial substrate resolution is used to ensure the generator's ability to model the global structure of the fingerprint.

4. The method according to claim 1, characterized in that, Based on the blind zone image interval, the synthesized spurious fingerprint image is subjected to image filtering to select synthetic samples that meet the image filtering conditions as synthetic samples that hit the blind zone. The image filtering criteria are determined based on the blind spot image interval, including: Calculate the ridge coverage, average brightness, and grayscale standard deviation for each synthetic spurious fingerprint image; Determine whether the synthesized spurious fingerprint image meets preset conditions, the preset conditions being: the ridge coverage is not lower than the lower bound of the coverage of the blind area image interval, the average brightness falls into the brightness range of the blind area image interval, and the grayscale standard deviation is not lower than the lower bound of the standard deviation of the blind area image interval. The synthetic spurious fingerprint image that meets the preset conditions is used as the synthetic sample for the hit blind zone.

5. The method according to claim 1, characterized in that, The step of using the synthetic samples that hit the blind zone as attack samples and injecting them into the training set of the liveness detection model according to a preset sampling density to perform targeted blind spot training on the liveness detection model includes: The sampling budget for attack samples in the training set is divided into multiple independent sampling segments, including real low-coverage attack segments, blind zone synthetic attack segments, and other real attack segments. Each segment is sampled independently according to the set budget ratio. A main classifier and a source classifier are set up in parallel after the shared backbone network of the liveness detection model. The source classifier is connected to the shared backbone network via a gradient inversion layer. The model is trained using the sum of the main task classification loss and the source classifier classification loss as the total loss. The gradient inversion layer is used to invert the gradient of the source classifier classification loss during backpropagation, and the inverted gradient is backpropagated to the shared backbone network. The source classifier is discarded after training is completed.

6. The method according to claim 1, characterized in that, The preset sampling density ensures that the density of the synthetic samples that hit the blind zone in the training batch is higher than the density when they are randomly mixed into the training set.

7. The method according to claim 1, characterized in that, After completing the targeted blind spot training, the trained liveness detection model is used to perform liveness detection on the spurious fingerprint test samples. When a new, continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot filling, including: After each round of targeted blind patching training, the current liveness detection model is evaluated on the validation set or blind zone diagnosis set to locate the set of spurious fingerprint samples that are still being missed. If there are still samples that are continuously missed, the profile features of the sample set are recalculated and the value range is taken to obtain the updated blind zone profile interval. Based on the updated blind zone profile interval, the steps of synthesizing spurious fingerprint images, profile filtering and targeted blind spot training are re-executed. Repeat the above process until no new continuously missed samples are detected.

8. A fingerprint liveness detection data completion system, characterized in that, include: The blind zone localization and image quantization module is used to locate the detection blind zone samples of the fingerprint liveness detection model, form a detection blind zone sample set, extract the image features of each sample in the detection blind zone sample set, and determine the blind zone image interval based on the image features. A native resolution synthesis module is used to construct a generative adversarial network that supports the native resolution of the sensor, and to synthesize a spurious fingerprint image at the native resolution of the sensor using the generative adversarial network. The image filtering module is used to perform image filtering on the synthesized spurious fingerprint image based on the blind zone image interval, and select the synthesized samples that meet the image filtering conditions as the synthesized samples that hit the blind zone. The image filtering criteria are determined based on the blind zone image interval; The targeted injection training module is used to inject the synthetic samples that hit the blind zone as attack samples into the training set of the liveness detection model according to a preset sampling density, and to perform targeted blind-filling training on the liveness detection model. The iterative blind spot filling module is used to perform liveness detection on the spurious fingerprint test samples using the trained liveness detection model after the targeted blind spot filling training is completed. When a new continuously missed sample is detected, the blind spot image interval is updated and the above steps are repeated to iteratively complete the fingerprint liveness detection data blind spot filling. The sensor's native resolution includes a non-power-of-2 square resolution or a non-square rectangular resolution; the image features include ridge coverage, average brightness, and grayscale standard deviation; the detection blind zone sample is a spurious fingerprint sample that is missed as a real finger by both the target liveness detection model and at least one reference liveness detection model on the validation set or blind zone diagnostic set under the same evaluation protocol and fixed decision threshold, and whose missed detection is stably reproduced in multiple training rounds and / or multiple independent training instances.

9. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a computer program stored in the memory to implement the method of any one of claims 1 to 7.

Images