Fingerprint anti-counterfeiting neural network training method, fingerprint anti-counterfeiting method and fingerprint anti-counterfeiting device

Abstract

The embodiment of the invention provides a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and a fingerprint anti-counterfeiting device, and the method comprises the steps: obtaining a plurality of groups of training data, each group of training data comprising first native domain data, second native domain data and third native domain data; multiple groups of training data are used to train the initial classification network to obtain a target classification network, and the initial classification network comprises a fusion sub-network and a classification sub-network; the fusion sub-network is used for generating a first fingerprint matching pair based on the feature description matrixes of the first native domain data and the second native domain data, and generating a second fingerprint matching pair based on the feature description matrixes of the first native domain data and the third native domain data; and the classification sub-network is used for performing fingerprint classification identification based on the first fingerprint matching pair and the second fingerprint matching pair. According to the fingerprint identification method and device, the problem that the identification accuracy of true and false fingerprints is low in the prior art is solved, and the effect of improving the identification accuracy of the true and false fingerprints is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of fingerprint recognition, and in particular, to a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and a fingerprint anti-counterfeiting device. Background Art

[0002] In daily life, fingerprint recognition technology is often used to authenticate users, such as unlocking smart devices, logging into applications, and making online payments. In recent years, various types of sensors such as capacitive, optical, and ultrasonic sensors can be used to collect fingerprint data.

[0003] In the field of fingerprint anti-counterfeiting, the use of neural network models can improve the recognition rate of real and fake fingerprints. However, the current neural network model training usually uses fingerprint data collected by one type of sensor. Since there are many types of sensors for collecting fingerprints on the market, different neural network models need to be trained for different types of sensors. The training of neural network models is complicated and there is a problem of wasting resources. In addition, the trained neural network model can only recognize fingerprints collected by one type of sensor, and cannot recognize fingerprint data collected by other types of sensors, which limits its scope of application.

[0004] In addition, when collecting fingerprint data, it is usually affected by the environment (such as temperature fluctuations, film interference, changes in finger status, etc.). Directly using the fingerprint data collected by the sensor to train the neural network model will affect the training effect of the neural network model, resulting in a lower recognition accuracy rate for the trained neural network model.

[0005] There is currently no effective solution to the above problems. Summary of the invention

[0006] The embodiments of the present invention provide a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and a device, so as to at least solve the problem of low accuracy in identifying genuine and fake fingerprints in the related art.

[0007] According to one embodiment of the present invention, a training method for a fingerprint anti-counterfeiting neural network is provided, comprising: obtaining multiple groups of training data, each group of training data comprising: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when a fake finger is used instead of the real finger for fingerprint verification; using the multiple groups of training data to train an initial classification network to obtain a target classification network, wherein the initial classification network comprises a fusion subnetwork and a classification subnetwork, for each group of training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

[0008] In an exemplary embodiment, multiple sets of training data are used to train an initial classification network to obtain a target classification network, including: using multiple sets of training data to iteratively train the initial classification network until the number of times the initial classification network is trained reaches a target number of times, or when the loss value output by the target loss function of the initial classification network meets a preset training end condition, the training is terminated to obtain the target classification network; wherein the initial classification network also includes a preprocessing subnetwork, and the preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into a corresponding feature description matrix; the loss value output by the target loss function is a loss value determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork; the first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the real label corresponding to the feature description matrix input by the classification subnetwork.

[0009] In an exemplary embodiment, multiple sets of training data are used to iteratively train the initial classification network, including: training the initial classification network for the kth time through the following steps, wherein k is a positive integer: obtaining the kth set of training data used for the kth training of the initial classification network from the multiple sets of training data; inputting the first native domain data, the second native domain data, and the third native domain data in the kth set of training data into a preprocessing subnetwork in the classification network trained for the k-1th time, respectively obtaining a first feature description matrix, a second feature description matrix, and a third feature description matrix, wherein the initial classification network trained for the 0th time is an untrained initial classification network, and the preprocessing subnetwork is used to convert native domain data obtained by sensors of different types into corresponding feature description matrices; inputting the first feature description matrix, the second feature description matrix, and the third feature description matrix into a feature extraction subnetwork in the classification network trained for the k-1th time for dimensionality reduction processing, respectively obtaining a fourth feature description matrix, a fifth feature description matrix, and a sixth feature description matrix; inputting the fourth feature description matrix and the fifth feature description matrix into a fusion subnetwork in the classification network trained for the k-1th time, respectively obtaining a first fingerprint matching pair of the kth training;

[0010] Input the fourth feature description matrix and the sixth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain a second fingerprint matching pair for the kth time;

[0011] Inputting the first fingerprint matching pair of the k-th training and the second fingerprint matching pair of the k-th training into the classification subnetwork in the classification network of the k-1-th training to obtain the prediction label of the k-th training, wherein the prediction label of the k-th training is used to represent the authenticity recognition result of the real finger fingerprint corresponding to the second native domain data in the k-th group of training data by the classification network of the k-1-th training, and the authenticity recognition result of the fake finger fingerprint corresponding to the third native domain data in the k-th group of training data by the classification network of the k-1-th training;

[0012] According to the first feature description matrix, the second feature description matrix and the third feature description matrix and the known feature description matrices corresponding to the first native domain data, the second native domain data and the third native domain data in the kth group of training data, the loss value output by the first loss function of the kth training is determined; according to the predicted label of the kth training and the known true label of the kth training, the loss value output by the second loss function of the kth training is determined; according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, the loss value output by the target loss function of the kth training is determined; when k is less than the target number, or when the loss value output by the target loss function of the kth training does not meet the training end condition, the values ​​of the parameters in the initial classification network of the k-1th training are adjusted to obtain the initial classification network of the kth training.

[0013] In an exemplary embodiment, the objective loss function is:

[0014] loss = α × loss MSE +β×loss BCE

[0015] Among them, α and β are preset values, and loss MSE is the first loss function of the preprocessing sub-network, loss BCE is the second loss function of the classification subnetwork;

[0016] The classification network also includes a feature extraction subnetwork. The preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a corresponding feature description matrix and input it into the feature extraction subnetwork. The feature extraction subnetwork is used to perform dimensionality reduction processing on the input feature description matrix and input the matrix obtained after dimensionality reduction processing into the fusion subnetwork.

[0017] In an exemplary embodiment, the loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0018]

[0019] loss MSE =loss MSE_1 +loss MSE_2 +loss MSE_3

[0020] Among them, t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix of the output of the preprocessing sub-network, y(i,j) is the known feature description matrix of the native domain data of the input of the preprocessing sub-network, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1 The input data of the preprocessing sub-network is the loss value of the first native domain data, loss MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data, loss MSE_3 is the loss value of the third native domain data whose input data is the preprocessing sub-network.

[0021] In an exemplary embodiment, the loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0022] loss BCE = -y×log(x)-(1-y)×log(1-x)

[0023] Wherein, y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification subnetwork.

[0024] According to another embodiment of the present invention, a fingerprint anti-counterfeiting method is provided, comprising: obtaining native domain data of a fingerprint to be detected; preprocessing the native domain data of the fingerprint to be detected by a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; extracting features of the seventh feature description matrix by a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; determining a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores multiple feature description matrices of a real finger when entering a fingerprint template; fusing the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; inputting the fused feature matrix into a classification subnetwork in a target classification network to obtain an authenticity recognition result of the fingerprint to be detected, wherein the authenticity recognition result is used to indicate whether the fingerprint to be detected is a fingerprint of a real finger or a fingerprint of a fake finger.

[0025] In an exemplary embodiment, before determining a ninth feature description matrix that matches the eighth feature description matrix in the fingerprint template feature library, the method further includes: obtaining native domain data of a real finger when entering the fingerprint template; preprocessing the native domain data using a preprocessing subnetwork in a target classification network to obtain a feature description matrix of the native domain data; performing dimensionality reduction processing on the feature description matrix using a feature extraction subnetwork in a target classification network to obtain a feature description matrix after dimensionality reduction; and storing the feature description matrix after dimensionality reduction in the fingerprint template feature library.

[0026] According to another embodiment of the present invention, there is provided a training method for fingerprint anti-counterfeiting neural network, including: a first acquisition module, used to acquire multiple groups of training data, each group of training data includes: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when the fake finger is used instead of the real finger for fingerprint verification; a training module, used to train the initial classification network using the multiple groups of training data to obtain a target classification network, wherein the initial classification network includes a fusion subnetwork and a classification subnetwork, for each group of training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on the feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on the feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

[0027] According to another embodiment of the present invention, a fingerprint anti-counterfeiting device is provided, comprising: a second acquisition module, used to acquire native domain data of a fingerprint to be detected; a preprocessing module, used to preprocess the native domain data of the fingerprint to be detected through a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; a feature extraction module, used to extract features from the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; a determination module, used to determine a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when entering a fingerprint template; a fusion module, used to fuse the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; and an identification module, used to input the fused feature matrix into a classification subnetwork in the target classification network to obtain an authenticity identification result of the fingerprint to be detected, wherein the authenticity identification result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

[0028] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0029] According to another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0030] Through the present invention, the first native domain data of the real finger collected when the real finger is recorded into the fingerprint template, the second native domain data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data collected when the fake finger is used to replace the real finger for fingerprint verification are used as training data to train the initial classification network;

[0031] The present application adopts a sample feature fusion method, and fuses the features of the first native domain data and the second native domain data through a fusion sub-network to generate a first fingerprint matching pair, and fuses the features of the first native domain data and the third native domain data to generate a second fingerprint matching pair;

[0032] This application uses the fused first fingerprint matching pair and second fingerprint matching pair classification subnet for training. It can reduce the impact of environmental factors when collecting fingerprint data on the training neural network model, achieving the purpose of preventing environmental factors from affecting the training effect of the neural network model, thereby solving the problem in the related art that the recognition accuracy of the trained neural network model is low due to the environmental influence when collecting fingerprint data, thereby achieving the effect of improving the recognition accuracy of true and false fingerprints.

[0033] In addition, since the preprocessing subnetwork in the application can convert the native domain data collected by the fingerprint sensor into a feature description matrix. The trained target classification network can then identify fingerprints collected by various types of sensors, solving the problem in the prior art that different neural network models need to be trained for different types of sensors, resulting in complex neural network model training and waste of resources, as well as the trained neural network model being unable to identify fingerprint data collected by other types of sensors, and the application scope is limited. The target classification network of the present application can identify fingerprints collected by various types of sensors, with a wider range of applications and more resource savings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a hardware structure block diagram of a fingerprint anti-counterfeiting neural network training method and a mobile terminal of the fingerprint anti-counterfeiting method according to an embodiment of the present invention;

[0035] Figure 2 is a flow chart of a fingerprint anti-counterfeiting neural network training method according to an embodiment of the present invention;

[0036] Figure 3 is an overall architecture diagram of a classification network according to an embodiment of the present invention;

[0037] Figure 4 is a flow chart of authenticity identification of a fingerprint to be detected according to an embodiment of the present invention;

[0038] Figure 5 is a schematic diagram of a similarity metric classification network architecture according to an embodiment of the present invention;

[0039] Figure 6 is a deep convolutional neural network feature extraction structure table according to an embodiment of the present invention;

[0040] Figure 7 is a schematic diagram of an overall process according to an embodiment of the present invention;

[0041] Figure 8 is a schematic diagram of comparing the measured benefits of ultrasonic anti-counterfeiting according to an embodiment of the present invention;

[0042] Fig. 9is a schematic diagram of comparing benefits of similarity measurement mechanisms according to an embodiment of the present invention;

[0043] Fig.10 4 is a structural block diagram of a device for identifying genuine and fake fingerprints according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0045] Native domain data: refers to the digital matrix obtained by converting the collected fingerprint signal through analog-to-digital conversion when using the sensor to collect fingerprints. Native domain data is raw data that has not been preprocessed, and the fingerprint texture information cannot be directly observed. However, native domain data contains all the original information that has not been filtered out, and is more suitable for feature extraction with deep learning technology.

[0046] FLRR: True Finger False Rejection Rate.

[0047] SAR: false fingerprint false acceptance rate.

[0048] Conv: Convolution operator in deep convolutional neural networks.

[0049] Avg Pooling: Average pooling, an average-based downsampling operator.

[0050] Depth-wise Conv: Depthwise Separable Convolutions in Deep Convolutional Neural Networks.

[0051] Transformer: A deep learning network architecture with attention mechanism.

[0052] MLP: Multilayer Perceptron, a classic network architecture.

[0053] Binary Cross Entropy: Binary cross entropy, a classic classification loss function.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0055] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a fingerprint anti-counterfeiting neural network training method and a mobile terminal of the fingerprint anti-counterfeiting method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0056] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the training method of the fingerprint anti-counterfeiting neural network in the embodiment of the present invention, and the computer program corresponding to the fingerprint anti-counterfeiting method. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0057] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0058] In this embodiment, a training method for a fingerprint anti-counterfeiting neural network running on the above mobile terminal is provided. Figure 2 FIG. 1 is a flow chart of a training method for a fingerprint anti-counterfeiting neural network according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0059] Step S202, obtaining multiple sets of training data, each set of training data includes: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when the fake finger is used instead of the real finger for fingerprint verification;

[0060] The multiple sets of training data include native domain data collected by at least two types of sensors, for example, native domain data collected by multiple types of sensors such as capacitance, optics, and ultrasonic waves.

[0061] Real finger fingerprint template entry means: put the real finger on the fingerprint recognition device (the fingerprint recognition device is equipped with a sensor to collect fingerprint data) and follow the instructions of the device to enter the fingerprint template. After the entry is completed, the system will generate a unique fingerprint template for subsequent fingerprint recognition verification.

[0062] Fingerprint verification: Compare the collected fingerprint data with the previously entered fingerprint template to determine whether the user's identity matches.

[0063] The above-mentioned fake fingers include but are not limited to fake fingerprints made by various processes including printing paper, silicone, wood glue, latex, etc.

[0064] Step S204, use multiple groups of training data to train the initial classification network to obtain a target classification network, wherein the initial classification network includes a fusion subnetwork and a classification subnetwork, and for each group of training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on the feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on the feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

[0065] Optionally, the executor of the above steps may be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least an image acquisition device and a data processing device, wherein the image acquisition device may include a graphics acquisition module such as a camera, and the data processing device may include a computer, a mobile phone or other terminal, but is not limited thereto.

[0066] The initial classification network includes: a preprocessing subnetwork, a fusion subnetwork, a feature subnetwork and a classification subnetwork. The above subnetworks can select a neural network model in the prior art, for example, a convolutional neural network. The initial classification network training is iteratively trained using the above multiple sets of training data. The number of iterative trainings can be a target number of times. When the number of trainings reaches the target number of times, the training is terminated to obtain the target classification network. Alternatively, when the loss value output by the target loss function meets the preset training termination condition, the training is terminated to obtain the target classification network.

[0067] The loss value output by the target loss function includes two parts: the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork. The target loss function is as follows:

[0068] loss = α × loss MSE +β×loss BCE

[0069] Among them, α and β are preset values, and loss MSE is the first loss function of the preprocessing sub-network, loss BCE is the second loss function of the classification sub-network.

[0070] The preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into a corresponding feature description matrix, and the fingerprint sensor is one of a capacitive fingerprint sensor, an optical fingerprint sensor, or an ultrasonic fingerprint sensor. The first loss function represents the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the true label corresponding to the feature description matrix input by the classification subnetwork.

[0071] The loss value output by the first loss function of the above preprocessing subnetwork is determined by the following formula:

[0072]

[0073] loss MSE =loss MSE_1 +loss MSE_2 +loss MSE_3

[0074] Among them, t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix of the output of the preprocessing sub-network, y(i,j) is the known feature description matrix of the native domain data of the input of the preprocessing sub-network, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1The input data of the preprocessing sub-network is the loss value of the first native domain data, loss MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data, loss MSE_3 is the loss value of the third native domain data whose input data is the preprocessing sub-network.

[0075] The loss value output by the second loss function of the above classification subnetwork is determined by the following formula:

[0076] loss BCE = -y×log(x)-(1-y)×log(1-x)

[0077] Wherein, y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification subnetwork.

[0078] Take the k-th training of the initial classification network as an example, k is a positive integer, Figure 3 It is the overall architecture diagram of the classification network, including the following:

[0079] A kth group of training data for the kth training of the initial classification network is obtained from multiple groups of training data. The kth group of training data may be fingerprint data collected using the same or different types of sensors, and specifically may include the following fingerprint data: a first native domain data collected when a real finger enters a fingerprint template on a smart device, a second native domain data collected when a real finger performs fingerprint verification on a smart device, and a third native domain data collected when a fake finger replaces a real finger for fingerprint verification on a smart device (a fake finger performs a fingerprint attack, impersonating a real finger for fingerprint verification).

[0080] According to the information collection characteristics of the sensor, each time the sensor collects a fingerprint, a fixed-scale native domain data (including the first native domain data, the second native domain data, and the third native domain data) with a specific number of channels N (an integer greater than or equal to 1) can be obtained for each press, and the resolution is [Height, Width, N]. Taking the native domain data collected by the ultrasonic sensor as an example, the native domain data includes two channels, the real part and the imaginary part, and the scale of the native domain data can be [80, 80, 2] (this is just an example, and the specific scale of the native domain data can be determined according to the actual situation).

[0081] In order to organize the training effectively, the fingerprint template recorded by the real finger is used as the guidance of the training initial classification network to identify the real fingerprint and the fake fingerprint. The first native domain data collected when the real finger is recorded in the fingerprint template, the second native domain data collected when the real finger is used for fingerprint verification, and the third native domain data collected when the fake finger is used instead of the real finger for fingerprint verification (fingerprint attack) are a set of training data (also called a native domain triplet, with a feature scale of [3, Height, Width, N]).

[0082] Input the first native domain data, the second native domain data and the third native domain data in the kth group of training data into the preprocessing subnetwork in the classification network trained for the k-1th time, and obtain the first feature description matrix, the second feature description matrix and the third feature description matrix respectively, wherein the initial classification network trained for the 0th time is an untrained initial classification network, and the preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a feature description matrix;

[0083] The preprocessing subnetwork can be a network obtained by combining a convolution layer and a pooling layer, which can realize the conversion of the native domain data acquired by the sensor into a feature description matrix (the feature description matrix is ​​a digital matrix obtained by enhancing the conversion of the image domain fingerprint image). The preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into the corresponding feature description matrix.

[0084] Relying on the specified known feature description matrices of different types of sensors as external guidance, the preprocessing subnetwork is required to output a feature description matrix that is as consistent as possible with the given known feature description matrix, thereby achieving the purpose of identifying fingerprint data collected by multiple types of sensors through the same classification network.

[0085] Specifically, for example, the capacitive sensor uses the capacitive preprocessing result (known feature description matrix of the capacitive sensor) as the intermediate guide, requiring the preprocessing sub-network output feature description matrix to be as consistent as possible with the capacitive preprocessing result. The optical sensor uses the optical preprocessing result (known feature description matrix of the optical sensor) as the intermediate guide, requiring the preprocessing sub-network output feature description matrix to be as consistent as possible with the optical preprocessing result.

[0086] The preprocessing sub-network is constrained by the first loss function so as to convert the native domain data acquired by the fingerprint sensor into a feature description matrix corresponding to the native domain data.

[0087] Input the first feature description matrix, the second feature description matrix and the third feature description matrix into the feature extraction subnetwork in the classification network trained for the k-1th time for dimensionality reduction processing, and obtain the fourth feature description matrix, the fifth feature description matrix and the sixth feature description matrix respectively;

[0088] The classification network also includes a feature extraction subnetwork. The preprocessing subnetwork inputs the output feature description matrix (including the above-mentioned first feature description matrix, second feature description matrix and third feature description matrix) into the feature extraction subnetwork. The feature extraction subnetwork performs dimensionality reduction processing on the input matrix and inputs the matrix obtained after the dimensionality reduction processing into the fusion subnetwork (the matrix obtained after the dimensionality reduction processing includes the above-mentioned fourth feature description matrix, fifth feature description matrix and sixth feature description matrix).

[0089] The above-mentioned feature extraction subnetwork can use the deep separable convolution technology, combine the convolution layer and the pooling layer, and construct a feature escape module to achieve large-scale dimensionality reduction and concentration of the feature description matrix, retain the anti-counterfeiting identifiable information to the greatest extent, and filter out clutter and interference.

[0090] Input the fourth feature description matrix and the fifth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain the first fingerprint matching pair of the kth training; input the fourth feature description matrix and the sixth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain the second fingerprint matching pair of the kth training;

[0091] Inputting the first fingerprint matching pair of the k-th training and the second fingerprint matching pair of the k-th training into the classification subnetwork in the classification network of the k-1-th training to obtain the prediction label of the k-th training, wherein the prediction label of the k-th training is used to represent the authenticity recognition result of the real finger fingerprint corresponding to the second native domain data in the k-th group of training data by the classification network of the k-1-th training, and the authenticity recognition result of the fake finger fingerprint corresponding to the third native domain data in the k-th group of training data by the classification network of the k-1-th training;

[0092] The above fusion subnetwork and classification subnetwork use fully connected layers and softmax to output the predicted label of the current input, distinguish between the two input modal forms of real finger + real finger fingerprint template and fake finger + real finger fingerprint template, and realize the function of real and fake fingerprint identification.

[0093] According to the first feature description matrix, the second feature description matrix and the third feature description matrix and the known feature description matrices corresponding to the first native domain data, the second native domain data and the third native domain data in the kth group of training data, the loss value output by the first loss function of the kth training is determined; according to the predicted label of the kth training and the known true label of the kth training, the loss value output by the second loss function of the kth training is determined; according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, the loss value output by the target loss function of the kth training is determined; when k is less than the target number, or when the loss value output by the target loss function of the kth training does not meet the training end condition, the values ​​of the parameters in the initial classification network of the k-1th training are adjusted to obtain the initial classification network of the kth training.

[0094] As an optional implementation, the initial classification network is iteratively trained using multiple sets of training data until the number of training times for the initial classification network reaches a target number of times, or when the loss value output by the target loss function of the initial classification network meets a preset training end condition, the training is terminated to obtain the target classification network;

[0095] The above target times are to configure the basic training parameters. Using AdamW as the training optimizer, for example, 200 epochs can be trained.

[0096] The loss value output by the target loss function is a loss value determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork;

[0097] The first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the true label corresponding to the feature description matrix input by the classification subnetwork.

[0098] Iteratively train the initial classification network using multiple sets of training data until the number of times the initial classification network is trained reaches a target number of times, or when the loss value output by the target loss function of the initial classification network meets a preset training end condition, the training is terminated to obtain the target classification network;

[0099] The loss value output by the target loss function is a loss value determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork;

[0100] The first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix corresponding to the sensor type of the native domain data input by the preprocessing subnetwork. The second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the true label corresponding to the feature description matrix input by the classification subnetwork.

[0101] Specifically, the above objective loss function is:

[0102] loss = α × loss MSE +β×loss BCE

[0103] Among them, α and β are preset values, and loss MSE is the first loss function of the preprocessing subnetwork included in the classification network, loss BCE is the second loss function of the classification subnetwork;

[0104] The gradient backpropagation method combined with the AdamW optimizer is used to simultaneously fine-tune and update all sub-network parameters in the initial classification network, thereby gradually reducing the loss function. The loss function used is a linear combination of the preprocessing approximation loss function and the final anti-counterfeiting classification function, where the values ​​of α and β can be set according to actual conditions, for example, α is 0.1 and β is 0.9.

[0105] Each set of native domain data included in each training data set is acquired through one of the following sensors: capacitive sensor, optical sensor, and ultrasonic sensor;

[0106] In order to enable the preprocessing subnetwork to universally process native domain data collected by various types of sensors including capacitance, optics, and ultrasound, we first constructed a mean square error (MSE) loss function between the feature description matrix generated by a specific preprocessing pipeline and the feature description matrix output by the preprocessing subnetwork. This function determines whether the preprocessing subnetwork can accurately output the feature description matrix of the corresponding type of sensor by calculating the global sum of squares of the data difference between the feature description matrix output by the preprocessing subnetwork and the feature description matrix obtained based on specific preprocessing. The smaller the value of this function, the more consistent the result of the constructed preprocessing network output is with the feature description matrix obtained by specific preprocessing.

[0107] The loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0108]

[0109] loss MSE =loss MSE_1+loss MSE_2 +loss MSE_3

[0110] Among them, t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix of the output of the preprocessing sub-network, y(i,j) is the known feature description matrix of the native domain data of the input of the preprocessing sub-network, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1 The input data of the preprocessing sub-network is the loss value of the first native domain data, loss MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data, loss MSE_3 is the loss value of the third native domain data whose input data is the preprocessing sub-network.

[0111] The loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0112] loss BCE = -y×log(x)-(1-y)×log(1-x)

[0113] Wherein, y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification subnetwork.

[0114] As an optional implementation, when the number of epochs corresponding to the training cycle reaches the preset value of 200, the optimization is stopped, the network parameters stop updating, and the network training is completed. Experience shows that the true and false fingerprint data matching pairs constructed using a certain scale of native domain triple data can improve the effective information filtering ability of the preprocessing sub-network after the above training; on the other hand, the feature extraction network can also greatly reduce the feature dimensions that need to be pre-stored in practical applications, improving ease of use; and finally, the iterative training mode of true and false data can also make the network finally have effective anti-counterfeiting performance.

[0115] After the training task is completed, the preprocessing subnetwork and feature extraction subnetwork are called for the fingerprint template in the fingerprint template registration phase of the real finger, and each fingerprint template is compressed into a 4*4*10 feature description matrix and stored in the fingerprint template feature library. Compared with storing the native domain data 80*80*2, the storage space requirement is only 1 / 80 of the original, saving storage space.

[0116] According to another embodiment of the present invention, a fingerprint anti-counterfeiting method is provided, comprising: obtaining native domain data of a fingerprint to be detected; preprocessing the native domain data of the fingerprint to be detected by a preprocessing subnetwork in the target classification network to obtain a seventh feature description matrix; extracting features from the seventh feature description matrix by a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix;

[0117] like Figure 4 The flowchart of authenticity identification of the fingerprint to be detected shown also includes the following steps:

[0118] Step S401, determining a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of the real finger when recording the fingerprint template;

[0119] Among them, the fingerprint template feature library can be obtained in the following ways:

[0120] Obtain the native domain data of the real finger when entering the fingerprint template; use the preprocessing sub-network in the target classification network to preprocess the native domain data to obtain the feature description matrix of the native domain data;

[0121] The feature description matrix is ​​subjected to dimensionality reduction processing using a feature extraction subnetwork in a target classification network to obtain a feature description matrix after dimensionality reduction; and the feature description matrix after dimensionality reduction is stored in the fingerprint template feature library.

[0122] Step S404, fusing the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix;

[0123] Step S405, inputting the fused feature matrix into the classification sub-network in the target classification network to obtain the authenticity identification result of the fingerprint to be detected, wherein the authenticity identification result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

[0124] Take the scale of the native domain data collected by the ultrasonic sensor [80,80,2] as an example. The following is a hierarchical introduction to the data flow of feature descriptor extraction and final fusion classification, such as Figure 5 The similarity metric classification network architecture diagram shown includes the following steps:

[0125] Step 501, the scale of the native domain data of the fingerprint template registered by the real finger and the native domain data of the finger to be detected are both 80*80*2, and the native domain data of the fingerprint template registered by the real finger and the native domain data of the finger to be detected are input into the preprocessing sub-network, where the length is 80, the width is 80, and it contains two channels of real part and imaginary part.

[0126] Step 502, output two feature description matrices with a scale of 80*80*1 through the preprocessing subnetwork (including the seventh feature description matrix obtained by preprocessing the native domain data of the fingerprint to be detected by the above-mentioned preprocessing subnetwork, and the feature description matrix obtained by preprocessing the native domain data of the real finger when entering the fingerprint template when constructing the fingerprint template feature library by the preprocessing subnetwork).

[0127] The preprocessing subnetwork can realize fingerprint extraction and enhancement of native domain data collected by different types of sensors through training and guidance of subsequent data, thereby significantly reducing the dependence of the fingerprint preprocessing process and greatly improving the scope of application of the solution.

[0128] Step 503, the feature extraction subnetwork uses a depth-separable convolution block, combined with a pooling layer, to extract feature descriptors from the above 80*80*1 feature description matrix for the template and the sample to be classified, respectively, to obtain two feature description matrices of scale 4*4*10 (including the eighth feature description matrix mentioned above, and when constructing the fingerprint template feature library, the feature extraction subnetwork extracts features from the feature description matrix of the real finger after preprocessing when entering the fingerprint template). The feature extraction subnetwork reduces the data dimension while eliminating noise and redundant information, significantly alleviating the pressure of feature storage.

[0129] Step 504: The fusion sub-network uses feature concatenation and fusion technology to concatenate the two 4*4*10 feature description matrices through channels, and constructs a feature matrix (also called a fingerprint matching pair) with a scale of 4*4*20.

[0130] The fingerprint recognition mechanism of the fingerprint recognition system is used to index the feature description matrix of the same finger in the corresponding registration link, and perform feature fusion to form a 4*4*20 fusion feature matrix.

[0131] Step 505: In the final classification subnetwork, the 4*4*20-dimensional feature matrix is ​​reorganized through continuous convolution blocks, converted into a 128-dimensional descriptor, and mapped to a 2-dimensional true or false probability using full connectivity. Using nonlinear mapping, the 2-dimensional output is converted into a true or false probability value.

[0132] The fusion feature matrix is ​​sent to the trained fusion sub-network and classification sub-network to obtain the probability that the data is a real finger matching fusion feature and a fake fingerprint matching fusion feature, and the authenticity recognition result is generated based on the probability result.

[0133] In the application phase, the preprocessing sub-network and feature extraction sub-network are called to compress each unlocking and pressing data into a 4*4*10 feature description matrix. Figure 6The figure shows the structure of the deep convolutional neural network feature extraction. Based on deep separable convolution, full connection and nonlinear mapping, the authenticity recognition result can be obtained, including the score corresponding to the real finger.

[0134] Since the valley ridges of the same finger have 3D structural stability under certain conditions, the live pressing frame data should maintain relative consistency with the registered real finger in 3D structure. In addition, in order to improve the accuracy of 3D structure matching and avoid the tedious and inefficient manual traditional similarity feature extraction, a convolutional neural network (CNN) is used to build a universal fingerprint anti-counterfeiting network.

[0135] like Figure 7 The figure shows the overall process diagram, including:

[0136] Step S701, training data collection, including native domain data collected when a real finger is registered as a fingerprint template, second native domain data collected when a real finger is verified, and third native domain data collected when a fake finger is used instead of the real finger for fingerprint verification;

[0137] Step S702, building a segmented initial classification network based on CNN, including a preprocessing subnetwork, a feature extraction subnetwork, a fusion subnetwork, and a classification subnetwork;

[0138] Step S703, optimizing the overall parameters of the initial classification network using the training data;

[0139] Step S704: on the basis of capturing the native domain data of the fingerprint to be detected, calling the preprocessing sub-network and the feature extraction sub-network for each registered fingerprint template to obtain the feature description matrix of the native domain data of the fingerprint to be detected;

[0140] Step S705, indexing the feature description matrix of the fingerprint template that can be matched by the traditional fingerprint matching algorithm;

[0141] In actual use, during the registration phase, the preprocessing sub-network and feature extraction sub-network are called for each fingerprint template registered by a real finger to obtain the feature description matrix of the fingerprint template, thereby retaining the fingerprint template features of each press at a relatively low storage cost;

[0142] Step 706, the feature description matrix of the paired fingerprint template and the feature description matrix of the fingerprint to be detected are fused and sent to the trained special classification sub-network to obtain the authenticity identification result of the fingerprint to be detected, thereby completing the identification.

[0143] In response to the lack of universal fingerprint anti-counterfeiting technology, this application proposes a fingerprint anti-counterfeiting solution based on template and sample fusion features. It models the physical dimension of "the true and false separability of the fingerprint template and the unlocking and pressing native domain fusion input signal", and uses the combined data of true and false matching pairs to guide the training of the template-sample fusion information authenticity identification network. With low computational complexity and zero external hardware sensor cost, it achieves high-precision detection and anti-counterfeiting for highly realistic fake fingerprints.

[0144] In such Figure 8 The comparison of the actual measured benefits of ultrasonic anti-counterfeiting in this application and Fig. 9 The comparison of the benefits of similar measurement mechanisms of this application is shown, using the false liveness rejection rate (FLRR) and the false pass rate (SAR) of fake fingerprints as performance evaluation indicators. Among them, the parameter No.Liveness Rejection represents the number of times the real finger is rejected, the parameter No.Liveness Pressing represents the number of times the real finger is pressed, the parameter No.SpoofAccept represents the number of times the fake finger is mistakenly accepted, and the parameter No.Spoof Attack represents the total number of attacks of the fake finger.

[0145]

[0146] Taking ultrasonic fingerprint anti-counterfeiting as an example, six 3D stereo images that can be cracked with a high probability were produced. The materials used included gelatin, latex and other fingerprint stereo extraction materials. Five fingers were made for each type of fake fingerprint, and each finger was attacked 30 times, for a total of about 150 attacks. The experimental results show that compared with the current mainstream simple anti-counterfeiting method of manually constructing traditional features for distribution comparison, this solution effectively utilizes the fusion information of fingerprint templates and unlocking samples, combined with the high robustness of artificial intelligence algorithms, and can robustly achieve effective defense against various types of fake fingerprints without significantly affecting the real finger unlocking experience. The average cracking rate has dropped from 81.51% to 7.17%.

[0147] In the above fusion sub-network, preprocessing steps including rotation, affine transformation, displacement, flipping, denoising, enhancement and other methods can also be added. In addition to direct channel splicing, fusion techniques such as channel-by-channel subtraction and spatial (horizontal or vertical) splicing can be used. In addition, in addition to fusing the original data in the original Raw domain, the network can also be used to extract and fuse intermediate features to achieve feature interaction between templates and samples.

[0148] In the feature extraction subnetwork, a specific convolutional neural network is constructed to extract features from the fusion information of the template and the sample. In addition, other CNN (Resnet, Densenet, Unet, etc.) architectures and Transformer architectures, MLP architectures and other classification models can also be used as feature extraction and classification networks for this application.

[0149] In addition to implementing parameter updates and training based on machine learning and deep learning training mechanisms, fixed network parameters can also be manually configured while the network is being built, which can be directly used for chip-side deployment without any training to achieve universal fingerprint anti-counterfeiting.

[0150] By default, the registered template information that can be matched with the texture is used for feature fusion. In addition, on the one hand, the registered template information can be shielded, and only the unlocked sample can be used to achieve independent anti-counterfeiting driven by the artificial intelligence network; on the other hand, the remaining templates that cannot be matched can be collected to construct other information fusion methods.

[0151] This application can achieve the following technical effects:

[0152] According to the physical principle of the echo of fingerprint data collected by ultrasonic under-screen fingerprint, a three-dimensional anti-counterfeiting solution based on the fusion information of the registered template and the unlocking and pressing raw domain is constructed, filling the gap in ultrasonic under-screen fingerprint anti-counterfeiting technology; a segmented network is constructed, and the pre-processing sub-network, feature extraction sub-network and feature fusion classification sub-network architecture are innovatively combined to achieve end-to-end integrated training. The involved pre-processing sub-network can be guided by different system modal data to achieve the pre-processing process fitting and approximation of different fingerprint collection systems to achieve the pre-processing effect. On the basis of fingerprint matching, the template information that needs to be stored is greatly optimized, and the entire template image is replaced by the feature domain descriptor. The memory usage is optimized to 1 / 80 of the original. The registered template and unlocking sample signals are fused in the feature domain, which improves the effectiveness and usability of ultrasonic fingerprint anti-counterfeiting; mainstream artificial intelligence technology is applied to build a convolutional neural network, and the implicit feature extraction method of neural network features is used to replace the manual explicit feature extraction (acoustic impedance, distance, medium characteristics, etc.), and the fingerprint true and false information similarity measurement network of the feature domain is constructed in a directional manner, which further improves the robustness of anti-counterfeiting technology to the system environment. The constructed template-sample fusion feature fingerprint anti-counterfeiting method can be combined with a static anti-counterfeiting method that does not rely on a template, reflecting performance gains.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0154] In this embodiment, a training device for a fingerprint anti-counterfeiting neural network is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0155] Fig.10 is a structural block diagram of a training device for a fingerprint anti-counterfeiting neural network according to an embodiment of the present invention. Fig.10 As shown, the device comprises:

[0156] The first acquisition module 1002 is used to acquire multiple sets of training data, each set of training data includes: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when the fake finger is used instead of the real finger for fingerprint verification;

[0157] The training module 1004 is used to train the initial classification network using multiple groups of training data to obtain a target classification network, wherein the initial classification network includes a fusion subnetwork and a classification subnetwork. For each group of training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on the feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on the feature description matrix of the first native domain data and the third native domain data. The classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

[0158] In an exemplary embodiment, the above-mentioned device is also used to iteratively train the initial classification network using multiple sets of training data until the number of times the initial classification network is trained reaches the target number of times, or when the loss value output by the target loss function of the initial classification network meets the preset training end condition, the training is terminated to obtain the target classification network; wherein, the initial classification network also includes a preprocessing subnetwork, and the preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into a feature description matrix; the loss value output by the target loss function is a loss value determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork; the first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the real label corresponding to the feature description matrix input by the classification subnetwork.

[0159] In an exemplary embodiment, the above-mentioned device is also used to train the initial classification network for the kth time through the following steps, wherein k is a positive integer: obtaining the kth group of training data used for the kth training of the initial classification network from multiple groups of training data; inputting the first native domain data, the second native domain data, and the third native domain data in the kth group of training data into the preprocessing subnetwork in the classification network trained for the k-1th time, respectively obtaining the first feature description matrix, the second feature description matrix, and the third feature description matrix, wherein the initial classification network trained for the 0th time is an untrained initial classification network, and the preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a feature description matrix; inputting the first feature description matrix, the second feature description matrix, and the third feature description matrix into the feature extraction subnetwork in the classification network trained for the k-1th time for dimensionality reduction processing, respectively obtaining the fourth feature description matrix, the fifth feature description matrix, and the sixth feature description matrix; inputting the fourth feature description matrix and the fifth feature description matrix into the fusion subnetwork in the classification network trained for the k-1th time, and obtaining the first fingerprint matching pair of the kth training;

[0160] Input the fourth feature description matrix and the sixth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain a second fingerprint matching pair for the kth time;

[0161] Inputting the first fingerprint matching pair of the k-th training and the second fingerprint matching pair of the k-th training into the classification subnetwork in the classification network of the k-1-th training to obtain the prediction label of the k-th training, wherein the prediction label of the k-th training is used to represent the authenticity recognition result of the real finger fingerprint corresponding to the second native domain data in the k-th group of training data by the classification network of the k-1-th training, and the authenticity recognition result of the fake finger fingerprint corresponding to the third native domain data in the k-th group of training data by the classification network of the k-1-th training;

[0162] According to the first feature description matrix, the second feature description matrix and the third feature description matrix and the known feature description matrices corresponding to the first native domain data, the second native domain data and the third native domain data in the kth group of training data, the loss value output by the first loss function of the kth training is determined; according to the predicted label of the kth training and the known true label of the kth training, the loss value output by the second loss function of the kth training is determined; according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, the loss value output by the target loss function of the kth training is determined; when k is less than the target number, or when the loss value output by the target loss function of the kth training does not meet the training end condition, the values ​​of the parameters in the initial classification network of the k-1th training are adjusted to obtain the initial classification network of the kth training.

[0163] In an exemplary embodiment, the objective loss function is:

[0164] loss = α × loss MSE +β×loss BCE

[0165] Among them, α and β are preset values, and loss MSE is the first loss function of the preprocessing sub-network, loss BCE is the second loss function of the classification subnetwork;

[0166] The classification network also includes a feature extraction subnetwork. The preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a feature description matrix and input it into the feature extraction subnetwork. The feature extraction subnetwork is used to reduce the dimension of the input matrix and input the matrix description matrix obtained after the dimension reduction process into the fusion subnetwork.

[0167] In an exemplary embodiment, the loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0168]

[0169] loss MSE =loss MSE_1+loss MSE_2 +loss MSE_3

[0170] Among them, t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix of the output of the preprocessing sub-network, y(i,j) is the known feature description matrix of the native domain data of the input of the preprocessing sub-network, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1 The input data of the preprocessing sub-network is the loss value of the first native domain data, loss MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data, loss MSE_3 is the loss value of the third native domain data whose input data is the preprocessing sub-network.

[0171] In an exemplary embodiment, the loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0172] loss BCE = -y×log(x)-(1-y)×log(1-x)

[0173] Wherein, y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification subnetwork.

[0174] According to another embodiment of the present invention, a fingerprint anti-counterfeiting device is provided, comprising: a second acquisition module, used to acquire native domain data of a fingerprint to be detected; a preprocessing module, used to preprocess the native domain data of the fingerprint to be detected through a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; a feature extraction module, used to extract features from the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; a determination module, used to determine a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when entering a fingerprint template; a fusion module, used to fuse the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; and an identification module, used to input the fused feature matrix into a classification subnetwork in the target classification network to obtain an authenticity identification result of the fingerprint to be detected, wherein the authenticity identification result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

[0175] In an exemplary embodiment, the above-mentioned device is also used to obtain the native domain data of the real finger when the fingerprint template is recorded before determining the ninth feature description matrix that matches the eighth feature description matrix in the fingerprint template feature library; use the preprocessing subnetwork in the target classification network to preprocess the native domain data to obtain the feature description matrix of the native domain data; use the feature extraction subnetwork in the target classification network to extract features from the feature description matrix to obtain the target feature description matrix of the native domain data; and store the target feature description matrix in the fingerprint template feature library.

[0176] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0177] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0178] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0179] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0180] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0181] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0182] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method in each embodiment of the present application when the computer program is executed by a processor.

[0183] An embodiment of the present invention further provides a computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0184] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0185] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fingerprint anti-counterfeiting neural network training method, characterized in that: include: Acquire multiple groups of training data, each group of the training data includes: first native domain data, second native domain data, and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when a fake finger is used instead of the real finger for fingerprint verification; The initial classification network is trained using the multiple groups of training data to obtain a target classification network, wherein the initial classification network includes a fusion subnetwork and a classification subnetwork, and for each group of the training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

2. The method according to claim 1, characterized in that The method of using the multiple sets of training data to train the initial classification network to obtain the target classification network includes: Iteratively training the initial classification network using the multiple sets of training data until the number of times the initial classification network is trained reaches a target number of times, or when the loss value output by the target loss function of the initial classification network meets a preset training end condition, the training is terminated to obtain the target classification network; Wherein, the initial classification network further includes the preprocessing subnetwork, and the preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into a feature description matrix; The loss value output by the target loss function is a loss value determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork; The first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the true label corresponding to the feature description matrix input by the classification subnetwork.

3. The method according to claim 2, characterized in that The iterative training of the initial classification network using the multiple sets of training data comprises: The initial classification network is trained for the kth time by the following steps, where k is a positive integer: Acquire a kth group of training data from the multiple groups of training data for use in performing a kth training on the initial classification network; Inputting the first native domain data, the second native domain data, and the third native domain data in the kth group of training data into the preprocessing subnetwork in the classification network trained for the k-1th time, to obtain a first feature description matrix, a second feature description matrix, and a third feature description matrix, respectively, wherein the initial classification network trained for the 0th time is the untrained initial classification network; Input the first feature description matrix, the second feature description matrix and the third feature description matrix into the feature extraction subnetwork in the classification network trained for the k-1th time for dimensionality reduction processing, and obtain a fourth feature description matrix, a fifth feature description matrix and a sixth feature description matrix respectively; Inputting the fourth feature description matrix and the fifth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain a first fingerprint matching pair for the kth time; Inputting the fourth feature description matrix and the sixth feature description matrix into the fusion sub-network in the classification network trained for the k-1th time to obtain a second fingerprint matching pair for the kth time training; Inputting the first fingerprint matching pair of the k-th training and the second fingerprint matching pair of the k-th training into the classification subnetwork in the classification network of the k-1-th training to obtain the prediction label of the k-th training, wherein the prediction label of the k-th training is used to represent the authenticity recognition result of the classification network of the k-1-th training for the real finger fingerprint corresponding to the second native domain data in the k-th group of training data, and the authenticity recognition result of the classification network of the k-1-th training for the fake finger fingerprint corresponding to the third native domain data in the k-th group of training data; According to the first feature description matrix, the second feature description matrix, the third feature description matrix and the known feature description matrices corresponding to the first native domain data, the second native domain data and the third native domain data in the kth group of training data, determine the loss value output by the first loss function of the kth training; according to the predicted label of the kth training and the known true label of the kth training, determine the loss value output by the second loss function of the kth training; according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, determine the loss value output by the target loss function of the kth training; when k is less than the target number, or when the loss value output by the target loss function of the kth training does not meet the training end condition, adjust the values ​​of the parameters in the initial classification network of the k-1th training to obtain the initial classification network of the kth training.

4. The method according to claim 2, characterized in that: The objective loss function is: loss=α×loss MSE +β×loss BCE Among them, α and β are preset values, and loss MSE is the first loss function of the preprocessing sub-network, loss BCE is the second loss function of the classification subnetwork; The classification network also includes a feature extraction subnetwork, the preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a feature description matrix and input it into the feature extraction subnetwork, the feature extraction subnetwork is used to perform dimensionality reduction processing on the input feature description matrix, and input the matrix obtained after the dimensionality reduction processing into the fusion subnetwork.

5. The method according to claim 4, characterized in that The loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula: loss MSE =loss MSE_1 +loss MSE_2 +loss MSE_3 Among them, t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix output by the preprocessing sub-network, y(i,j) is the known feature description matrix of the native domain data of the input of the preprocessing sub-network, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1 The input data of the preprocessing sub-network is the loss value of the first native domain data, loss MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data, loss MSE_3 The input data of the preprocessing sub-network is the loss value of the third native domain data.

6. The method according to claim 4, characterized in that The loss value output by the second loss function of the classification subnetwork is determined by the following formula: loss BCE =-y×log(x)-(1-y)×log(1-x) Wherein, y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification subnetwork.

7. A fingerprint anti-counterfeiting method, characterized in that: include: Obtain the native domain data of the fingerprint to be detected; The native domain data of the fingerprint to be detected is preprocessed by the preprocessing sub-network in the target classification network to obtain the seventh feature description matrix; Performing feature extraction on the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; Determining a ninth feature description matrix that matches the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when entering a fingerprint template; fusing the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; The fused feature matrix is ​​input into the classification subnetwork in the target classification network to obtain an authenticity recognition result of the fingerprint to be detected, wherein the authenticity recognition result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

8. The method according to claim 7, characterized in that Before determining a ninth feature description matrix matching the eighth feature description matrix in the fingerprint template feature library, the method further includes: Acquire native domain data of the real finger when entering the fingerprint template; Preprocessing the native domain data using a preprocessing subnetwork in the target classification network to obtain a feature description matrix of the native domain data; Using the feature extraction subnetwork in the target classification network to perform dimensionality reduction processing on the feature description matrix to obtain a feature description matrix after dimensionality reduction; The feature description matrix after dimension reduction is stored in the fingerprint template feature library.

9. A fingerprint anti-counterfeiting neural network training device, characterized in that: include: A first acquisition module is used to acquire multiple groups of training data, each group of the training data includes: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the original data collected when a fake finger is used instead of the real finger for fingerprint verification; A training module is used to train an initial classification network using the multiple groups of training data to obtain a target classification network, wherein the initial classification network includes a fusion subnetwork and a classification subnetwork, and for each group of the training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and to generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

10. A fingerprint anti-counterfeiting device, characterized in that: include: The second acquisition module is used to acquire the native domain data of the fingerprint to be detected; A preprocessing module, used for preprocessing the native domain data of the fingerprint to be detected through the preprocessing sub-network in the target classification network to obtain a seventh feature description matrix; A feature extraction module, used for performing feature extraction on the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; A determination module, used to determine a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when recording a fingerprint template; A fusion module, used for fusing the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; The recognition module is used to input the fused feature matrix into the classification sub-network in the target classification network to obtain the authenticity recognition result of the fingerprint to be detected, wherein the authenticity recognition result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 6 or 7 to 8 when executed by a processor.

12. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 6 or 7 to 8.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method described in any one of claims 1 to 6 or 7 to 8 is implemented.

Images