Fingerprint anti-counterfeiting neural network training method, fingerprint anti-counterfeiting method and device

Abstract

Embodiments of the present application provide a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and device, comprising: obtaining multiple sets of training data, each set of training data including: first native domain data, second native domain data and third native domain data; using the multiple sets of training data to train an initial classification network to obtain a target classification network, the initial classification network including a fusion subnetwork and a classification subnetwork, for each set of training data, the fusion subnetwork is used to generate a first fingerprint matching pair based on the feature description matrix of the first native domain data and the second native domain data, and generate a second fingerprint matching pair based on the feature description matrix of the first native domain data and the third native domain data, and the classification subnetwork is used for fingerprint classification and identification based on the first fingerprint matching pair and the second fingerprint matching pair. Through the present application, the problem of low identification accuracy of true and false fingerprints in related technologies is solved, and the effect of improving the identification accuracy of true and false fingerprints is achieved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of fingerprint identification, and in particular, to a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and device. BACKGROUND

[0002] In daily life, fingerprint identification technology is often used to authenticate the identity of a user, for example, smart device unlocking, application login, online payment, and the like. In recent years, various types of sensors such as capacitive, optical, and ultrasonic sensors can be used to collect fingerprint data.

[0003] In the field of fingerprint anti-counterfeiting, using a neural network model can improve the recognition rate of real and fake fingerprints. However, the fingerprint data collected by a type of sensor is currently used to train the neural network model. Since there are various types of sensors on the market for collecting fingerprints, different neural network models need to be trained for different types of sensors, which complicates the training of the neural network model and wastes resources. Moreover, the trained neural network model can only recognize the fingerprint data collected by one type of sensor and cannot recognize the fingerprint data collected by other types of sensors, limiting the scope of application.

[0004] In addition, the collection of fingerprint data is often affected by environmental factors (such as temperature fluctuations, film interference, and changes in finger state). Directly using the fingerprint data collected by the sensor to train the neural network model can affect the training effect of the neural network model, resulting in a low recognition accuracy of the trained neural network model.

[0005] To address the above problems, there is currently no effective solution. SUMMARY

[0006] Embodiments of the present application provide a fingerprint anti-counterfeiting neural network training method, a fingerprint anti-counterfeiting method and device to at least solve the problem of low recognition accuracy of real and fake fingerprints in related technologies.

[0007] According to one embodiment of the present application, a training method of a fingerprint anti-counterfeiting neural network is provided, comprising: obtaining a plurality of sets of training data, each set of training data comprising: first native domain data, second native domain data, and third native domain data, wherein the first native domain data is original data of a real finger collected when a real finger is input into a fingerprint template, the second native domain data is original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is original data collected when a fake finger is used instead of the real finger for fingerprint verification; training an initial classification network using the plurality of sets of training data to obtain a target classification network, wherein the initial classification network comprises a fusion sub-network and a classification sub-network, and for each set of training data, the fusion sub-network is used to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data, and the classification sub-network is used to perform fingerprint classification and identification based on the first fingerprint matching pair and the second fingerprint matching pair.

[0008] In one exemplary embodiment, training the initial classification network using the plurality of sets of training data to obtain the target classification network comprises: iteratively training the initial classification network using the plurality of sets of training data until a target number of times of training the initial classification network reaches a target number of times, or a loss value output by a target loss function of the initial classification network meets a preset training end condition, and then ending the training to obtain the target classification network; wherein the initial classification network further comprises a preprocessing sub-network, the preprocessing sub-network is used to convert native domain data collected by a fingerprint sensor into a corresponding feature description matrix; the loss value output by the target loss function is a loss value determined according to a loss value output by a first loss function of the preprocessing sub-network and a loss value output by a second loss function of the classification sub-network; the first loss function is used to represent a loss between the feature description matrix output by the preprocessing sub-network and a known feature description matrix of the native domain data input by the preprocessing sub-network, and the second loss function is used to represent a loss between a predicted label value output by the classification sub-network and a true label corresponding to the feature description matrix input by the classification sub-network.

[0009] In one example embodiment, the initial classification network is iteratively trained using multiple sets of training data, including: training the initial classification network for the kth time by the following steps, where k is a positive integer: obtaining, from the multiple sets of training data, a kth set of training data used for training the initial classification network for the kth time; inputting first native domain data, second native domain data and third native domain data in the kth set of training data into a preprocessing subnetwork in the classification network trained for the (k-1)th time to obtain a first feature description matrix, a second feature description matrix and a third feature description matrix, respectively, wherein the initial classification network trained for the 0th time is the untrained initial classification network, and the preprocessing subnetwork is configured to convert native domain data obtained by different types of sensors into corresponding feature description matrices; inputting the first feature description matrix, the second feature description matrix and the third feature description matrix into a feature extraction subnetwork in the classification network trained for the (k-1)th time for dimension reduction processing to obtain a fourth feature description matrix, a fifth feature description matrix and a sixth feature description matrix, respectively; inputting the fourth feature description matrix and the fifth feature description matrix into a fusion subnetwork in the classification network trained for the (k-1)th time to obtain a first fingerprint matching pair for the kth time;

[0010] inputting the fourth feature description matrix and the sixth feature description matrix into the fusion subnetwork in the classification network trained for the (k-1)th time to obtain a second fingerprint matching pair for the kth time;

[0011] inputting the first fingerprint matching pair for the kth time and the second fingerprint matching pair for the kth time into a classification subnetwork in the classification network trained for the (k-1)th time to obtain a predicted label for the kth time, wherein the predicted label for the kth time is configured to represent a result of authenticity identification of a true finger fingerprint corresponding to the second native domain data in the kth set of training data by the classification network trained for the (k-1)th time, and a result of authenticity identification of a false finger fingerprint corresponding to the third native domain data in the kth set of training data by the classification network trained for the (k-1)th time;

[0012] determining a loss value output by a first loss function for the kth time according to the first feature description matrix, the second feature description matrix and the third feature description matrix and known feature description matrices corresponding to the first native domain data, the second native domain data and the third native domain data in the kth set of training data, determining a loss value output by a second loss function for the kth time according to the predicted label for the kth time and a known true label for the kth time, determining a loss value output by a target loss function for the kth time according to the loss value output by the first loss function for the kth time and the loss value output by the second loss function for the kth time, and adjusting a value of a parameter in the initial classification network trained for the (k-1)th time to obtain the initial classification network trained for the kth time when k is less than a target number, or when the loss value output by the target loss function for the kth time does not satisfy a training end condition.

[0013] In an example embodiment, the target loss function is:

[0014] loss = a x loss MSE + b x loss BCE

[0015] wherein a, b are preset values, loss MSE is the first loss function of the preprocessing subnetwork, loss BCE is the second loss function of the classification subnetwork;

[0016] The classification network further comprises a feature extraction subnetwork, the preprocessing subnetwork is configured to convert native domain data acquired by a fingerprint sensor into a corresponding feature description matrix and input the feature description matrix into the feature extraction subnetwork, and the feature extraction subnetwork is configured to perform dimension reduction processing on the input feature description matrix and input a matrix obtained after the dimension reduction processing into the fusion subnetwork.

[0017] In an example embodiment, the loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0018]

[0019] loss MSE = loss MSE_1 + loss MSE_2 + loss MSE_3

[0020] wherein t e (1, 3), i e (1, h), j e (1, w), f(i, j) is the feature description matrix output by the preprocessing subnetwork, y(i, j) is a known feature description matrix of the native domain data input into the preprocessing subnetwork, h is a vertical feature size of the feature description matrix, w is a horizontal feature size of the feature description matrix, loss MSE_1 is a loss value when the input data of the preprocessing subnetwork is the first native domain data, loss MSE_2 is a loss value when the input data of the preprocessing subnetwork is the second native domain data, and loss MSE_3 is a loss value when the input data of the preprocessing subnetwork is the third native domain data.

[0021] In an example embodiment, the loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0022] loss BCE = -y x log(x) - (1 - y) x log(1 - x)

[0023] Wherein, y is the true label value of the first or second fingerprint matching pair, and x is the predicted label value of the first or second fingerprint matching pair output by the classification subnetwork.

[0024] According to another embodiment of the present application, a fingerprint anti-counterfeiting method is provided, comprising: obtaining native domain data of a fingerprint to be detected; preprocessing the native domain data of the fingerprint to be detected through a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; performing feature extraction on the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; determining a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when the fingerprint template is entered; fusing the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; and inputting the fused feature matrix into a classification subnetwork in the target classification network to obtain a true or false identification result of the fingerprint to be detected, wherein the true or false identification result is used to indicate that the fingerprint to be detected is a fingerprint of a real finger or a fingerprint of a fake finger.

[0025] In an exemplary embodiment, before determining the ninth feature description matrix matching the eighth feature description matrix in the fingerprint template feature library, the above method further comprises: obtaining native domain data of a real finger when the fingerprint template is entered; preprocessing the native domain data using the preprocessing subnetwork in the target classification network to obtain a feature description matrix of the native domain data; performing dimension reduction processing on the feature description matrix using the feature extraction subnetwork in the target classification network to obtain a dimension-reduced feature description matrix; and storing the dimension-reduced feature description matrix in the fingerprint template feature library.

[0026] According to another embodiment of the present application, a training of a fingerprint anti-counterfeiting neural network is provided, comprising: a first acquisition module configured to acquire a plurality of groups of training data, each group of training data comprising: first native domain data, second native domain data, and third native domain data, wherein the first native domain data is original data of a real finger collected when the real finger is entered into a fingerprint template, the second native domain data is original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is original data collected when a fake finger is used instead of the real finger for fingerprint verification; and a training module configured to train an initial classification network using the plurality of groups of training data to obtain a target classification network, wherein the initial classification network comprises a fusion subnetwork and a classification subnetwork, and for each group of training data, the fusion subnetwork is configured to generate a first fingerprint matching pair based on feature description matrices of the first and second native domain data, and generate a second fingerprint matching pair based on feature description matrices of the first and third native domain data, and the classification subnetwork is configured to perform fingerprint classification and identification based on the first and second fingerprint matching pairs.

[0027] According to another embodiment of the present application, a fingerprint anti-counterfeiting device is provided, comprising: a second acquisition module configured to acquire original domain data of a fingerprint to be detected; a preprocessing module configured to preprocess the original domain data of the fingerprint to be detected through a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; a feature extraction module configured to extract features of the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; a determination module configured to determine a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when a fingerprint template is recorded; a fusion module configured to fuse the eighth feature description matrix and the ninth feature description matrix to obtain a fusion feature matrix; and an identification module configured to input the fusion feature matrix into a classification subnetwork in the target classification network to obtain a true or false identification result of the fingerprint to be detected, wherein the true or false identification result is used to indicate whether the fingerprint to be detected is a fingerprint of the real finger or a fingerprint of a fake finger.

[0028] According to still another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method in any of the above embodiments.

[0029] According to still another embodiment of the present application, an electronic device is also provided, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0030] According to the present application, the first original domain data of the real finger collected when the real finger is recorded in a fingerprint template, the second original domain data of the real finger collected when the real finger is used for fingerprint verification, and the third original domain data collected when a fake finger is used to replace the real finger for fingerprint verification are used as training data to train an initial classification network;

[0031] The present application adopts a sample feature fusion manner, and a fusion subnetwork is used to fuse the features of the first original domain data and the second original domain data to generate a first fingerprint matching pair, and fuse the features of the first original domain data and the third original domain data to generate a second fingerprint matching pair;

[0032] The application trains the classified subnetwork using the fused first and second fingerprint matching pairs. The influence of environmental factors on training of the neural network model can be reduced when collecting fingerprint data, thereby preventing environmental factors from affecting the training effect of the neural network model, and thereby solving the problem in the related art that the recognition accuracy of the trained neural network model is low due to environmental influence when collecting fingerprint data, and thereby achieving the effect of improving the recognition accuracy of real and fake fingerprints.

[0033] In addition, the preprocessing subnetwork in the application can convert the native domain data collected by the fingerprint sensor into a feature description matrix. The target classification network trained further can recognize fingerprints collected by multiple types of sensors, thereby solving the problems in the prior art that different neural network models need to be trained for different types of sensors, resulting in complex neural network model training, resource waste, and the trained neural network model being unable to recognize fingerprint data collected by other types of sensors, and the application range being limited. Further, the target classification network of the application can recognize fingerprints collected by multiple different types of sensors, and has a wider application range and saves resources. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a hardware structure block diagram of a mobile terminal of a fingerprint anti-counterfeiting neural network training method and a fingerprint anti-counterfeiting method according to an embodiment of the application;

[0035] Figure 2 is a flowchart of a fingerprint anti-counterfeiting neural network training method according to an embodiment of the application;

[0036] Figure 3 is a whole architecture diagram of a classification network according to an embodiment of the application;

[0037] Figure 4 is a real and fake identification flowchart of a to-be-detected fingerprint according to an embodiment of the application;

[0038] Figure 5 is a similarity measurement classification network architecture schematic diagram according to an embodiment of the application;

[0039] Figure 6 is a deep convolutional neural network feature extraction structure table according to an embodiment of the application;

[0040] Figure 7 is a whole flowchart schematic diagram according to an embodiment of the application;

[0041] Figure 8 is an ultrasonic anti-counterfeiting actual measurement benefit comparison schematic diagram according to an embodiment of the application;

[0042] Figure 9is a benefit comparison diagram of a similarity measurement mechanism according to an embodiment of the present application;

[0043] Figure 10 is a structural block diagram of an identification device of a real and fake fingerprint according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] Embodiments of the present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0045] Raw domain data: refers to the digital matrix obtained after the fingerprint signal collected by the sensor is subjected to analog-to-digital conversion. The raw domain data is original data without preprocessing, and the information of the finger lines cannot be directly observed. However, the raw domain data contains all the original information without filtering, and is more suitable for feature extraction with deep learning technology.

[0046] FLRR: false finger rejection rate.

[0047] SAR: false fingerprint acceptance rate.

[0048] Conv: convolution operator in deep convolutional neural network.

[0049] Avg Pooling: average pooling, a kind of average-based downsampling operator.

[0050] Depth-wise Conv: depth-wise separable convolution in deep convolutional neural network.

[0051] Transformer: a kind of attention mechanism deep learning network architecture.

[0052] MLP: multilayer perceptron, a kind of classic network architecture.

[0053] Binary Cross Entropy: binary cross entropy, a kind of classic classification loss function.

[0054] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0055] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the case of running on a mobile terminal, Figure 1 is a hardware structural block diagram of a mobile terminal of a training method of a fingerprint anti-fake neural network and a fingerprint anti-fake method according to an embodiment of the present application. As shown in Figure 1 , the mobile terminal can include one or more Figure 1The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that, Figure 1 The structure shown is only schematic, and does not limit the structure of the mobile terminal. For example, the mobile terminal can include more or fewer components than those shown, or have a different configuration of components than those shown. Figure 1 The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that, Figure 1 The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that,

[0056] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as the training method of the fingerprint anti-counterfeiting neural network and the computer program corresponding to the fingerprint anti-counterfeiting method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and these remote memories can be connected to the mobile terminal through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0057] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0058] In the present embodiment, a training method of a fingerprint anti-counterfeiting neural network running on the mobile terminal is provided, Figure 2 is a flowchart of the training method of the fingerprint anti-counterfeiting neural network according to the embodiments of the present application, as shown in Figure 2 The flowchart includes the following steps:

[0059] In step S202, a plurality of sets of training data are obtained, each set of training data including first native domain data, second native domain data, and third native domain data. The first native domain data is original data of a real finger collected when the real finger is registered into a fingerprint template. The second native domain data is original data of the real finger collected when the real finger is used for fingerprint verification. The third native domain data is original data collected when a fake finger is used to replace the real finger for fingerprint verification.

[0060] The plurality of sets of training data include at least two types of native domain data collected by sensors. For example, the native domain data collected by capacitive, optical, and ultrasonic sensors.

[0061] The real finger registered into the fingerprint template refers to that the real finger is placed on a fingerprint recognition device (the fingerprint recognition device is provided with a sensor for collecting fingerprint data), and the operation of registering the fingerprint template is performed according to the instruction of the device. After the registration is completed, a unique fingerprint template is generated for subsequent fingerprint recognition verification.

[0062] Fingerprint verification refers to comparing the collected fingerprint data with the pre-registered fingerprint template to determine whether the identity of the user matches.

[0063] The fake finger includes, but is not limited to, a variety of fake fingerprints manufactured by processes such as printed paper, silica gel, wood glue, and latex.

[0064] In step S204, the initial classification network is trained using the plurality of sets of training data to obtain a target classification network. The initial classification network includes a fusion sub-network and a classification sub-network. For each set of training data, the fusion sub-network is used to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data. The classification sub-network is used to perform fingerprint classification and identification based on the first fingerprint matching pair and the second fingerprint matching pair.

[0065] Optionally, the execution subject of the above steps can be a background processor, or other devices with similar processing capabilities, and can also be a machine integrated with at least an image acquisition device and a data processing device. The image acquisition device can include a camera and other image acquisition modules. The data processing device can include a computer, a mobile phone, and other terminals, but is not limited thereto.

[0066] The initial classification network comprises a preprocessing subnetwork, a fusion subnetwork, a feature subnetwork, and a classification subnetwork. The subnetworks can be neural network models in the prior art, for example, can be convolutional neural networks. The initial classification network is iteratively trained using the multiple sets of training data. The number of iterations can be a target number. When the number of training reaches the target number, the training is ended to obtain a target classification network. Alternatively, when a loss value output by a target loss function meets a preset training end condition, the training is ended to obtain the target classification network.

[0067] The loss value output by the target loss function comprises two parts: a loss value output by a first loss function of the preprocessing subnetwork and a loss value output by a second loss function of the classification subnetwork. The target loss function is as follows:

[0068] loss = α × loss MSE + β × loss BCE

[0069] wherein α and β are preset values, loss MSE is the first loss function of the preprocessing subnetwork, and loss BCE is the second loss function of the classification subnetwork.

[0070] The preprocessing subnetwork is configured to convert native domain data collected by a fingerprint sensor into a corresponding feature description matrix. The fingerprint sensor can be one of a capacitive fingerprint sensor, an optical fingerprint sensor, or an ultrasonic fingerprint sensor. The first loss function represents a loss between the feature description matrix output by the preprocessing subnetwork and a known feature description matrix of the native domain data input into the preprocessing subnetwork. The second loss function is configured to represent a loss between a predicted label value output by the classification subnetwork and a true label corresponding to the feature description matrix input into the classification subnetwork.

[0071] The loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0072]

[0073] loss MSE = loss MSE_1 + loss MSE_2 + loss MSE_3

[0074] wherein t is an integer between 1 and 3, i is an integer between 1 and h, j is an integer between 1 and w, f(i, j) is the feature description matrix output by the preprocessing subnetwork, y(i, j) is the known feature description matrix of the native domain data input into the preprocessing subnetwork, h is a vertical feature size of the feature description matrix, w is a horizontal feature size of the feature description matrix, loss MSE_1is a loss value when the input data of the preprocessing subnetwork is the first native domain data, loss MSE_2 is a loss value when the input data of the preprocessing subnetwork is the second native domain data, loss MSE_3 is a loss value when the input data of the preprocessing subnetwork is the third native domain data.

[0075] The loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0076] loss BCE =-y×log(x)-(1-y)×log(1-x)

[0077] Wherein, y is the true label value of the first or second fingerprint matching pair, and x is the predicted label value of the first or second fingerprint matching pair output by the classification subnetwork.

[0078] Taking the kth training of the initial classification network as an example, k is a positive integer, Figure 3 is the overall architecture diagram of the classification network, which includes the following contents:

[0079] In the plurality of sets of training data, the kth set of training data used for the kth training of the initial classification network is obtained, and the kth set of training data can be fingerprint data collected using the same or different types of sensors. Specifically, it can include the following fingerprint data: the first native domain data collected when a real finger enters a fingerprint template on a smart device, the second native domain data collected when a real finger performs fingerprint verification on a smart device, and the third native domain data collected when a fake finger replaces a real finger to perform fingerprint verification on a smart device (fake fingerprint attack, fake real finger to perform fingerprint verification).

[0080] According to the information acquisition characteristics of the sensor, when the sensor collects the fingerprint each time, a fixed scale native domain data (including the first, second and third native domain data) of a specific channel number N (an integer greater than or equal to 1) can be obtained for each press, and the resolution is [Height, Width, N]. Taking the native domain data collected by the ultrasonic sensor as an example, the native domain data contains two channels of real and imaginary parts, and the scale of the native domain data can be [80, 80, 2] (only for example, the specific scale of the native domain data can be determined according to the actual situation).

[0081] In order to effectively organize training, the fingerprint template input by the real finger is used as a guide to train the initial classification network to identify real fingerprints and fake fingerprints. The first native domain data collected when the real finger inputs the fingerprint template, the second native domain data collected when the real finger performs fingerprint verification, and the third native domain data collected when a fake finger is used to perform fingerprint verification (during fingerprint attack) form a set of training data (also referred to as a native domain triplet, and the feature scale is [3, Height, Width, N]).

[0082] The first native domain data, the second native domain data, and the third native domain data in the kth set of training data are input into the preprocessing subnetwork in the classification network trained for the k-1th time to obtain a first feature description matrix, a second feature description matrix, and a third feature description matrix, respectively. The initial classification network trained for the 0th time is an initial classification network that has not been trained, and the preprocessing subnetwork is used to convert the native domain data obtained by the fingerprint sensor into a feature description matrix.

[0083] The preprocessing subnetwork described above can be a network obtained by combining a convolutional layer and a pooling layer, and can convert the native domain data obtained by the sensor into a feature description matrix (which is a digital matrix obtained by enhancing and converting a fingerprint image in an image domain). The preprocessing subnetwork is used to convert the native domain data collected by the fingerprint sensor into a corresponding feature description matrix.

[0084] Depending on the specified known feature description matrix of different types of sensors as external guidance, the preprocessing subnetwork is required to output a feature description matrix that is as consistent as possible with the given known feature description matrix, so as to achieve the purpose of identifying the fingerprint data collected by multiple types of sensors through the same classification network.

[0085] Specifically, for example, the capacitance sensor uses a capacitance preprocessing result (a known feature description matrix of a capacitance type sensor) as an intermediate guide, and requires the preprocessing subnetwork to output a feature description matrix that is as consistent as possible with the capacitance preprocessing result. The optical sensor uses an optical preprocessing result (a known feature description matrix of an optical type sensor) as an intermediate guide, and requires the preprocessing subnetwork to output a feature description matrix that is as consistent as possible with the optical preprocessing result.

[0086] The preprocessing subnetwork is constrained by the first loss function described above to convert the native domain data obtained by the fingerprint sensor into a feature description matrix corresponding to the native domain data.

[0087] The first feature description matrix, the second feature description matrix, and the third feature description matrix are input into the feature extraction subnetwork in the classification network trained for the k-1th time for dimension reduction processing to obtain a fourth feature description matrix, a fifth feature description matrix, and a sixth feature description matrix, respectively.

[0088] The classification network further comprises a feature extraction subnetwork, the pre-processing subnetwork inputs the output feature description matrix (including the first feature description matrix, the second feature description matrix and the third feature description matrix) into the feature extraction subnetwork, the feature extraction subnetwork performs dimension reduction processing on the input matrix, and inputs the matrix obtained after the dimension reduction processing into the fusion subnetwork (the matrix obtained after the dimension reduction processing includes the fourth feature description matrix, the fifth feature description matrix and the sixth feature description matrix).

[0089] The feature extraction subnetwork can use a deep separable convolution technology, combine a convolution layer and a pooling layer, construct a feature conversion module, realize large-scale dimension reduction and condensation of the feature description matrix, and maximize the preservation of anti-counterfeiting identifiable information and the filtering of noise and interference.

[0090] The fourth feature description matrix and the fifth feature description matrix are input into the fusion subnetwork in the classification network of the k-1th training to obtain a first fingerprint matching pair of the kth training; the fourth feature description matrix and the sixth feature description matrix are input into the fusion subnetwork in the classification network of the k-1th training to obtain a second fingerprint matching pair of the kth training.

[0091] The first fingerprint matching pair of the kth training and the second fingerprint matching pair of the kth training are input into the classification subnetwork in the classification network of the k-1th training to obtain a prediction label of the kth training, wherein the prediction label of the kth training is used to represent the authenticity identification result of the true finger fingerprint corresponding to the second native domain data in the kth training data by the classification network of the k-1th training, and the authenticity identification result of the false finger fingerprint corresponding to the third native domain data in the kth training data by the classification network of the k-1th training.

[0092] The fusion subnetwork and the classification subnetwork use a full connection layer and a softmax to output a prediction label of the current input, distinguish between two input modal forms of a true finger + true finger fingerprint template and a false finger + true finger fingerprint template, and realize a true / false fingerprint identification function.

[0093] According to the first feature description matrix, the second feature description matrix, and the third feature description matrix, and the known feature description matrix corresponding to the first native domain data, the second native domain data, and the third native domain data in the kth group of training data, a loss value output by a first loss function of the kth training is determined, and according to the predicted label of the kth training and the known real label of the kth training, a loss value output by a second loss function of the kth training is determined; according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, a loss value output by a target loss function of the kth training is determined; when k is less than a target number of times, or in a case where the loss value output by the target loss function of the kth training does not satisfy a training end condition, a value of a parameter in the initial classification network of the (k-1)th training is adjusted to obtain an initial classification network of the kth training.

[0094] As an optional implementation, the initial classification network is iteratively trained using multiple groups of training data until a number of times of training the initial classification network reaches a target number of times, or a loss value output by a target loss function of the initial classification network satisfies a preset training end condition, and the training is ended to obtain a target classification network.

[0095] The target number of times is a configuration training base parameter, and AdamW is used as a training optimizer, for example, 200 epochs can be trained;

[0096] The loss value output by the target loss function is a loss value determined according to a loss value output by a first loss function of the preprocessing subnetwork and a loss value output by a second loss function of the classification subnetwork.

[0097] The first loss function is used to represent a loss between the feature description matrix output by the preprocessing subnetwork and a known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent a loss between a predicted label value output by the classification subnetwork and a real label corresponding to the feature description matrix input by the classification subnetwork.

[0098] The initial classification network is iteratively trained using multiple groups of training data until a number of times of training the initial classification network reaches a target number of times, or a loss value output by a target loss function of the initial classification network satisfies a preset training end condition, and the training is ended to obtain a target classification network.

[0099] The loss value output by the target loss function is a loss value determined according to a loss value output by a first loss function of the preprocessing subnetwork and a loss value output by a second loss function of the classification subnetwork.

[0100] The first loss function is used to represent a loss between a feature description matrix output by the preprocessing subnetwork and a known feature description matrix corresponding to a sensor type of the native domain data input to the preprocessing subnetwork, and the second loss function is used to represent a loss between a predicted label value output by the classification subnetwork and a true label corresponding to the feature description matrix input to the classification subnetwork.

[0101] Specifically, the target loss function is:

[0102] loss = a x loss MSE + b x loss BCE

[0103] wherein a and b are preset values, loss MSE is a first loss function of a preprocessing subnetwork included in the classification network, and loss BCE is a second loss function of the classification subnetwork.

[0104] The gradient backpropagation method is used in combination with the AdamW optimizer to realize simultaneous fine-tuning update of all subnetwork parameters in the initial classification network, so as to gradually reduce the loss function. The loss function used is a linear combination of the preprocessing approximation loss function and the final anti-fake classification function, wherein the values of a and b can be set according to actual conditions, for example, a is 0.1 and b is 0.9.

[0105] Each native domain data included in each set of training data is acquired by one of the following sensors: a capacitive sensor, an optical sensor, and an ultrasonic sensor.

[0106] In order to make the preprocessing subnetwork capable of processing native domain data collected by various types of sensors including capacitive, optical, and ultrasonic sensors, a mean square error (MSE) loss function is first constructed between a feature description matrix generated based on a specific preprocessing pipeline and a feature description matrix output by the preprocessing subnetwork. The function defines whether the preprocessing subnetwork can accurately output the feature description matrix of the corresponding type of sensor by calculating the global sum of the square of the difference between the feature description matrix output by the preprocessing subnetwork and the feature description matrix based on the specific preprocessing. The smaller the value of the function, the more consistent the output result of the constructed preprocessing network is with the feature description matrix obtained by the specific preprocessing.

[0107] The loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0108]

[0109] loss MSE = loss MSE_1+loss MSE_2 +loss MSE_3

[0110] wherein t e (1, 3), i e (1, h), j e (1, w), f(i, j) is the feature description matrix output by the preprocessing subnetwork, y(i, j) is the known feature description matrix of the native domain data input to the preprocessing subnetwork, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, loss MSE_1 is the loss value when the input data of the preprocessing subnetwork is the first native domain data, loss MSE_2 is the loss value when the input data of the preprocessing subnetwork is the second native domain data, loss MSE_3 is the loss value when the input data of the preprocessing subnetwork is the third native domain data.

[0111] The loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0112] loss BCE = -y x log(x) - (1 - y) x log(1 - x)

[0113] wherein y is the true label value of the first or second fingerprint matching pair, and x is the predicted label value of the first or second fingerprint matching pair output by the classification subnetwork.

[0114] As an optional implementation, when the number of epochs corresponding to the training cycle reaches a preset value of 200, the optimization is stopped, the network parameters stop updating, and the network training is completed. Experience shows that after the above training, the true and false fingerprint data matching pairs constructed using a certain scale of native domain triple data can improve the effective information filtering capability of the preprocessing subnetwork on the one hand; on the other hand, the feature extraction network can also greatly reduce the pre-stored feature dimension in actual application, improving the ease of use; and finally, the iterative training mode of true and false data can also enable the network to have effective anti-counterfeiting performance.

[0115] After the training task is completed, the preprocessing subnetwork and the feature extraction subnetwork are called in the fingerprint template registration link of the true finger, and each fingerprint template is compressed into a 4*4*10 feature description matrix and stored in the fingerprint template feature library. Compared with storing the native domain data 80*80*2, the storage space requirement is only 1 / 80 of the original, saving storage space.

[0116] According to another embodiment of the application, a fingerprint anti-counterfeiting method is provided, comprising: obtaining native domain data of a fingerprint to be detected; preprocessing the native domain data of the fingerprint to be detected through a preprocessing subnetwork in the target classification network to obtain a seventh feature description matrix; extracting features of the seventh feature description matrix through a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix;

[0117] As shown in the fingerprint to be detected Figure 4 The flowchart of the authenticity identification of the fingerprint to be detected also includes the following steps:

[0118] In step S401, a ninth feature description matrix matching the eighth feature description matrix is determined in a fingerprint template feature library, wherein a plurality of feature description matrices of the real finger when entering a fingerprint template are stored in the fingerprint template feature library.

[0119] The fingerprint template feature library can be obtained in the following manner:

[0120] The native domain data of the real finger when entering the fingerprint template is obtained; the native domain data is preprocessed using the preprocessing subnetwork in the target classification network to obtain a feature description matrix of the native domain data;

[0121] The feature description matrix is processed by dimension reduction using the feature extraction subnetwork in the target classification network to obtain a feature description matrix after dimension reduction; and the feature description matrix after dimension reduction is stored in the fingerprint template feature library.

[0122] In step S404, the eighth feature description matrix and the ninth feature description matrix are fused to obtain a fusion feature matrix.

[0123] In step S405, the fusion feature matrix is input into a classification subnetwork in the target classification network to obtain a true-false identification result of the fingerprint to be detected, wherein the true-false identification result is used to indicate whether the fingerprint to be detected is a fingerprint of the real finger or a fingerprint of the fake finger.

[0124] Taking the scale [80, 80, 2] of the native domain data collected by the ultrasonic sensor as an example. The data flow of feature description extraction and final fusion classification is introduced in layers as follows: Figure 5 As shown in the similarity measurement classification network architecture diagram, the following steps are included:

[0125] In step 501, the scale of the native domain data of the fingerprint template registered by the real finger and the native domain data of the fingerprint to be detected is 80*80*2, and the native domain data of the fingerprint template registered by the real finger and the native domain data of the fingerprint to be detected are input into the preprocessing subnetwork, wherein the length is 80, the width is 80, and two channels of real and imaginary parts are included.

[0126] Step 502: The preprocessing subnetwork outputs two feature description matrices with a scale of 80*80*1 (including the seventh feature description matrix obtained by the preprocessing subnetwork preprocessing the native domain data of the fingerprint to be detected, and the feature description matrix obtained by the preprocessing subnetwork preprocessing the native domain data of the real finger when the fingerprint template is entered when constructing the fingerprint template feature library).

[0127] The preprocessing subnetwork can extract and enhance fingerprints from native domain data collected by different types of sensors through training and guidance of subsequent data, thereby significantly reducing the dependence on the fingerprint preprocessing process and greatly improving the applicability of the solution.

[0128] Step 503: The feature extraction subnetwork uses depthwise separable convolutional blocks, combined with pooling layers, to extract feature descriptors from the 80*80*1 feature description matrix for both the template and the sample to be classified, resulting in two feature description matrices with a scale of 4*4*10 (including the eighth feature description matrix mentioned above, and the feature description matrix obtained by the feature extraction subnetwork from the preprocessed feature description matrix of the real finger when the fingerprint template is entered during the construction of the fingerprint template feature library). The feature extraction subnetwork reduces the data dimensionality while removing noise and redundant information, significantly alleviating the pressure of feature storage.

[0129] Step 504: The fusion sub-network uses feature splicing fusion technology to splice the two 4*4*10 feature description matrices through channels to construct a feature matrix with a scale of 4*4*20 (also known as a fingerprint matching pair).

[0130] The fingerprint recognition mechanism of the fingerprint recognition system is used to index the feature description matrix of the same finger in the corresponding registration process, and the features are fused to form a 4*4*20 fused feature matrix.

[0131] Step 505: In the final classification sub-network, the 4*4*20 dimensional feature matrix is ​​reorganized into a 128-dimensional descriptor through continuous convolutional blocks, and then mapped to 2-dimensional true / false probabilities using a fully connected layer. A non-linear mapping is then used to convert the 2-dimensional output into true / false probability values.

[0132] The fusion feature matrix is ​​fed into the trained fusion sub-network and classification sub-network to obtain the probability that the data is a genuine fingerprint matching fusion feature and a fake fingerprint matching fusion feature. Based on the probability results, a genuine / fake identification result is generated.

[0133] In the application phase, the preprocessing subnetwork and feature extraction subnetwork are invoked to compress each unlock press data into a 4*4*10 feature description matrix. For example... Figure 6A deep convolutional neural network feature extraction structure table is shown. Based on deep separable convolution, full connection and nonlinear mapping, a true and false identification result can be obtained, including a score corresponding to a true finger.

[0134] Since the same finger valley ridge has 3D structure stability under certain conditions, the living body pressing frame data should be relatively consistent with the registered true finger on the 3D structure. In addition, in order to improve the accuracy of 3D structure matching and avoid tedious and inefficient manual traditional similar feature extraction, a convolutional neural network (CNN) is used to build a general fingerprint anti-counterfeiting network.

[0135] As shown in Figure 7 A whole process schematic diagram is shown, including:

[0136] Step S701, training data collection, including the original domain data collected when the fingerprint template of the true finger is registered, the second original domain data collected when the true finger fingerprint is verified, and the third original domain data collected when the fingerprint is verified by using a false finger instead of the true finger;

[0137] Step S702, based on CNN, a segmented initial classification network is built, including a preprocessing subnetwork, a feature extraction subnetwork, a fusion subnetwork, and a classification subnetwork;

[0138] Step S703, the initial classification network is optimized as a whole by using the training data;

[0139] Step S704, based on the original domain data of the captured fingerprint to be detected, the preprocessing subnetwork and the feature extraction subnetwork are called for each registered fingerprint template to obtain the feature description matrix of the original domain data of the fingerprint to be detected;

[0140] Step S705, the feature description matrix of the fingerprint template that can be matched is indexed by a traditional fingerprint matching algorithm;

[0141] In actual use, the preprocessing subnetwork and the feature extraction subnetwork are called for each fingerprint template registered by the true finger in the registration stage to obtain the feature description matrix of the fingerprint template, and the features of the fingerprint template pressed each time are retained at a small storage cost;

[0142] Step 706, the feature description matrix of the paired fingerprint template and the feature description matrix of the fingerprint to be detected are fused and sent to the trained special classification subnetwork to obtain the true and false identification result of the fingerprint to be detected, and the identification is completed.

[0143] The present application aims at the general fingerprint anti-counterfeiting technology vacancy, and proposes a fingerprint anti-counterfeiting scheme based on template and sample fusion features. From the physical dimension of "fingerprint template and unlocking pressing original domain fusion input signal true and false separability", the template-sample fusion information true and false recognition network is trained by using true and false matching pair combination data guidance, so as to realize high-precision detection anti-counterfeiting for high-simulation false fingerprints with low calculation complexity cost and zero external hardware sensor cost.

[0144] As shown in Figure 8 The ultrasound anti-counterfeiting actual measurement benefit comparison of the present application and Figure 9 The similarity measurement mechanism benefit comparison of the present application is shown in the figure, and the false liveness reject rate (FLRR) and the spoof accept rate (SAR) are used as performance evaluation indexes. Among them, the parameter No. Liveness Rejection represents the number of times of true finger rejection, the parameter No. Liveness Pressing represents the number of times of true finger pressing, the parameter No. Spoof Accept represents the number of times of false finger being wrongly accepted Pass, and the parameter No. Spoof Attack represents the total number of false finger attacks.

[0145]

[0146] Taking ultrasonic fingerprint anti-counterfeiting as an example, six kinds of 3D solids that can be cracked with high probability are made, and the materials used include condensate gel, latex and other fingerprint solid extraction materials. Five fingers of each type of false fingerprint are made, and each finger attacks 30 times, a total of about 150 attacks. The experimental results show that, compared with the current mainstream simple anti-counterfeiting method of manually constructing traditional features for distribution comparison, the fusion information of the fingerprint template and the unlocking sample effectively used by the present scheme combines the high robustness of artificial intelligence algorithm, and can stably realize effective defense against various false fingerprints without significantly affecting the true finger unlocking experience. The average cracking rate is reduced from 81.51% to 7.17%.

[0147] In the above fusion sub-network, preprocessing steps including rotation, affine transformation, displacement, flip, denoising, enhancement and the like can be added. In addition to directly performing channel splicing, channel-by-channel subtraction, spatial (horizontal or vertical) direction splicing and other fusion technologies can be used. In addition, in addition to fusion in the original Raw domain raw data, network can also be used to extract and fuse intermediate features to realize feature interaction between the template and the sample.

[0148] In the feature extraction sub-network, a specific convolutional neural network is constructed for feature extraction of the fusion information of the template and the sample. In addition, other CNN (Resnet, Densenet, Unet, etc.) architectures and Transformer architectures, MLP architectures, and other classification models can also be used as the feature extraction and classification network of the present application.

[0149] In addition to the parameter update and training based on the machine learning and deep learning training mechanism, fixed network parameters can also be manually configured at the same time of network construction, which can be directly used for chip end deployment without any training, to realize general fingerprint anti-counterfeiting.

[0150] The default is to use the registered template information that can match the texture to perform feature fusion. In addition, on the one hand, the registered template information can be shielded, and only the unlocking sample is used to realize independent anti-counterfeiting driven by artificial intelligence network; on the other hand, the remaining templates that fail to match can also be used to construct other information fusion methods.

[0151] The present application can achieve the following technical effects:

[0152] According to the physical principle of ultrasonic under-screen fingerprint data echo, a three-dimensional anti-counterfeiting scheme based on the fusion information of the registered template and the unlocking pressing raw domain is constructed, which fills the gap of ultrasonic under-screen fingerprint anti-counterfeiting technology; a segmented network is constructed, which innovatively combines the preprocessing sub-network, the feature extraction sub-network, and the feature fusion and classification sub-network architecture to realize end-to-end integrated training. The preprocessing sub-network involved can be guided based on different system modal data to realize the fitting and approximation of the preprocessing process of different fingerprint collection systems, and achieve the effect of preprocessing. Innovatively, on the basis of fingerprint matching, the template information that needs to be stored is greatly optimized, the feature domain description sub is used to replace the whole template image, the memory occupation is optimized to 1 / 80 of the original, the registered template and the unlocking sample signal are fused in the feature domain, and the effectiveness and usability of the ultrasonic fingerprint anti-counterfeiting are improved; the mainstream artificial intelligence technology is applied, a convolutional neural network is built, the neural network feature implicit extraction method is used to replace the manual explicit (acoustic impedance, distance, medium characteristics, etc.) feature extraction, and a feature domain fingerprint true and false information similarity measurement network is directionally constructed, which further improves the robustness of the anti-counterfeiting technology to the system environment. The template-sample fusion feature fingerprint anti-counterfeiting method constructed can be combined with the static anti-counterfeiting method that does not depend on the template, and the performance gain is embodied.

[0153] Those skilled in the art can clearly understand that the method according to the above-mentioned embodiments can be realized by means of software and necessary general hardware platforms, of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0154] In this embodiment, a fingerprint anti-counterfeiting neural network training device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0155] Figure 10 is a structural block diagram of the fingerprint anti-counterfeiting neural network training device according to the embodiment of the present application, as shown in Figure 10 , the device comprises:

[0156] The first acquisition module 1002 is configured to acquire a plurality of sets of training data, each set of training data comprising: first native domain data, second native domain data, and third native domain data, wherein the first native domain data is original data of a real finger collected when the real finger is input into a fingerprint template, the second native domain data is original data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is original data collected when a fake finger is used instead of the real finger for fingerprint verification.

[0157] The training module 1004 is configured to train an initial classification network using the plurality of sets of training data to obtain a target classification network, wherein the initial classification network comprises a fusion sub-network and a classification sub-network, and for each set of training data, the fusion sub-network is configured to generate a first fingerprint matching pair based on a feature description matrix of the first native domain data and the second native domain data, and generate a second fingerprint matching pair based on a feature description matrix of the first native domain data and the third native domain data, and the classification sub-network is configured to perform fingerprint classification and identification based on the first fingerprint matching pair and the second fingerprint matching pair.

[0158] In an example embodiment, the apparatus is further configured to perform iterative training on the initial classification network using multiple sets of training data, and stop the training to obtain a target classification network when a number of times of training on the initial classification network reaches a target number of times, or a loss value output by a target loss function of the initial classification network meets a preset training end condition; wherein the initial classification network further comprises a preprocessing subnetwork configured to convert native domain data collected by the fingerprint sensor into a feature description matrix; the loss value output by the target loss function is determined according to a loss value output by a first loss function of the preprocessing subnetwork and a loss value output by a second loss function of the classification subnetwork; the first loss function is configured to represent a loss between the feature description matrix output by the preprocessing subnetwork and a known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is configured to represent a loss between a predicted label value output by the classification subnetwork and a true label corresponding to the feature description matrix input by the classification subnetwork.

[0159] In an example embodiment, the apparatus is further configured to perform the kth training on the initial classification network by the following steps, wherein k is a positive integer: obtaining a kth set of training data used for the kth training on the initial classification network from the multiple sets of training data; inputting first, second and third native domain data in the kth set of training data into a preprocessing subnetwork in the classification network trained in the (k-1)th training to obtain first, second and third feature description matrices, respectively, wherein the initial classification network trained in the 0th training is the initial classification network without training, and the preprocessing subnetwork is configured to convert native domain data collected by the fingerprint sensor into a feature description matrix; inputting the first, second and third feature description matrices into a feature extraction subnetwork in the classification network trained in the (k-1)th training for dimension reduction processing to obtain fourth, fifth and sixth feature description matrices, respectively; inputting the fourth and fifth feature description matrices into a fusion subnetwork in the classification network trained in the (k-1)th training to obtain a first fingerprint matching pair for the kth training;

[0160] inputting the fourth and sixth feature description matrices into the fusion subnetwork in the classification network trained in the (k-1)th training to obtain a second fingerprint matching pair for the kth training;

[0161] input the first fingerprint matching pair of the kth training and the second fingerprint matching pair of the kth training into a classification subnetwork in the (k-1)th trained classification network, to obtain a predicted label of the kth training, wherein the predicted label of the kth training is used to represent a result of identifying the authenticity of the true finger fingerprint corresponding to the second native domain data in the kth training data by the (k-1)th trained classification network, and a result of identifying the authenticity of the false finger fingerprint corresponding to the third native domain data in the kth training data by the (k-1)th trained classification network;

[0162] According to the first feature description matrix, the second feature description matrix and the third feature description matrix and the known feature description matrix corresponding to the first native domain data, the second native domain data and the third native domain data in the kth training data, a loss value output by a first loss function of the kth training is determined, according to the predicted label of the kth training and a known true label of the kth training, a loss value output by a second loss function of the kth training is determined, according to the loss value output by the first loss function of the kth training and the loss value output by the second loss function of the kth training, a loss value output by a target loss function of the kth training is determined, when k is less than a target number, or in the case that the loss value output by the target loss function of the kth training does not meet a training end condition, the value of the parameter in the initial classification network of the (k-1)th training is adjusted to obtain an initial classification network of the kth training.

[0163] In an example embodiment, the target loss function is:

[0164] loss=α×loss MSE +β×loss BCE

[0165] Wherein, α, β are preset values, loss MSE is the first loss function of the preprocessing subnetwork, loss BCE is the second loss function of the classification subnetwork.

[0166] The classification network further comprises a feature extraction subnetwork, the preprocessing subnetwork is used to convert the native domain data acquired by the fingerprint sensor into a feature description matrix and input to the feature extraction subnetwork, and the feature extraction subnetwork is used to perform dimension reduction processing on the input matrix, and input the matrix description matrix obtained after the dimension reduction processing to the fusion subnetwork.

[0167] In an example embodiment, the loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula:

[0168]

[0169] loss MSE =loss MSE_1+loss MSE_2 +loss MSE_3

[0170] wherein t e (1, 3), i e (1, h), j e (1, w), f(i, j) is a feature description matrix output by the preprocessing subnetwork, y(i, j) is a known feature description matrix of the native domain data input to the preprocessing subnetwork, h is a vertical feature size of the feature description matrix, w is a horizontal feature size of the feature description matrix, loss MSE_1 is a loss value when the input data of the preprocessing subnetwork is the first native domain data, loss MSE_2 is a loss value when the input data of the preprocessing subnetwork is the second native domain data, loss MSE_3 is a loss value when the input data of the preprocessing subnetwork is the third native domain data.

[0171] In an example embodiment, the loss value output by the second loss function of the classification subnetwork is determined by the following formula:

[0172] loss BCE = -y x log(x) - (1 - y) x log(1 - x)

[0173] wherein y is a true label value of the first or second fingerprint matching pair, and x is a predicted label value of the first or second fingerprint matching pair output by the classification subnetwork.

[0174] According to another embodiment of the present application, there is provided a fingerprint anti-counterfeiting device, comprising: a second acquisition module configured to acquire native domain data of a fingerprint to be detected; a preprocessing module configured to preprocess the native domain data of the fingerprint to be detected by a preprocessing subnetwork in a target classification network to obtain a seventh feature description matrix; a feature extraction module configured to extract features from the seventh feature description matrix by a feature extraction subnetwork in the target classification network to obtain an eighth feature description matrix; a determination module configured to determine a ninth feature description matrix matching the eighth feature description matrix in a fingerprint template feature library, wherein the fingerprint template feature library stores a plurality of feature description matrices of a real finger when a fingerprint template is recorded; a fusion module configured to fuse the eighth feature description matrix and the ninth feature description matrix to obtain a fusion feature matrix; and an identification module configured to input the fusion feature matrix into a classification subnetwork in the target classification network to obtain a true or false identification result of the fingerprint to be detected, wherein the true or false identification result is used to indicate whether the fingerprint to be detected is a fingerprint of the real finger or a fingerprint of a fake finger.

[0175] In an example embodiment, the device is further configured to, before determining that the ninth feature descriptor matrix matches the eighth feature descriptor matrix in the fingerprint template feature library, acquire native domain data of the real finger when the fingerprint template is input; pre-process the native domain data using a pre-processing sub-network in the target classification network to obtain a feature descriptor matrix of the native domain data; extract features of the feature descriptor matrix using a feature extraction sub-network in the target classification network to obtain a target feature descriptor matrix of the native domain data; and store the target feature descriptor matrix in the fingerprint template feature library.

[0176] It should be noted that the above modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0177] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method in any of the above embodiments.

[0178] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0179] Embodiments of the present application also provide an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the method in any of the above embodiments.

[0180] In an example embodiment, the electronic device can further comprise a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0181] The specific examples in the present embodiment can refer to the examples described in the above embodiments and example embodiments, and will not be described herein again.

[0182] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method in the embodiments of the present application.

[0183] The embodiments of the present application further provide a computer program product comprising a non-transitory computer readable medium storing a computer program, wherein the computer program is executed by a processor to implement the steps of the method described in various embodiments of the present application.

[0184] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, which can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be respectively manufactured into individual integrated circuit modules, or multiple modules or steps among them can be manufactured into a single integrated circuit module to realize. Thus, the present application is not limited to any specific combination of hardware and software.

[0185] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a fingerprint anti-counterfeiting neural network, characterized in that, include: Multiple sets of training data are acquired, each set of training data including: first native domain data, second native domain data and third native domain data, wherein the first native domain data is the raw data of the real finger collected when the real finger is entered into the fingerprint template, the second native domain data is the raw data of the real finger collected when the real finger is used for fingerprint verification, and the third native domain data is the raw data collected when a fake finger is used to replace the real finger for fingerprint verification; The initial classification network is trained using the multiple sets of training data to obtain the target classification network. The initial classification network includes a fusion subnetwork and a classification subnetwork. For each set of training data, the fusion subnetwork performs feature fusion based on the feature description matrices of the first native domain data and the second native domain data to generate a first fingerprint matching pair, and performs feature fusion based on the feature description matrices of the first native domain data and the third native domain data to generate a second fingerprint matching pair. The classification subnetwork performs fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

2. The method according to claim 1, characterized in that, The step of training the initial classification network using the multiple sets of training data to obtain the target classification network includes: The initial classification network is iteratively trained using the multiple sets of training data until the number of times the initial classification network is trained reaches the target number, or until the loss value output by the target loss function of the initial classification network meets the preset training termination condition, at which point the training ends and the target classification network is obtained. The initial classification network further includes a preprocessing subnetwork, which is used to convert the native domain data collected by the fingerprint sensor into a feature description matrix. The loss value output by the target loss function is determined based on the loss value output by the first loss function of the preprocessing subnetwork and the loss value output by the second loss function of the classification subnetwork. The first loss function is used to represent the loss between the feature description matrix output by the preprocessing subnetwork and the known feature description matrix of the native domain data input by the preprocessing subnetwork, and the second loss function is used to represent the loss between the predicted label value output by the classification subnetwork and the true label corresponding to the feature description matrix input by the classification subnetwork.

3. The method according to claim 2, characterized in that, The iterative training of the initial classification network using the multiple sets of training data includes: The initial classification network is trained for the kth time using the following steps, where k is a positive integer: From the multiple sets of training data, obtain the kth set of training data used for the kth training of the initial classification network; The first native domain data, the second native domain data, and the third native domain data from the kth training data are input into the preprocessing subnetwork of the classification network trained in the (k-1)th time to obtain the first feature description matrix, the second feature description matrix, and the third feature description matrix, respectively. The initial classification network trained in the 0th time is the untrained initial classification network. The first feature description matrix, the second feature description matrix, and the third feature description matrix are input into the feature extraction subnetwork of the classification network trained in the (k-1)th time for dimensionality reduction, to obtain the fourth feature description matrix, the fifth feature description matrix, and the sixth feature description matrix, respectively. The fourth feature description matrix and the fifth feature description matrix are input into the fusion subnetwork of the classification network trained in the (k-1)th time to obtain the first fingerprint matching pair trained in the kth time. The fourth feature description matrix and the sixth feature description matrix are input into the fusion subnetwork of the classification network trained in the (k-1)th time to obtain the second fingerprint matching pair trained in the kth time. The first fingerprint matching pair and the second fingerprint matching pair trained in the kth training are input into the classification subnetwork in the classification network trained in the (k-1)th training to obtain the predicted label of the kth training. The predicted label of the kth training is used to represent the authenticity recognition result of the classification network trained in the (k-1)th training on the real fingerprint corresponding to the second native domain data in the kth training data, and the authenticity recognition result of the classification network trained in the (k-1)th training on the fake fingerprint corresponding to the third native domain data in the kth training data. Based on the first feature description matrix, the second feature description matrix, and the third feature description matrix, and the known feature description matrices corresponding to the first native domain data, the second native domain data, and the third native domain data in the k-th training data, the loss value output by the first loss function in the k-th training is determined. Based on the predicted label in the k-th training and the known true label in the k-th training, the loss value output by the second loss function in the k-th training is determined. Based on the loss value output by the first loss function in the k-th training and the loss value output by the second loss function in the k-th training, the loss value output by the target loss function in the k-th training is determined. When k is less than the target number of training iterations, or when the loss value output by the target loss function in the k-th training does not meet the training termination condition, the values ​​of the parameters in the initial classification network in the (k-1)-th training are adjusted to obtain the initial classification network in the k-th training.

4. The method according to claim 2, characterized in that, The target loss function is: loss=α×loss MSE +β×loss BCE Where α and β are preset values, and loss MSE It is the first loss function of the preprocessing subnetwork, loss BCE It is the second loss function of the classification subnetwork; The classification network further includes a feature extraction sub-network. The preprocessing sub-network is used to convert the native domain data acquired by the fingerprint sensor into a feature description matrix and input it into the feature extraction sub-network. The feature extraction sub-network is used to perform dimensionality reduction processing on the input feature description matrix and input the matrix obtained after the dimensionality reduction processing into the fusion sub-network.

5. The method according to claim 4, characterized in that, The loss value output by the first loss function of the preprocessing subnetwork is determined by the following formula: loss MSE =loss MSE_1 +loss MSE_2 +loss MSE_3 Where t∈(1,3), i∈(1,h), j∈(1,w), f(i,j) is the feature description matrix output by the preprocessing subnetwork, y(i,j) is the known feature description matrix of the native domain data input to the preprocessing subnetwork, h is the vertical feature size of the feature description matrix, w is the horizontal feature size of the feature description matrix, and loss MSE_1 The input data of the preprocessing sub-network is the loss value of the first native domain data. MSE_2 The input data of the preprocessing sub-network is the loss value of the second native domain data. MSE_3 The input data of the preprocessing subnetwork is the loss value of the third native domain data.

6. The method according to claim 4, characterized in that, The loss value output by the second loss function of the classification subnetwork is determined by the following formula: loss BCE =-y×log(x)-(1-y)×log(1-x) Where y is the true label value of the first fingerprint matching pair or the second fingerprint matching pair, and x is the predicted label value of the first fingerprint matching pair or the second fingerprint matching pair output by the classification sub-network.

7. A fingerprint anti-counterfeiting method, characterized in that, include: Obtain the native domain data of the fingerprint to be detected; The seventh feature description matrix is ​​obtained by preprocessing the original domain data of the fingerprint to be detected through the preprocessing subnetwork in the target classification network. The seventh feature description matrix is ​​extracted by the feature extraction subnetwork in the target classification network to obtain the eighth feature description matrix. A ninth feature description matrix that matches the eighth feature description matrix is ​​determined in the fingerprint template feature library, wherein the fingerprint template feature library stores multiple feature description matrices of a real finger when the fingerprint template is entered. The eighth feature description matrix and the ninth feature description matrix are fused to obtain a fused feature matrix; The fused feature matrix is ​​input into the classification subnetwork of the target classification network to obtain the authenticity identification result of the fingerprint to be detected, wherein the authenticity identification result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

8. The method according to claim 7, characterized in that, Before determining the ninth feature description matrix that matches the eighth feature description matrix in the fingerprint template feature library, the method further includes: Obtain the native domain data of the real finger when the fingerprint template is entered; The native domain data is preprocessed using the preprocessing subnetwork in the target classification network to obtain the feature description matrix of the native domain data; The feature description matrix is ​​reduced in dimensionality using the feature extraction subnetwork in the target classification network to obtain the reduced-dimensional feature description matrix. The reduced-dimensional feature description matrix is ​​stored in the fingerprint template feature library.

9. A training device for a fingerprint anti-counterfeiting neural network, characterized in that, include: The first acquisition module is used to acquire multiple sets of training data. Each set of training data includes: first native domain data, second native domain data, and third native domain data. The first native domain data is the original data of the real finger collected when the real finger is entered into the fingerprint template. The second native domain data is the original data of the real finger collected when the real finger is used for fingerprint verification. The third native domain data is the original data collected when a fake finger is used to replace the real finger for fingerprint verification. The training module is used to train an initial classification network using the multiple sets of training data to obtain a target classification network. The initial classification network includes a fusion sub-network and a classification sub-network. For each set of training data, the fusion sub-network is used to perform feature fusion based on the feature description matrices of the first native domain data and the second native domain data to generate a first fingerprint matching pair, and to perform feature fusion based on the feature description matrices of the first native domain data and the third native domain data to generate a second fingerprint matching pair. The classification sub-network is used to perform fingerprint classification and recognition based on the first fingerprint matching pair and the second fingerprint matching pair.

10. A fingerprint anti-counterfeiting device, characterized in that, include: The second acquisition module is used to acquire the native domain data of the fingerprint to be detected; The preprocessing module is used to preprocess the native domain data of the fingerprint to be detected through the preprocessing subnetwork in the target classification network to obtain the seventh feature description matrix; The feature extraction module is used to extract features from the seventh feature description matrix through the feature extraction subnetwork in the target classification network to obtain the eighth feature description matrix; The determining module is used to determine a ninth feature description matrix that matches the eighth feature description matrix in the fingerprint template feature library, wherein the fingerprint template feature library stores multiple feature description matrices of a real finger when the fingerprint template is entered. The fusion module is used to fuse the eighth feature description matrix and the ninth feature description matrix to obtain a fused feature matrix; The identification module is used to input the fused feature matrix into the classification subnetwork of the target classification network to obtain the authenticity identification result of the fingerprint to be detected, wherein the authenticity identification result is used to indicate whether the fingerprint to be detected is the fingerprint of the real finger or the fingerprint of the fake finger.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 6 or 7 to 8.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 6 or 7 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6 or 7 to 8.

Images