Method and device for identifying consistency of fingerprint media

Abstract

The invention relates to the technical field of signal processing, and discloses a fingerprint medium consistency identification method comprising the following steps: obtaining current verification fingerprint data; searching target fingerprint template data matched with the current verification fingerprint data in a registered fingerprint template data set; searching a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set; and generating a verification feature descriptor by using the pre-trained feature extraction network according to the current verification fingerprint data, and generating a current medium consistency probability of verification fusion features by using a pre-trained feature fusion classification network according to the verification feature descriptor and the target feature descriptor. According to the invention, the accuracy of fingerprint classification identification is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of signal processing technology, and in particular to a method and device for identifying fingerprint medium consistency. Background Art

[0002] Fingerprint recognition is a popular biometric technology. This technology plays an important role in identity authentication in many fields such as smartphone unlocking, financial transactions, and access control. In the past decade, with the upgrade of sensor technology, fingerprint recognition has undergone a system change from capacitive fingerprint to under-screen optical fingerprint to under-screen ultrasonic fingerprint. The above-mentioned fingerprint sensor obtains sound, light, and electrical signals to perform two-dimensional imaging of the fingerprint pattern pressed on the sensor, and performs two-dimensional matching with the registered fingerprint template data to achieve identity recognition.

[0003] In recent years, anti-counterfeiting solutions for ultrasonic fingerprints have received increasing attention. These methods are mainly based on explicit calculation and manual fitting of real-time physical characteristics such as acoustic impedance and surface distance based on ultrasonic echo signals. By comparing these manual features with the distance of offline static real finger physical characteristic fitting, the binary classification of live real fingers and non-live fake fingerprints can be achieved.

[0004] However, the above-mentioned fingerprint recognition method uses static offline collected fingers as the analysis object, and the set classification threshold interval and statistical distribution law are heavily dependent on the statistical characteristics of the collected real and fake fingers. When identifying real fingerprints of extreme people whose fingerprints are out of statistical characteristics (such as peeling, wide lines, shallow lines, etc.) or fake fingerprints made of new media materials, fingerprint classification errors are prone to occur, resulting in the situation that the real fingerprint that is the same as the template cannot be unlocked or the fake fingerprint of the new media material is successfully impersonated. Summary of the invention

[0005] In view of this, the present disclosure provides a method for identifying fingerprint medium consistency to solve the problem of how to reduce the current fingerprint classification and recognition errors.

[0006] In a first aspect, the present disclosure provides a method for identifying fingerprint medium consistency, the method comprising:

[0007] Get the current verification fingerprint data;

[0008] In the registered fingerprint template data set, search for target fingerprint template data that matches the current verification fingerprint data;

[0009] Searching for a target feature descriptor corresponding to the target fingerprint template data in the template feature descriptor set, wherein the template feature descriptor set is generated based on the registered fingerprint template data set using a pre-trained feature extraction network;

[0010] A pre-trained feature extraction network is used to generate a verification feature descriptor based on the current verification fingerprint data. A pre-trained feature fusion classification network is used to generate a current medium consistency probability of a verification fusion feature based on the verification feature descriptor and the target feature descriptor. The fusion feature is obtained by fusing the verification feature descriptor and the target feature descriptor.

[0011] In a second aspect, the present disclosure provides a device for identifying fingerprint medium consistency, the device comprising:

[0012] A data acquisition module is used to acquire the current verification fingerprint data;

[0013] A data search module, used to search for target fingerprint template data matching the current verification fingerprint data in the registered fingerprint template data set;

[0014] A feature search module is used to search for a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set, where the template feature descriptor set is generated based on the registered fingerprint template data set using a pre-trained feature extraction network;

[0015] The probability generation module is used to generate a verification feature descriptor based on the current verification fingerprint data using a pre-trained feature extraction network, and to generate a current medium consistency probability of a verification fusion feature based on the verification feature descriptor and the target feature descriptor using a pre-trained feature fusion classification network. The fusion feature is obtained by fusing the verification feature descriptor and the target feature descriptor.

[0016] In a third aspect, the present disclosure provides a device, comprising:

[0017] Fingerprint sensor system, used to obtain current verification fingerprint data;

[0018] Memory systems; and

[0019] The control system is configured to electrically communicate with the ultrasonic fingerprint sensor system. The control system is also configured to: execute the method for identifying fingerprint medium consistency according to the first aspect or any corresponding embodiment thereof.

[0020] In a fourth aspect, the present disclosure provides a device, comprising:

[0021] Fingerprint sensor system, used to obtain current verification fingerprint data;

[0022] Memory systems; and

[0023] A control device is used to enable a computer to execute the method for identifying fingerprint medium consistency of the first aspect or any corresponding embodiment thereof.

[0024] The method for identifying fingerprint medium consistency provided by the above-mentioned embodiment of the present disclosure adopts a pre-trained feature extraction network to generate a verification feature descriptor of the current verification fingerprint data, and determines a target feature descriptor of the target fingerprint template data matching the current verification fingerprint data, and then adopts a pre-trained feature fusion classification network to generate a current medium consistency probability corresponding to the verification fusion feature according to the verification feature descriptor and the target feature descriptor, so that when the current medium consistency probability is greater than a probability threshold, it is determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data, without considering the statistical distribution of the collected true and false finger fingerprints, thereby avoiding the problems of poor finger stability, poor system stability and poor scalability in determining fingerprint classification according to the statistical distribution of fingerprints, and improving the accuracy of fingerprint classification and identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the specific embodiments of the present disclosure, the drawings required for use in the description of the specific embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a flow chart of a method for identifying fingerprint medium consistency according to an embodiment of the present disclosure;

[0027] Figure 2 It is a flowchart of a method for training a feature extraction network and a feature fusion classification network according to an embodiment of the present disclosure;

[0028] Figure 3a is a flowchart of network training and fingerprint identification of a method for identifying fingerprint medium consistency according to an embodiment of the present disclosure;

[0029] Figure 3b is an exemplary schematic diagram of a triple pair sample according to an embodiment of the present disclosure;

[0030] Figure 3c is a schematic diagram of native domain data of fingerprint data according to an embodiment of the present disclosure;

[0031] Figure 3d is a schematic diagram of fingerprint imaging data of fingerprint data according to an embodiment of the present disclosure;

[0032] Figure 3e is a schematic diagram of fingerprint data preprocessing and enhanced fingerprint imaging according to an embodiment of the present disclosure;

[0033] Figure 3f is a schematic diagram of a feature domain vector representation based on a deep convolutional neural network according to an embodiment of the present disclosure;

[0034] Figure 3g is a flow chart of offline network training and online chip deployment according to an embodiment of the present disclosure;

[0035] Figure 4 is a structural block diagram of a device for identifying fingerprint medium consistency according to an embodiment of the present disclosure;

[0036] Figure 5 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0038] Fingerprint recognition is a popular biometric technology that plays an important role in identity verification in many fields, such as smartphone unlocking, financial transactions, and access control.

[0039] Over the past decade, along with the upgrade of sensor technology, fingerprint recognition has undergone a system change from capacitive fingerprint to under-screen optical fingerprint and then to under-screen ultrasonic fingerprint. The above fingerprint sensors obtain sound, light, and electrical signals to perform two-dimensional imaging of the fingerprint pattern pressed on the sensor, and perform two-dimensional matching with the registered fingerprint template data to achieve identity recognition. This stage is called the "plane matching" stage.

[0040] In recent years, anti-counterfeiting solutions for ultrasonic fingerprints have received increasing attention. These methods are mainly based on explicit calculation and manual fitting of real-time physical characteristics such as acoustic impedance and surface distance based on ultrasonic echo signals, and achieve binary classification of live real fingers and non-live fake fingerprints by comparing these manual features with the distance of offline static real finger physical characteristic fittings.

[0041] The main technical disadvantages of the above-mentioned related technologies include:

[0042] a. Poor finger stability: The above-mentioned "manual feature"-oriented solution uses static offline collected fingers as the analysis object, and the set classification threshold interval and statistical distribution law are heavily dependent on the statistical characteristics of the collected real and fake fingers. This leads to serious problems such as misclassification or even failure to unlock the phone for some extreme people's fingers (peeling, wide lines, shallow lines, etc.).

[0043] b. Poor system stability: In practical applications, considering the differences between the system environment of mass-produced machines and the system environment of the machines from which data is collected, it is difficult to directly apply the statistical distribution obtained by collecting samples from fixed machines to all machines. This also makes the system robustness of the mentioned method low and the efficiency of mass production adaptation low.

[0044] c. Poor scalability: The above-mentioned related technologies focus on the binary classification problem of identifying "real" and "fake". Once the "real" fingers of special groups and the "fake" fingerprints made of new media materials appear, the classification model built based on historical data will become invalid and the parameters need to be updated to adapt to the new media binary classification problem.

[0045] In order to effectively measure the media identity of the registered fingerprint template data and the current verification fingerprint data (i.e., the current unlocking and pressing sample) in high-dimensional space based on pattern matching, the present invention relies on deep learning technology to propose a method for identifying fingerprint media consistency, which is not limited to the fixed concept of classifying "live" and "non-live", but focuses on measuring the media identity of the registered fingerprint template data and the current verification fingerprint data (i.e., the current unlocking and pressing sample). The invention is divided into two main stages: offline training and online chip-side deployment.

[0046] According to an embodiment of the present disclosure, a method embodiment for identifying fingerprint medium consistency is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] In this embodiment, a method for identifying fingerprint medium consistency is provided, and a client or a server can be used as the execution subject. Among them, the client can be a mobile terminal, such as a mobile phone, a tablet computer, etc., or a fixed terminal device. The server can obtain the current verification fingerprint data sent by the client, and return the result obtained by executing the method for identifying fingerprint medium consistency to the client. The result can be: the current medium consistency probability, the medium consistency identification result determined based on the current medium consistency probability, or the fingerprint identification result determined based on the current medium consistency probability.

[0048] Figure 1 is a flow chart of a method for identifying fingerprint medium consistency according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:

[0049] Step S101, obtaining current verification fingerprint data.

[0050] In this embodiment, the execution subject may obtain the original Raw domain fingerprint data of the object to be detected in its original state without any processing through a fingerprint sensor in current or future technology, for example, obtaining Raw domain image data or signal data.

[0051] For example, the above-mentioned execution subject can obtain the current verification fingerprint data through a capacitive fingerprint sensor: when the finger is pressed on the sensor, the ups and downs of the fingerprint will form different capacitance values, which will be converted into a digital fingerprint signal.

[0052] The above-mentioned execution entity can use an optical fingerprint sensor to obtain the current verification fingerprint data: the optical fingerprint sensor captures the light reflected by the fingerprint pressed on the screen to form a fingerprint image.

[0053] The above-mentioned execution subject may use an ultrasonic fingerprint sensor to obtain the current verification fingerprint data: the ultrasonic fingerprint sensor draws a three-dimensional image of the fingerprint by sending ultrasonic pulses and receiving their reflected signals.

[0054] Afterwards, the execution subject may use the Raw domain fingerprint data or a base fingerprint image obtained by preprocessing the Raw domain fingerprint data as the current verification fingerprint data.

[0055] Step S102: searching the registered fingerprint template data set for target fingerprint template data that matches the current verification fingerprint data.

[0056] In this embodiment, after the current verification fingerprint data is acquired, it can be compared with the templates in the registered fingerprint template data set to find the target fingerprint template data that matches the current verification fingerprint data.

[0057] The registered fingerprint template data set may include a plurality of fingerprint feature templates, which are obtained by converting the user's fingerprint image into templates in advance and then storing them.

[0058] The process of comparing the current verification fingerprint data with the templates in the registered fingerprint template data set usually involves extracting the features of the current verification fingerprint data (such as ridge endpoints, bifurcation points, detail points, etc.) and comparing them with the features in the registered fingerprint template data.

[0059] The comparison algorithm may adopt various matching strategies in existing technologies or future technologies, which are not limited in the present disclosure. For example, the comparison algorithm may adopt matching based on minutiae points, matching based on patterns, etc., so as to find the target fingerprint template data that matches the current verification fingerprint data.

[0060] It is understandable that before comparing the current verification fingerprint data with the registered fingerprint template data, the current verification fingerprint data may also be preprocessed. The preprocessing steps may include denoising, contrast enhancement, binarization, direction estimation, segmentation, etc., so as to better extract fingerprint features.

[0061] In some specific examples, preprocessing the current verification fingerprint data may include at least one of the following: spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing, and time domain amplitude extraction.

[0062] Step S103: searching the template feature descriptor set for the target feature descriptor corresponding to the target fingerprint template data.

[0063] In this embodiment, the template feature descriptor set is generated according to the registered fingerprint template data set using a pre-trained feature extraction network. Therefore, there is a one-to-one mapping relationship between the registered fingerprint template in the registered fingerprint template data set and the template feature descriptor in the template feature descriptor set. According to the mapping relationship, the above-mentioned execution subject can map the position of the target fingerprint template data in the registered fingerprint template data set to the corresponding position in the registered fingerprint template data set, and use the feature descriptor of the corresponding position as the target feature descriptor.

[0064] Step S104, using a pre-trained feature extraction network to generate a verification feature descriptor based on the current verification fingerprint data, and using a pre-trained feature fusion classification network to generate a current medium consistency probability of the verification fusion feature based on the verification feature descriptor and the target feature descriptor.

[0065] In this embodiment, the pre-trained feature extraction network can be implemented using a deep learning model to extract a highly representative feature vector, that is, a feature descriptor, from the original fingerprint image or fingerprint signal.

[0066] These feature descriptors capture key information in fingerprint images or fingerprint signals, such as fingerprint ridges, valleys, frequencies, detail points (such as endpoints and bifurcation points), etc., but remove unnecessary details and noise in order to convert high-dimensional, complex fingerprint image data into low-dimensional, easy-to-compare and process feature vectors.

[0067] After using a pre-trained feature extraction network to generate a verification feature descriptor based on the current verification fingerprint data, the above-mentioned execution entity can use a pre-trained feature fusion classification network to fuse the verification feature descriptor and the target feature descriptor to obtain a verification fusion feature, and then determine the current medium consistency probability of the fusion feature.

[0068] The method of fusing to obtain the verified fusion features may include simple concatenation, weighted summation, or a more complex learning-based fusion method. In some specific examples, at least one of the following may be used to generate the verified fusion features: a dimension-by-dimension mean regularization fusion mode, a quantization fusion mode, a spatial channel stacking fusion mode, and a dimension-by-dimensional differential fusion mode.

[0069] The current medium consistency probability may be a medium identity score. The probability threshold is a score value for determining whether the verification feature descriptor and the target feature descriptor belong to the same medium.

[0070] After determining the current medium consistency probability of the fused feature, when the current medium consistency probability is greater than the probability threshold, it can be determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data. In a specific example, when the medium identity score is greater than 0.5, it can be determined that the current verification fingerprint data and the target fingerprint template data are the same medium, otherwise they are different media.

[0071] The medium here may include: fingerprint medium in the existing technology or future development technology, which is not limited in this application. In some specific examples, the medium of the target fingerprint template data may include at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder and dust.

[0072] It is understandable that when it is determined that the current verification fingerprint data and the target fingerprint template data are of the same medium or different media, the verification and identification result of the current verification fingerprint can also be output according to the determination result.

[0073] The method for identifying fingerprint medium consistency provided in the present embodiment uses a pre-trained feature extraction network to generate a verification feature descriptor of the current verification fingerprint data, and determines a target feature descriptor of the target fingerprint template data that matches the current verification fingerprint data. Then, a pre-trained feature fusion classification network is used to generate a current medium consistency probability corresponding to the verification fusion feature according to the verification feature descriptor and the target feature descriptor. Therefore, when the current medium consistency probability is greater than a probability threshold, it is determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data, without considering the statistical distribution of the fingerprint, thereby avoiding the problems of poor finger stability, poor system stability and poor scalability in determining fingerprint classification based on the statistical distribution of the fingerprint, and improving the accuracy of fingerprint classification and identification.

[0074] In this embodiment, a method for training a feature extraction network and a feature fusion classification network is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 2 is a flow chart of a method for training a feature extraction network and a feature fusion classification network according to an embodiment of the present disclosure. Figure 2 As shown, the process includes the following steps:

[0075] Step S201, obtaining a triple pair sample in a triple pair sample set.

[0076] In this embodiment, the ternary pair samples include an anchor sample, a positive training sample, and a negative training sample of the fingerprint data. The anchor sample and the positive training sample belong to the same template medium, and the anchor sample and the negative training sample belong to different template media.

[0077] When acquiring the ternary pair samples, the native domain signal sampling of more than 2 (including 2) media can be performed using the established fingerprint sensor (capacitive, optical, ultrasonic). The pressed native domain data sets of different media components can be represented as A, B, C, ...

[0078] After that, a triple pair of samples can be constructed. In a specific example, each triple pair of samples includes two samples i and j from the same medium data set, and another sample k from a different medium data set. Among them, samples i, j and k can correspond to positive training samples, anchor point samples and negative training samples respectively.

[0079] Step S202, using the initial feature extraction network, based on the triple pair samples, three feature descriptors are generated.

[0080] In this embodiment, the three feature descriptors may include: an anchor feature descriptor of an anchor sample, a positive feature descriptor of a positive training sample, and a negative feature descriptor of a negative training sample.

[0081] The initial feature extraction network is the initial network of the feature extraction network pre-trained in step S104, and has the same structure and function, except that the network parameter values ​​are the initial parameter values ​​that have not been updated and iterated, which will not be described in detail here.

[0082] In a specific example, the initial feature extraction network can be implemented using a deep convolutional neural network. Perform feature mapping to obtain three feature vectors of length 10

[0083] Step S203, using the initial feature fusion classification network, based on the three feature descriptors, generates the medium consistency probability of the sample fusion feature, generates the loss of the medium consistency probability and the true label of the sample fusion feature, and uses the reverse gradient conduction of the loss to update the parameter values ​​of the initial feature fusion classification network and the parameter values ​​of the initial feature extraction network until the number of training reaches the preset number of iterations or the loss meets the preset target, thereby obtaining the pre-trained feature extraction network and the pre-trained feature fusion classification network.

[0084] In this embodiment, the sample fusion features may include: positive sample fusion features obtained by fusing an anchor feature descriptor and a positive feature descriptor, and negative sample fusion features obtained by fusing an anchor feature descriptor and a negative feature descriptor.

[0085] Specifically, in a specific example, the three feature vectors of length 10 obtained by mapping the ternary pair can be combined in pairs, where two 10-dimensional feature vectors from the same medium data set are combined into a same-medium positive sample fusion feature The label is assigned as 1, and the two 10-dimensional feature vectors from different medium datasets are combined into a non-homologous negative sample fusion feature Assign label 0.

[0086] The initial feature fusion classification network can be obtained by cascading the network used for feature fusion and classification in the prior art, and the present disclosure does not limit this. In a specific example, a fully connected network can be used to construct a decision classification layer, and based on the positive and negative sample fusion features, the medium consistency probability of the positive and negative sample fusion features is predicted to complete the medium identity classification and identification of the positive and negative sample fusion features.

[0087] Among them, the loss of the medium consistency probability and the true label of the positive and negative sample fusion features can be realized by using the loss function used in existing technologies or future technologies to determine the loss between the predicted value and the true value of the classification problem, and the present disclosure does not limit this.

[0088] In a specific example, based on the consistency between the classification results of the fusion features of positive and negative samples and the true labels, a binary cross entropy loss function can be used for reverse gradient conduction:

[0089] Loss BCE = -y·log(x)-(1-y)·log(1-x)

[0090] Among them, Loss BCE It is the loss of the medium consistency probability predicted based on the fusion features of positive and negative samples and the true label, y is the true label (0 or 1), and x is the probability of predicting the positive sample fusion feature. When yh = 1 (that is, the sample belongs to the positive class), the loss function is simplified to Loss BCE = -log(x), the loss value is close to 0, and the loss will be very large. When y = 0 (that is, the sample belongs to the negative class), the loss function is simplified to Loss BCE =-log(1-x), the loss value is close to 1, and the loss will be very large.

[0091] At this time, in the order of gradient reverse conduction, the parameters of the fully connected decision classification layer and the deep convolutional neural network are updated in turn to achieve performance optimization and training iterations until the number of epochs for the provided training data set reaches 200 times, or the loss function reaches the preset value, the iteration ends, the reverse gradient conduction is terminated, the weight coefficients in the network are no longer updated, and the fully connected decision classification layer and the deep convolutional neural network are saved separately.

[0092] The method for training the feature extraction network and the feature fusion classification network in this embodiment uses the label guidance of the sample fusion feature of the registered fingerprint template data, and uses the pre-trained feature extraction network and the pre-trained feature fusion classification network to predict the medium consistency probability of the positive and negative sample fusion features according to the ternary pair samples, generates the loss with the real label, and then uses the reverse gradient conduction of the loss to update the parameter value of the initial feature fusion classification network and the parameter value of the initial feature extraction network until the number of training reaches the preset number of iterations or the loss meets the preset target. The pre-trained feature extraction network and the pre-trained feature fusion classification network obtained through this training process effectively realize the feature aggregation of the same medium and the extraction of implicit medium features, realize the prediction of the medium consistency probability, overcome the differentiated interference of different system hardware environment parameters and different finger types, present exclusive distribution characteristics for non-homologous media, and have more robust non-same medium attack detection capabilities.

[0093] In some optional implementations of this embodiment, the initial feature extraction network includes: a spatial domain enhancement sub-network, a feature extraction sub-network, and a medium description sub-network connected in sequence. Using the initial feature extraction network, three feature descriptors are generated based on the ternary pair samples, including: using the spatial domain enhancement sub-network to pre-process and enhance the ternary pair samples to obtain a pre-processed feature tensor; using the feature extraction sub-network to reduce the spatial resolution of the pre-processed feature tensor and increase the number of channels of the pre-processed feature tensor to obtain a spatial domain condensed feature tensor; using the medium description sub-network, the spatial domain condensed feature tensor is integrated and semantically extracted to obtain a feature descriptor of a preset number of dimensions that characterizes the characteristics of the physical medium.

[0094] In this implementation, the initial feature extraction network, through the cascade of the spatial enhancement subnetwork, the feature extraction subnetwork and the medium description subnetwork, can extract deep descriptors with physical medium characteristics from the input fingerprint data, providing powerful feature representation for subsequent classification, recognition or analysis tasks.

[0095] The spatial domain enhancement sub-network can preprocess and enhance the input fingerprint data, improve the image quality through spatial domain operations (such as spatial domain filtering, contrast enhancement, sharpening, etc.), remove noise, enhance the visibility and contrast of fingerprint lines, and obtain preprocessed feature tensors. This process ensures that the subsequent processing stage can extract features from clearer and higher quality images.

[0096] The feature extraction subnetwork can further process the preprocessed feature tensor, reduce the spatial resolution through operations such as convolutional layers and pooling layers, and increase the number of channels to achieve feature concentration and abstraction, and obtain a spatially concentrated feature tensor. This reduces the computational burden while maintaining key fingerprint information, increases the feature description capability, and prepares for subsequent advanced feature extraction.

[0097] The medium descriptor network can perform in-depth information integration and semantic extraction on the spatially concentrated feature tensor. It can obtain the intrinsic relationship and semantic meaning between features through advanced feature extraction methods such as fully connected layers, recurrent neural networks or attention mechanisms, thereby obtaining feature descriptors of a preset number of dimensions that characterize the characteristics of the physical medium. It can extract deep features that reflect the characteristics of the physical medium (such as the microstructure of the finger skin, sweat distribution, contact pressure, etc.) from the fingerprint image, so as to more effectively perform fingerprint recognition, classification or analysis.

[0098] The initial feature extraction network in this embodiment realizes a high-level representation from raw fingerprint data to physical medium characteristics through a gradually deepening feature extraction process, completes the mapping from low-level visual features or sensory features to high-level semantic features, and improves the effectiveness of the feature descriptor of the generated input fingerprint data.

[0099] In some optional implementations of the present embodiment, the above-mentioned spatial domain enhancement subnetwork includes: a first convolutional layer; the feature extraction subnetwork includes: a first convolutional block, a first transition layer, a second convolutional block and a second transition layer connected in sequence; wherein the first convolutional block uses a plurality of convolutional kernels of preset sizes to be stacked continuously; the first transition layer includes a second convolutional layer and a first average pooling layer connected in sequence; the second convolutional block uses a plurality of convolutional kernels of preset sizes to be stacked continuously; the second transition layer includes a third convolutional layer and a second average pooling layer connected in sequence; the medium description subnetwork includes: a depthwise separable convolutional layer, and the depthwise separable convolutional layer includes a fourth convolutional layer and a third average pooling layer connected in sequence.

[0100] In this implementation, the spatial domain enhancement subnetwork starts with the first convolutional layer to perform preliminary feature extraction and spatial domain image enhancement on the input fingerprint data. The first convolutional layer applies a convolution kernel (also called a filter) in a sliding window manner to extract image features, which can capture local features in the image, such as edges, corners, etc., providing a basis for subsequent processing.

[0101] The feature extraction subnetwork is composed of a first convolutional block, a first transition layer, a second convolutional block, and a second transition layer connected in sequence.

[0102] Among them, the first convolution block uses multiple convolution kernels of preset sizes to stack continuously, that is, multiple convolution layers with convolution kernels of preset sizes in the deep convolutional neural network are connected in series. The first layer usually learns low-level features such as edges and textures, while the deep convolution kernels can combine the features of the previous layer to learn more complex and abstract features, such as the local structure, shape or pattern of the object, thereby deeply extracting image features.

[0103] The first transition layer includes the second convolution layer and the first average pooling layer. The second convolution layer may be used for further feature extraction or feature conversion, translation and local information reorganization. The first average pooling layer is used to reduce the spatial dimension of the feature map (i.e., downsampling), while retaining important information, reducing the amount of calculation, and integrating information.

[0104] The second convolution block, similar to the first convolution block, can use the same or different convolution kernel configuration as the first convolution block to extract deeper features.

[0105] The second transition layer, including the third convolutional layer and the second average pooling layer, has a similar function to the first transition layer and can be implemented with the same or different configuration as the first transition layer to integrate and optimize information for deeper feature maps.

[0106] The medium description subnetwork ends the feature extraction network with a depthwise separable convolutional layer. The depthwise separable convolutional layer includes the fourth convolutional layer and the third average pooling layer. The fourth convolutional layer realizes the fusion of high-dimensional information after the information is integrated, and the third average pooling layer completes the integration of the fused information, thereby realizing the description of the medium.

[0107] The initial feature extraction network in this implementation method realizes the extraction of high-level features of the input fingerprint data by further clarifying the network structure of the spatial enhancement subnetwork, the feature extraction subnetwork and the medium description subnetwork, thereby further improving the effectiveness of the feature descriptor of the input fingerprint data generated by the feature extraction network.

[0108] In some optional implementations of the present embodiment, the pre-trained feature extraction network and the pre-trained feature fusion classification network are trained through the following steps, and also include: performing at least one of the following pre-processing imaging on the ternary pair samples: spatial rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing and time domain amplitude extraction; using the initial feature extraction network to generate three feature descriptors based on the ternary pair samples, including: using the initial feature extraction network to generate three feature descriptors based on the ternary pair samples after pre-processing imaging.

[0109] In this implementation, considering that different fingerprint sensors have significant differences in signal modalities of native domain fingerprint data, a specific preprocessing imaging process can be constructed before the data is sent to the feature extraction network, so as to use one or more of spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing and time domain amplitude extraction to perform preprocessing imaging of native domain fingerprint data. Afterwards, the preprocessed imaged ternary pair samples are input into the initial feature extraction network to obtain the corresponding three feature descriptors.

[0110] The method of generating three feature descriptors based on ternary pair samples in this implementation can input the ternary pairs into the initial feature extraction network after preprocessing, thereby achieving spatial and temporal signal enhancement while retaining the corresponding quantitative physical properties of the medium.

[0111] In some optional implementations of this embodiment, the initial feature fusion classification network generates the medium consistency probability of the sample fusion feature based on at least one of the following: a dimension-by-dimension mean regularization fusion mode, a quantization fusion mode, a spatial channel stacking fusion mode, and a dimension-by-dimensional differential fusion mode.

[0112] In this implementation, the dimension-by-dimension mean regularization fusion mode calculates the mean of the feature data involved in the fusion in each dimension. The calculated mean values ​​of each dimension are regularized: the size or proportion of the mean value is adjusted according to the needs to ensure that the fused features have a reasonable numerical range. Then, the regularized mean values ​​of each dimension are recombined into a new feature vector or matrix to obtain the fused features.

[0113] In the quantization fusion mode, you can select appropriate quantization operators, such as uniform quantization, non-uniform quantization, etc., according to the data characteristics and fusion requirements. After that, the input data is quantized, that is, the continuous data values ​​are mapped to discrete quantization levels. If multiple quantization operators are used, their outputs are fused according to certain rules (such as weighted average, maximum / minimum value selection, etc.). After that, the results of the quantization fusion can be subjected to necessary post-processing, such as denoising, smoothing, etc., to obtain fusion features.

[0114] The channel stacking fusion mode in the spatial domain can separate each channel of a multi-channel image (such as an RGB image) as an independent feature map. After that, the feature maps from different sources or after different processing are stacked according to the channel dimension to form a new multi-channel image. After that, necessary adjustments are made to the stacked multi-channel image, such as adjusting the channel order, applying normalization, etc., to obtain the fused features.

[0115] The differential fusion mode of each dimension, for multidimensional data, calculates the difference (first-order difference or high-order difference) between adjacent dimensions or adjacent time points to capture the changing trend of data in different dimensions. After that, the calculated differences of each dimension are fused according to certain rules, such as weighted average, summation, etc. After that, the differential fusion results can be subjected to nonlinear transformation or feature selection operations to obtain fusion features.

[0116] It is understandable that the above-mentioned fusion method can be adjusted according to the actual application scenario or the experience of technical personnel in this field, and the present disclosure does not limit this.

[0117] The method of generating the medium consistency probability of the sample fusion feature in the present implementation clarifies that the method of generating the sample fusion feature can be one or more of the dimension-by-dimension mean regularization fusion mode, the quantization fusion mode, the spatial channel stacking fusion mode and the dimension-by-dimensional differential fusion mode, thereby ensuring the quality of the fusion feature and improving the accuracy of the medium consistency probability predicted based on the fusion feature.

[0118] As one or more specific application embodiments of the present disclosure, the following is combined with specific application scenarios Figure 3a The present invention describes the method of offline network training and online chip deployment provided by the present invention.

[0119] In this example, taking ultrasonic fingerprint as an example, the following steps are included:

[0120] Step S301: Use an offline training data set with medium labels to construct and obtain ultrasonic echo signals of multiple phases. The ultrasonic echo signal can be recorded as a raw signal.

[0121] Step 1.1: Use a given fingerprint sensor (capacitive, optical, ultrasonic, etc.) to sample native domain signals for more than 2 (including 2) media. The pressed native domain data sets of different media components are represented as A, B, C, …

[0122] Step 1.2: Construct three-tuple pairs, each of which contains two samples i and j from the same medium dataset, and another sample k from a different medium dataset. In an example, Figure 3b As shown, Figure 3b An exemplary schematic diagram of a triplet pair sample is shown.

[0123] Step S302: Preprocess the native (Raw) domain signal into an image.

[0124] Considering that the native domain signal modalities of different fingerprint sensor inputs are significantly different, a specific preprocessing imaging process can be constructed before sending the data into the network. The native Raw can be preprocessed to obtain the basic Base fingerprint imaging. After that, the fingerprint imaging is preprocessed and enhanced to achieve spatial and temporal domain signal enhancement while retaining the corresponding quantitative physical properties of the medium, and retaining the three-dimensional characteristics to the greatest extent. In an example, Figure 3c , Figure 3d and Figure 3e As shown, Figure 3c It is a schematic diagram of native domain data; Figure 3d This is a schematic diagram of Base fingerprint imaging data; Figure 3e It is a schematic diagram of preprocessing enhanced fingerprint imaging.

[0125] Step S303: Generate feature (DL) domain vector representation based on the deep convolutional neural network.

[0126] Step 3.1: This example constructs a set of feature vector extraction and representation technology based on deep convolutional neural network, which converts the single-frame fingerprint echo signal tensor with a size of 2×72×72 output after the native (Raw) domain signal preprocessing in step 2 into a 10-dimensional implicit physical medium feature vector of 10×1×1. Specifically, Figure 3f As shown, Figure 3f The feature (DL) domain vector representation based on the Deep Convolutional Neutral Network is shown.

[0127] The deep convolutional neural network contains three subnetworks, namely the spatial domain enhancement subnetwork, the feature extraction subnetwork and the medium description subnetwork. The spatial domain enhancement subnetwork is used to simulate preprocessing enhancement without changing the spatial resolution and the number of channels; the feature extraction layer reduces the spatial resolution and increases the number of channels of the native domain signal through the stacking combination of convolutional layers and pooling layers, and prepares features for subsequent semantic description information extraction; finally, the medium description subnetwork integrates information and extracts semantics from the spatial domain concentrated feature tensor to generate a 10-dimensional feature descriptor to characterize the medium characteristics of the current single-frame compression data. This feature vector is essentially a 10-dimensional implicit physical medium characteristic description of this compression signal.

[0128] Specifically, the structure table of the feature extraction network using a deep convolutional neural network is shown in the following table.

[0129]

[0130]

[0131] Step 3.2: We analyze the three samples in the triple pair separately Perform feature mapping to obtain three feature vectors of length 10

[0132] Step S304: Combine the three feature vectors with a length of 10 obtained by mapping the ternary pairs in pairs.

[0133] Among them, two 10-dimensional feature vectors from the same medium data set are combined into a same medium positive sample The label is assigned as 1, and the two 10-dimensional feature vectors from different medium datasets are combined into a non-homologous negative sample. Assign label 0.

[0134] Step S305: Perform decision recognition training and decision recognition application.

[0135] Step 5.1: Build a fully connected network to construct the decision classification layer of the feature fusion classification network to achieve medium identity classification and identification based on the positive and negative fusion samples obtained in step 4.

[0136] Step 5.2: Based on the consistency between the classification results of the positive sample set and the negative sample set predicted by the feature fusion classification network and the true label, the binary cross entropy loss function is used for reverse gradient conduction:

[0137] Loss BCE = -y·log(x)-(1-y)·log(1-x)

[0138] At the same time, according to the order of gradient reverse conduction, the parameters of the fully connected decision classification layer and the deep convolutional neural network in step 3 are updated in turn to achieve performance optimization and training iterations, until the number of epochs of the loop iteration of the provided training data set reaches 200 times, or the loss function reaches the preset value, the iteration ends, the reverse gradient conduction is terminated, the weight coefficients in the network are no longer updated, and the fully connected decision classification layer and the deep convolutional neural network in step 3 are saved respectively.

[0139] Step 5.3: In actual deployment applications, the parameters at the end of iterative training can be used. First, the saved deep learning network weights are loaded, and the 10-dimensional physical medium characteristic descriptors are extracted using steps 2, 3, and 4 for all registered fingerprint template data and the current verification fingerprint data (that is, the current unlocking and pressing samples). and Where i is the ID of all recorded fingers, totaling N fingers. j is the number of times each recorded finger ID is recorded, totaling M registered presses.

[0140] Compared with the related art that needs to save the image data of the original resolution for i×j times of registered samples, the present disclosure only needs to retain the 10-dimensional descriptor for each registered fingerprint template data for subsequent fusion and final decision classification. Taking the 2×80×80 scale of ultrasonic under-screen fingerprint as an example, the storage consumption in the present disclosure is effectively reduced to 1 / 1280 of the related technical solution, which basically solves the storage bottleneck problem of storing such original templates for shallow fusion.

[0141] Step 5.4: Using the fingerprint matching result, index the registration number [i′, j′] that best matches the finger ID and pattern corresponding to the current verification fingerprint data (i.e., the current unlocking and pressing sample) k, and extract the corresponding medium descriptor from the pre-stored i×j registered fingerprint template data. Refer to step 4 and combine the extracted media descriptor with the unlocked media descriptor

[0142] Step 5.5: Load the fully connected classification layer training weights built in step 5.1, and perform fully connected feedforward calculation on the medium fusion vector in step 5.4. Perform decision identification on the regular feature to obtain the medium identity score. When the score is greater than 0.5, it is judged to be the same medium as the registered fingerprint template data, otherwise it is a different medium.

[0143] Specifically, you can also refer to Figure 3g , Figure 3g Shown above Figure 3a Schematic diagram of the process of offline network training and online chip deployment of the technical solution.

[0144] In the quantitative comparative analysis of fingerprint recognition using the method for identifying fingerprint medium consistency in the present invention, the true finger false interception rate (False Liveness Reject Rate, FLRR, also known as the true rejection rate) and the false fingerprint false acceptance rate (Spoof Accept Rate, SAR, also known as the false acceptance rate) can be used as performance evaluation indicators.

[0145]

[0146]

[0147] Among them, FLRR refers to the proportion of fingerprint images collected from the same finger that are judged as different fingers during 1:1 matching, expressed as a percentage. No.LivenessRejection refers to the number of live fingerprint data rejected; No.Liveness Pressing refers to the number of live fingerprint data pressed. SAR refers to the ratio of fingerprint images collected from different fingers that are mistakenly judged as the same finger during matching, expressed as a percentage. No.SpoofAccept refers to the number of fake fingerprint data accepted, and No.Spoof Attach refers to the number of fake fingerprint data involved in verification.

[0148] The following table shows the anti-counterfeiting task performance benefits brought by the experiment (Experiment) of the method for identifying fingerprint medium consistency in the present disclosure. In addition, compared with storing the registration template features in the original image domain, the use of the disclosed solution can convert the registration template information that needs to be pre-stored into a 10-dimensional feature descriptor of the registered fingerprint template data, and the pre-stored data is reduced to 1 / 1280 of the original, which completely solves the core difficulty of the registration template storage information being too large and unable to be mass-produced. Among them, the F1 score (F1Score), also known as the balanced F score, is an indicator used in statistics to measure the accuracy of a binary classification model. It is the harmonic mean of the precision and recall of the classification model.

[0149] Experiment FLRR SAR F1 Score No anti-counterfeiting function 2.43％ 61.51％ 0.5520 Static true and false fingerprint binary classification model 24.21％ 34.15％ 0.7047 Label-guided media identity detection 3.65％ 6.14％ 0.9508

[0150] In this embodiment, a device for identifying fingerprint medium consistency is also provided, which is used to implement the above embodiments and preferred implementations, and will not be repeated for what has been described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0151] This embodiment provides a device for identifying fingerprint medium consistency, such as Figure 4 As shown, the device comprises:

[0152] Data acquisition module 401, used to acquire current verification fingerprint data;

[0153] The data search module 402 is used to search the target fingerprint template data matching the current verification fingerprint data in the registered fingerprint template data set;

[0154] The feature search module 403 is used to search for a target feature descriptor corresponding to the target fingerprint template data in the template feature descriptor set, where the template feature descriptor set is generated based on the registered fingerprint template data set using a pre-trained feature extraction network;

[0155] The probability generation module 404 is used to use a pre-trained feature extraction network to generate a verification feature descriptor based on the current verification fingerprint data, and to use a pre-trained feature fusion classification network to generate a current medium consistency probability of a verification fusion feature based on the verification feature descriptor and the target feature descriptor. The fusion feature is obtained by fusing the verification feature descriptor and the target feature descriptor.

[0156] In some optional implementations of this embodiment, the pre-trained feature extraction network used by the feature search module 403 and the pre-trained feature fusion classification network used by the probability generation module are obtained by training the following modules:

[0157] The sample acquisition module 4031 is used to acquire a ternary pair sample from a ternary pair sample set, wherein the ternary pair sample includes an anchor point sample, a positive training sample, and a negative training sample of the fingerprint data, wherein the anchor point sample and the positive training sample belong to the same template medium, and the anchor point sample and the negative training sample belong to different template media;

[0158] A feature generation module 4032 is used to generate three feature descriptors based on the ternary pair samples using the initial feature extraction network, the three feature descriptors including: an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample, and a negative feature descriptor of the negative training sample;

[0159] The training network module 4033 is used to use the initial feature fusion classification network to generate the medium consistency probability of the sample fusion feature according to the three feature descriptors, generate the loss of the medium consistency probability and the real label of the sample fusion feature, and use the reverse gradient conduction of the loss to update the parameter value of the initial feature fusion classification network and the parameter value of the initial feature extraction network until the number of training reaches the preset number of iterations or the loss meets the preset target, thereby obtaining a pre-trained feature extraction network and a pre-trained feature fusion classification network; the sample fusion features include: positive sample fusion features obtained by fusion of anchor feature descriptors and positive feature descriptors, and negative sample fusion features obtained by fusion of anchor feature descriptors and negative feature descriptors.

[0160] In some optional implementations of the present embodiment, the initial feature extraction network trained by the feature generation module 4032 includes: a spatial enhancement subnetwork, a feature extraction subnetwork and a medium description subnetwork connected in sequence; the spatial enhancement subnetwork preprocesses the input fingerprint data to obtain a preprocessed feature tensor; the feature extraction subnetwork reduces the spatial resolution of the preprocessed feature tensor and increases the number of channels of the preprocessed feature tensor to obtain a spatial concentrated feature tensor; the medium description subnetwork performs information integration and semantic extraction on the spatial concentrated feature tensor to obtain a feature descriptor of a preset number of dimensions that characterizes the characteristics of the physical medium.

[0161] In some optional implementations of the present embodiment, the spatial domain enhancement subnetwork trained by the feature generation module 4032 includes: a first convolutional layer; the feature extraction subnetwork includes: a first convolutional block, a first transition layer, a second convolutional block and a second transition layer connected in sequence; wherein the first convolutional block uses a plurality of convolutional kernels of preset sizes to be stacked continuously; the first transition layer includes a second convolutional layer and a first average pooling layer connected in sequence; the second convolutional block uses a plurality of convolutional kernels of preset sizes to be stacked continuously; the second transition layer includes a third convolutional layer and a second average pooling layer connected in sequence; the medium description subnetwork includes: a depthwise separable convolutional layer, and the depthwise separable convolutional layer includes a fourth convolutional layer and a third average pooling layer connected in sequence.

[0162] In some optional implementations of the present embodiment, the pre-trained feature extraction network utilized by the feature search module 403 and the pre-trained feature fusion classification network utilized by the probability generation module are obtained through training of the following modules, and also include: a pre-processing module 4034, used to perform at least one of the following pre-processing imaging on the ternary pair samples: spatial rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing and time domain amplitude extraction; the feature generation module 4032 is further used to: utilize the initial feature extraction network to generate three feature descriptors based on the ternary pair samples after pre-processing imaging.

[0163] In some optional implementations of this embodiment, the initial feature fusion classification network utilized by the training network module 4033 generates the sample fusion feature based on at least one of the following when generating the medium consistency probability of the sample fusion feature: a dimension-by-dimension mean regularization fusion mode, a quantization fusion mode, a spatial channel stacking fusion mode, and a dimension-by-dimensional differential fusion mode.

[0164] In some optional implementations of this embodiment, the medium of the target fingerprint template data searched by the data search module 402 includes at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder and dust.

[0165] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments, and will not be repeated here. The device for identifying fingerprint medium consistency in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0166] In this embodiment, a device is also provided, comprising:

[0167] Fingerprint sensor system, used to obtain current verification fingerprint data;

[0168] Memory systems; and

[0169] The control system is configured to electrically communicate with the ultrasonic fingerprint sensor system. The control system is also configured to: execute the method for identifying fingerprint medium consistency of the above embodiment or any corresponding implementation manner thereof.

[0170] In this embodiment, a device is also provided, including:

[0171] Fingerprint sensor system, used to obtain current verification fingerprint data;

[0172] Memory systems; and

[0173] The control device is used to make a computer execute the method for identifying fingerprint medium consistency of the above embodiment or any corresponding implementation manner thereof.

[0174] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present disclosure, and the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0175] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0176] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0177] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0178] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0179] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means.

[0180] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0181] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0182] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0183] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for identifying the consistency of fingerprint media, It is characterized in that The method comprises: Get the current verification fingerprint data; In the registered fingerprint template data set, searching for target fingerprint template data matching the current verification fingerprint data; Searching for a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set, wherein the template feature descriptor set is generated according to the registered fingerprint template data set using a pre-trained feature extraction network; Using the pre-trained feature extraction network, a verification feature descriptor is generated according to the current verification fingerprint data, and using the pre-trained feature fusion classification network, a current medium consistency probability of the verification fusion feature is generated according to the verification feature descriptor and the target feature descriptor. The verification fusion feature is obtained by fusing the verification feature descriptor and the target feature descriptor.

2. The method according to claim 1, It is characterized in that The pre-trained feature extraction network and the pre-trained feature fusion classification network are trained by the following steps: Acquire a ternary pair sample from a ternary pair sample set, wherein the ternary pair sample includes an anchor point sample of fingerprint data, a positive training sample, and a negative training sample, wherein the anchor point sample and the positive training sample belong to the same template medium, and the anchor point sample and the negative training sample belong to different template media; Using an initial feature extraction network, based on the ternary pair samples, three feature descriptors are generated, wherein the three feature descriptors include: an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample, and a negative feature descriptor of the negative training sample; Using the initial feature fusion classification network, according to the three feature descriptors, the medium consistency probability of the sample fusion feature is generated, and the loss of the medium consistency probability and the real label of the sample fusion feature is generated. The parameter value of the initial feature fusion classification network and the parameter value of the initial feature extraction network are updated by using the reverse gradient conduction of the loss, until the number of training reaches a preset number of iterations or the loss meets the preset target, and a pre-trained feature extraction network and a pre-trained feature fusion classification network are obtained; the sample fusion feature includes: a positive sample fusion feature obtained by fusion of the anchor feature descriptor and the positive feature descriptor, and a negative sample fusion feature obtained by fusion of the anchor feature descriptor and the negative feature descriptor.

3. The method according to claim 2, It is characterized in that The initial feature extraction network includes: a spatial domain enhancement sub-network, a feature extraction sub-network and a medium description sub-network connected in sequence; The initial feature extraction network is used to generate three feature descriptors based on the ternary pair samples, including: Using the spatial domain enhancement sub-network, preprocessing and enhancing the ternary pair samples to obtain a preprocessed feature tensor; Using the feature extraction subnetwork, reducing the spatial resolution of the preprocessed feature tensor, increasing the number of channels of the preprocessed feature tensor, and obtaining a spatial domain concentrated feature tensor; The medium descriptor network is used to perform information integration and semantic extraction on the spatial domain concentrated feature tensor to obtain a feature descriptor of a preset number of dimensions that characterizes the characteristics of the physical medium.

4. The method according to claim 3, It is characterized in that The spatial domain enhancement sub-network includes: a first convolutional layer; The feature extraction subnetwork includes: a first convolution block, a first transition layer, a second convolution block and a second transition layer connected in sequence; wherein the first convolution block uses a plurality of convolution kernels of preset sizes to be stacked continuously; the first transition layer includes a second convolution layer and a first average pooling layer connected in sequence; the second convolution block uses a plurality of convolution kernels of preset sizes to be stacked continuously; the second transition layer includes a third convolution layer and a second average pooling layer connected in sequence; The medium description subnetwork includes: a depth-separable convolutional layer, and the depth-separable convolutional layer includes a fourth convolutional layer and a third average pooling layer connected in sequence.

5. The method according to claim 2, It is characterized in that The pre-trained feature extraction network and the pre-trained feature fusion classification network are obtained by training through the following steps, and further include: performing at least one of the following pre-processing imaging on the ternary pair samples: spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing and time domain amplitude extraction; The method of using an initial feature extraction network to generate three feature descriptors based on the ternary pair samples includes: using an initial feature extraction network to generate three feature descriptors based on the ternary pair samples after preprocessing imaging.

6. The method according to claim 2, It is characterized in that When generating the medium consistency probability of the sample fusion feature, the initial feature fusion classification network generates the sample fusion feature based on at least one of the following: a dimension-by-dimension mean regularization fusion mode, a quantization fusion mode, a spatial channel stacking fusion mode, and a dimension-by-dimensional differential fusion mode.

7. The method according to claim 1, It is characterized in that The medium of the target fingerprint template data includes at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder and dust.

8. A device for identifying the consistency of fingerprint media, It is characterized in that The device comprises: A data acquisition module is used to acquire the current verification fingerprint data; A data search module, used to search for target fingerprint template data matching the current verification fingerprint data in the registered fingerprint template data set; A feature search module, used to search for a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set, wherein the template feature descriptor set is generated according to the registered fingerprint template data set using a pre-trained feature extraction network; The probability generation module is used to generate a verification feature descriptor based on the current verification fingerprint data using the pre-trained feature extraction network, and to generate a current medium consistency probability of a verification fusion feature based on the verification feature descriptor and the target feature descriptor using the pre-trained feature fusion classification network, wherein the verification fusion feature is obtained by fusing the verification feature descriptor and the target feature descriptor.

9. A device, include: Fingerprint sensor system, used to obtain current verification fingerprint data; Memory system; and A control system is configured to electrically communicate with an ultrasonic fingerprint sensor system, and the control system is further configured to: execute the method for identifying fingerprint medium consistency according to any one of claims 1 to 7.

10. A device, include: Fingerprint sensor system, used to obtain current verification fingerprint data; Memory system; and A control device, used to execute the method for identifying fingerprint medium consistency according to any one of claims 1 to 7.

Images