Method and apparatus for identifying medium consistency of fingerprint

Abstract

The present disclosure relates to the technical field of signal processing, and disclosed is a method for identifying medium consistency of a fingerprint, comprising: acquiring current verification fingerprint data; searching a registered fingerprint template data set for target fingerprint template data matching the current verification fingerprint data; searching a template feature descriptor set for a target feature descriptor corresponding to the target fingerprint template data; and using a pre-trained feature extraction network to generate a verification feature descriptor on the basis of the current verification fingerprint data, and using a pre-trained feature fusion and classification network to generate a current medium consistency probability of a fused verification feature on the basis of the verification feature descriptor and the target feature descriptor. The present disclosure improves the accuracy of fingerprint classification and identification.

Description

Method and device for identifying consistency of fingerprint medium

[0001] The present application claims priority to the Chinese Patent Application No. 202411127429.7, filed on August 15, 2024, and entitled "Method and device for identifying consistency of fingerprint medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of signal processing, and in particular to a method and device for identifying consistency of fingerprint medium. BACKGROUND

[0003] Fingerprint recognition is a widely popular biometric technology. This technology plays an important role in identity verification in many fields such as unlocking of smart phones, financial transactions, and access control. In the past decade, with the upgrading of sensor technology, fingerprint recognition has undergone a system change from capacitive fingerprint to optical fingerprint under screen to ultrasonic fingerprint under screen. The above-mentioned fingerprint sensors obtain sound, light and electrical signals to two-dimensionally image the fingerprint lines pressed on the sensor, and perform two-dimensional matching with the registered fingerprint template data, thereby realizing identity recognition.

[0004] In recent years, anti-counterfeiting solutions for ultrasonic fingerprint have attracted more and more attention. These methods mainly perform explicit calculation and manual fitting on real-time physical characteristics such as acoustic impedance and surface distance based on ultrasonic echo signals, and realize binary classification recognition of living real fingers and non-living fake fingerprints by comparing these manual features with the distance of offline collected static real finger physical characteristic fitting quantities.

[0005] However, the above-mentioned fingerprint recognition method uses static offline collected fingers as the analysis object, and the set classification threshold interval and statistical distribution law are both heavily dependent on the statistical characteristics of the collected real and fake fingers. When identifying real fingerprints of extreme groups of fingerprints (such as peeling, wide lines, shallow lines, etc.) or fake fingerprints made of new medium materials, fingerprint classification recognition errors are likely to occur, resulting in situations where real fingerprints with the same template cannot be unlocked or fake fingerprints made of new medium materials can successfully impersonate. SUMMARY

[0006] Therefore, the present disclosure provides a method for identifying consistency of fingerprint medium to solve the problem of reducing current fingerprint classification recognition errors.

[0007] In a first aspect, the present disclosure provides a method for identifying consistency of fingerprint medium, the method comprising:

[0008] obtaining current verification fingerprint data;

[0009] In the registered fingerprint template data set, find the target fingerprint template data matched with the current verification fingerprint data;

[0010] In the template feature descriptor set, find the target feature descriptor corresponding to the target fingerprint template data, and the template feature descriptor set is generated according to the registered fingerprint template data set by using a pre-trained feature extraction network;

[0011] By using a pre-trained feature extraction network, generate a verification feature descriptor according to the current verification fingerprint data, and by using a pre-trained feature fusion classification network, generate a current medium consistency probability of a verification fusion feature according to the verification feature descriptor and the target feature descriptor, and the fusion feature is fused from the verification feature descriptor and the target feature descriptor.

[0012] In a second aspect, the present disclosure provides a device for identifying fingerprint medium consistency, the device comprising:

[0013] A data acquisition module is configured to acquire current verification fingerprint data;

[0014] A data search module is configured to search for target fingerprint template data matched with the current verification fingerprint data in the registered fingerprint template data set;

[0015] A feature search module is configured to search for a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set, and the template feature descriptor set is generated according to the registered fingerprint template data set by using a pre-trained feature extraction network;

[0016] A probability generation module is configured to generate a verification feature descriptor according to the current verification fingerprint data by using a pre-trained feature extraction network, and generate a current medium consistency probability of a verification fusion feature according to the verification feature descriptor and the target feature descriptor by using a pre-trained feature fusion classification network, and the fusion feature is fused from the verification feature descriptor and the target feature descriptor.

[0017] In a third aspect, the present disclosure provides a device, comprising:

[0018] A fingerprint sensor system is configured to acquire current verification fingerprint data;

[0019] A memory system; and

[0020] A control system is configured to be in electrical communication with the ultrasonic fingerprint sensor system, and the control system is further configured to perform the method for identifying fingerprint medium consistency of the first aspect or any of the corresponding embodiments thereof.

[0021] In a fourth aspect, the present disclosure provides a device, comprising:

[0022] The fingerprint sensor system is used to acquire current verification fingerprint data.

[0023] The memory system; and

[0024] The control device is used to make the computer execute the method for identifying the consistency of the fingerprint medium according to the first aspect or any implementation thereof.

[0025] The method for identifying the consistency of the fingerprint medium provided by the above-mentioned embodiments of the present disclosure generates a verification feature descriptor of the current verification fingerprint data by using a pre-trained feature extraction network, and determines a target feature descriptor of the target fingerprint template data matched with the current verification fingerprint data, and then generates a current medium consistency probability corresponding to a verification fusion feature according to the verification feature descriptor and the target feature descriptor by using a pre-trained feature fusion classification network, so that when the current medium consistency probability is greater than a probability threshold, it is determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data, without considering the statistical distribution of the collected real and fake finger fingerprints, thereby avoiding the problems of poor finger stability, poor system stability and poor scalability of the fingerprint classification determined according to the statistical distribution of the fingerprints, and improving the accuracy of the fingerprint classification identification. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the specific embodiments of the present disclosure, the drawings needed in the specific embodiment description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0027] FIG. 1 is a flowchart of a method for identifying the consistency of the fingerprint medium according to an embodiment of the present disclosure;

[0028] FIG. 2 is a flowchart of a method for training a feature extraction network and a feature fusion classification network according to an embodiment of the present disclosure;

[0029] FIG. 3a is a flowchart of network training and fingerprint identification of a method for identifying the consistency of the fingerprint medium according to an embodiment of the present disclosure;

[0030] FIG. 3b is an exemplary schematic diagram of a ternary pair sample according to an embodiment of the present disclosure;

[0031] FIG. 3c is a native domain data schematic diagram of fingerprint data according to an embodiment of the present disclosure;

[0032] FIG. 3d is a fingerprint imaging data schematic diagram of fingerprint data according to an embodiment of the present disclosure;

[0033] FIG. 3e is a pre-processing enhanced fingerprint imaging schematic diagram of fingerprint data according to an embodiment of the present disclosure;

[0034] FIG. 3f is a schematic diagram of feature domain vector representation based on deep convolutional neural network according to an embodiment of the present disclosure;

[0035] FIG. 3g is a schematic diagram of the flow of offline network training and online chip deployment according to an embodiment of the present disclosure;

[0036] FIG. 4 is a structural block diagram of an apparatus for identifying fingerprint medium consistency according to an embodiment of the present disclosure;

[0037] FIG. 5 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person skilled in the art without creative work fall within the protection scope of the present disclosure.

[0039] Fingerprint recognition is a widely popular biometric technology. This technology plays an important role in identity verification in many fields such as smart phone unlocking, financial transactions, and access control.

[0040] In the past decade, with the upgrading of sensor technology, fingerprint recognition has experienced system replacement from capacitive fingerprint to under-screen optical fingerprint to under-screen ultrasonic fingerprint. The above-mentioned fingerprint sensor obtains sound, light and electrical signals to perform two-dimensional imaging on the fingerprint lines pressed on the sensor, and performs two-dimensional matching with the registered fingerprint template data, thereby realizing identity recognition. This stage is called the "flat matching" stage.

[0041] In recent years, anti-counterfeiting solutions for ultrasonic fingerprint have attracted more and more attention. These methods are mainly based on explicit calculation and manual fitting of acoustic impedance, surface distance and other real-time physical characteristics from ultrasonic echo signals, and realize the binary classification recognition of living real fingers and non-living fake fingerprints by comparing these manual features with the distance of offline collected static real finger physical characteristic fitting quantities.

[0042] The main technical defects of the above-mentioned related technologies include:

[0043] a. Poor finger stability: The above-mentioned solutions facing the "manual features" use static offline collected fingers as analysis objects, and the set classification threshold interval and statistical distribution law are severely dependent on the statistical characteristics of the collected real and fake fingers. This leads to the serious problem of classification error or even inability to unlock for some extreme population fingers (peeling, wide ridges, shallow ridges, etc.).

[0044] b. Poor system stability: In actual application, considering the difference between the system environment of the mass production machine and the system environment of the collected machine, it is difficult to directly apply the statistical distribution obtained by collecting samples of fixed machines to all machines, which also makes the system robustness of the mentioned method low and the efficiency of mass production adaptation low.

[0045] c. Poor scalability: The above-mentioned related technologies focus on the binary classification problem of distinguishing "real" and "fake". Once the "real" finger of the special population and the "fake" fingerprint made of new medium material appear, the classification model based on historical data will fail, and the parameters need to be updated to adapt to the new medium binary classification problem.

[0046] In order to effectively measure the medium identity of the registered fingerprint template data and the current verification fingerprint data (i.e. the current unlocking pressing sample) in the high-dimensional space on the basis of the ridge matching, and solve the problems of poor generalization, high deployment cost and the like of the detection technology, the present application proposes a method for identifying the consistency of fingerprint medium based on deep learning technology, which is not limited to classifying the fixed concept of "live" and "non-live", but focuses on measuring the medium identity of the registered fingerprint template data and the current verification fingerprint data (i.e. the current unlocking pressing sample). The present application is divided into two main stages of offline training and online chip deployment.

[0047] According to the embodiments of the present disclosure, a method for identifying the consistency of fingerprint medium is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0048] In the present embodiment, a method for identifying the consistency of fingerprint medium is provided, which can use a client or a server as an execution subject. The client can be a mobile terminal such as a mobile phone, a tablet computer, etc., or a fixed terminal device. The server can obtain the current verification fingerprint data sent by the client and return the result obtained by executing the method for identifying the consistency of fingerprint medium to the client. The result can be: the current medium consistency probability, the medium consistency identification result determined based on the current medium consistency probability, or the fingerprint identification result determined based on the current medium consistency probability.

[0049] FIG. 1 is a flowchart of a method for identifying fingerprint medium consistency according to an embodiment of the present disclosure, as shown in FIG. 1, the flow includes the following steps:

[0050] In step S101, current verification fingerprint data is acquired.

[0051] In the present embodiment, the above-mentioned execution subject can acquire Raw domain fingerprint data of the initial, raw state of the object to be detected without any processing, for example, acquire Raw domain image data or signal data, through a fingerprint sensor in current or future technology.

[0052] For example, the above-mentioned execution subject can acquire current verification fingerprint data through a capacitive fingerprint sensor: when a finger is pressed on the sensor, the ups and downs of the fingerprint will form different capacitance values, which are converted into digital fingerprint signals.

[0053] The above-mentioned execution subject can acquire current verification fingerprint data using an optical fingerprint sensor: the optical fingerprint sensor captures the reflected light of the fingerprint pressed on the screen to form a fingerprint image.

[0054] The above-mentioned execution subject can acquire current verification fingerprint data using an ultrasonic fingerprint sensor: the ultrasonic fingerprint sensor draws a three-dimensional image of the fingerprint by sending ultrasonic pulses and receiving their reflected signals.

[0055] After that, the above-mentioned execution subject can take the Raw domain fingerprint data or the base (Base) fingerprint image obtained by preprocessing based on the Raw domain fingerprint data as the current verification fingerprint data.

[0056] In step S102, among the registered fingerprint template data set, find the target fingerprint template data matching the current verification fingerprint data.

[0057] In the present embodiment, after acquiring the current verification fingerprint data, it can be compared with the templates in the registered fingerprint template data set to find the target fingerprint template data matching the current verification fingerprint data.

[0058] The registered fingerprint template data set can include a plurality of fingerprint feature templates, which are obtained by pre-converting the user's fingerprint image into a template and storing it.

[0059] The process of comparing the current verification fingerprint data with the templates in the registered fingerprint template data set usually involves extracting the features (such as ridge end points, bifurcation points, minutiae points, etc.) of the current verification fingerprint data and comparing them with the features in the registered fingerprint template data.

[0060] The matching algorithm can adopt matching strategies in various existing technologies or future developments, and the present disclosure does not limit this. For example, the matching algorithm can adopt minutia-based matching, pattern-based matching, etc., so as to find the target fingerprint template data matched with the current verification fingerprint data.

[0061] It can be understood that the current verification fingerprint data can also be pre-processed before comparing the current verification fingerprint data with the registered fingerprint template data. The pre-processing step can include denoising, contrast enhancement, binarization, direction estimation, segmentation, etc., so as to better extract the features of the fingerprint.

[0062] In some specific examples, the pre-processing of the current verification fingerprint data can include at least one of spatial rotation, affine transformation, displacement, flipping, denoising, enhancement, value range quantization, time domain smoothing, and time domain amplitude extraction, etc.

[0063] Step S103, in the template feature descriptor set, find the target feature descriptor corresponding to the target fingerprint template data.

[0064] In the present embodiment, the template feature descriptor set is generated by using a pre-trained feature extraction network according to the registered fingerprint template data set. Therefore, there is a one-to-one mapping relationship between the registered fingerprint templates in the registered fingerprint template data set and the template feature descriptors in the template feature descriptor set. According to the mapping relationship, the above execution subject can map the position of the target fingerprint template data in the registered fingerprint template data set to the corresponding position in the registered fingerprint template data set, and take the feature descriptor of the corresponding position as the target feature descriptor.

[0065] Step S104, using a pre-trained feature extraction network, generating a verification feature descriptor according to the current verification fingerprint data, and using a pre-trained feature fusion classification network, generating a current medium consistency probability of a verification fusion feature according to the verification feature descriptor and the target feature descriptor.

[0066] In the present embodiment, the pre-trained feature extraction network can be implemented by using a deep learning model. From the original fingerprint image or fingerprint signal, a feature vector with high representativeness is extracted, that is, a feature descriptor is extracted.

[0067] These feature descriptors capture key information in the fingerprint image or fingerprint signal, such as ridges, valleys, frequencies, minutiae (such as end points and bifurcation points) of the fingerprint, etc., but remove unnecessary details and noise, so as to convert the high-dimensional and complex fingerprint image data into low-dimensional and easy-to-compare and process feature vectors.

[0068] After generating the verification feature descriptor from the current verification fingerprint data by using the pre-trained feature extraction network, the execution subject can generate a verification fusion feature by fusing the verification feature descriptor and the target feature descriptor by using the pre-trained feature fusion classification network, and then determine the current medium consistency probability of the fusion feature.

[0069] The method of fusing the verification fusion feature can include simple splicing, weighted summation or more complex learning-based fusion methods. In some specific examples, the verification fusion feature can be generated by using at least one of the following: a mean regular fusion mode in each dimension, a quantization fusion mode, a channel stacking fusion mode in the spatial domain, and a dimension difference fusion mode.

[0070] The current medium consistency probability can be a medium identity score. The probability threshold is a score value for determining whether the verification feature descriptor and the target feature descriptor belong to the same medium.

[0071] After determining the current medium consistency probability of the fusion feature, when the current medium consistency probability is greater than the probability threshold, it can be determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data. In one specific example, when the medium identity score is greater than 0.5, it can be determined that the current verification fingerprint data and the target fingerprint template data are the same medium, otherwise they are different media.

[0072] The medium here can include a fingerprint medium in existing technologies or future technologies, which is not limited in the present application. In some specific examples, the medium of the target fingerprint template data can include at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder and dust.

[0073] It can be understood that when it is determined that the current verification fingerprint data and the target fingerprint template data are the same medium or different media, a verification recognition result of the current verification fingerprint can also be output according to the determination result.

[0074] The method for identifying the consistency of the fingerprint medium provided in the embodiment generates a verification feature descriptor of the current verification fingerprint data by using a pre-trained feature extraction network, and determines a target feature descriptor of the target fingerprint template data matched with the current verification fingerprint data, and then generates a current medium consistency probability corresponding to the verification fusion feature according to the verification feature descriptor and the target feature descriptor by using a pre-trained feature fusion classification network, so that when the current medium consistency probability is greater than a probability threshold, it is determined that the medium of the current verification fingerprint data is the same as the medium of the target fingerprint template data, without considering the statistical distribution of the fingerprint, thereby avoiding the problems of poor finger stability, poor system stability and poor scalability in determining the classification of the fingerprint according to the statistical distribution of the fingerprint, and improving the accuracy of the classification and identification of the fingerprint.

[0075] In the embodiment, a method for training a feature extraction network and a feature fusion classification network is provided, which can be used in the mobile terminal such as a mobile phone, a tablet computer and the like. FIG. 2 is a flowchart of the method for training the feature extraction network and the feature fusion classification network according to the embodiment of the present disclosure. As shown in FIG. 2, the flow includes the following steps:

[0076] In step S201, a triad sample in a triad sample set is obtained.

[0077] In the embodiment, the triad sample includes an anchor sample, a positive training sample and a negative training sample of the fingerprint data. The anchor sample and the positive training sample belong to the same template medium, and the anchor sample and the negative training sample belong to different template media.

[0078] In obtaining the triad sample, more than two (including two) media can be sampled in a native domain signal using a predetermined fingerprint sensor (capacitive, optical, ultrasonic). The pressing native domain data set of different media components can be represented as A, B, C, ….

[0079] Then, the triad sample can be constructed. In a specific example, each triad sample includes two samples i and j from the same medium data set and another sample k from a different medium data set. The samples i, j and k can correspond to the positive training sample, the anchor sample and the negative training sample, respectively.

[0080] In step S202, three feature descriptors are generated based on the triad sample using an initial feature extraction network.

[0081] In the embodiment, the three feature descriptors can include an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample and a negative feature descriptor of the negative training sample.

[0082] The initial feature extraction network is an initial network of the feature extraction network pre-trained in step S104, which has the same structure and function as the initial network, and only the parameter value of the network is the initial parameter value without updating iteration. Details are not repeated here.

[0083] In a specific example, the initial feature extraction network can be implemented by using a deep convolutional neural network. For the three samples in the triad feature mapping is performed to obtain three feature vectors with a length of 10

[0084] Step S203, using the initial feature fusion classification network, generating the medium consistency probability of the sample fusion feature according to the three feature descriptors, generating the loss of the medium consistency probability and the true label of the sample fusion feature, updating the parameter values of the initial feature fusion classification network and the initial feature extraction network by using the reverse gradient conduction of the loss until the training times reach the preset iteration times or the loss meets the preset target, obtaining the pre-trained feature extraction network and the pre-trained feature fusion classification network.

[0085] In the embodiment, the sample fusion feature can include: a positive sample fusion feature fused by the anchor feature descriptor and the positive feature descriptor, and a negative sample fusion feature fused by the anchor feature descriptor and the negative feature descriptor.

[0086] Specifically, in one specific example, three 10-length feature vectors mapped in a triple pair can be combined two by two, wherein two 10-dimensional feature vectors from the same medium dataset are combined into a same-medium positive sample fusion feature assigned a label 1, and two 10-dimensional feature vectors from different medium datasets are combined into a non-same-origin negative sample fusion feature assigned a label 0.

[0087] The initial feature fusion classification network can be obtained by cascading the network for feature fusion and classification in the prior art, which is not limited in the present disclosure. In one specific example, a fully connected network can be used to construct a decision classification layer to predict the medium consistency probability of the positive and negative sample fusion features based on the positive and negative sample fusion features, and to complete the medium identity classification and identification of the positive and negative sample fusion features.

[0088] The loss of the medium consistency probability and the true label of the positive and negative sample fusion features can be achieved by using the loss function for determining the loss between the predicted value and the true value of the classification problem in the prior art or future development, which is not limited in the present disclosure.

[0089] In one specific example, based on the consistency of the classification results of the positive and negative sample fusion features and the true label, a binary cross-entropy loss function can be used for reverse gradient conduction: Loss BCE = -y·log(x)-(1-y)·log(1-x)

[0090] wherein Loss BCE is the loss of the medium consistency probability predicted based on the positive and negative sample fusion features and the true label, y is the true label (0 or 1), and x is the probability of predicting the positive sample fusion feature. When y = 1 (i.e. the sample belongs to the positive class), the loss function is simplified as Loss BCE= -log(x), the loss value is close to 0, and the loss will be very large. When y = 0 (i.e. the sample belongs to the negative class), the loss function simplifies to Loss BCE = -log(1-x), the loss value is close to 1, and the loss will be very large.

[0091] At this time, according to the order of gradient back propagation, the parameters of the fully connected decision classification layer and the deep convolutional neural network are updated in turn to realize performance optimization and training iteration, until the number of loop iterations of the provided training data set reaches 200 times or the loss function reaches the preset value, the iteration ends, the reverse gradient propagation terminates, and the weight coefficients in the network are no longer updated. The fully connected decision classification layer and the deep convolutional neural network are saved respectively.

[0092] The method for training the feature extraction network and the feature fusion classification network in the embodiment uses the label guidance of the sample fusion features of the registered fingerprint template data, predicts the medium consistency probability of the positive and negative sample fusion features according to the ternary pair sample, generates the loss with the real label, and then updates the parameter values of the initial feature fusion classification network and the parameter values of the initial feature extraction network by using the reverse gradient conduction of the loss until the number of training times reaches the preset iteration number or the loss meets the preset target. The pre-trained feature extraction network and the pre-trained feature fusion classification network obtained through this training process effectively realize the feature aggregation of the same medium and the extraction of the implicit medium feature, realize the prediction of the medium consistency probability, overcome the differentiated interference of different system hardware environment parameters and different finger types, and have more robust non-same medium attack detection capability.

[0093] In some optional implementations of the embodiment, the initial feature extraction network includes a spatial domain enhancement sub-network, a feature extraction sub-network and a medium description sub-network connected in sequence. Based on the ternary pair sample, the initial feature extraction network generates three feature descriptors, including: using the spatial domain enhancement sub-network to pre-process and enhance the ternary pair sample to obtain a pre-processed feature tensor; using the feature extraction sub-network to reduce the spatial resolution of the pre-processed feature tensor and increase the channel number of the pre-processed feature tensor to obtain a spatial domain condensed feature tensor; and using the medium description sub-network to integrate information and extract semantics of the spatial domain condensed feature tensor to obtain a feature descriptor of a preset number of dimensions representing physical medium characteristics.

[0094] In the implementation, the initial feature extraction network can extract deep-level feature descriptors with physical medium characteristics from the input fingerprint data through the cascade of the spatial domain enhancement sub-network, the feature extraction sub-network and the medium description sub-network, and provide powerful feature representation for subsequent classification, recognition or analysis tasks.

[0095] The spatial domain enhancement sub-network can preprocess and enhance the input fingerprint data, improve image quality through spatial domain operations such as spatial domain filtering, contrast enhancement, sharpening, and the like, remove noise, enhance the visibility and contrast of fingerprint lines, and obtain a preprocessed feature tensor. This processing process can ensure that the subsequent processing stage can extract features from clearer and higher quality images.

[0096] The feature extraction sub-network can further process the preprocessed feature tensor, reduce the spatial resolution while increasing the number of channels through convolution layers, pooling layers and the like, realize feature condensation and abstraction, obtain a spatial domain condensed feature tensor, thereby reducing the computational burden while maintaining key fingerprint information, increasing the description ability of the features, and preparing for subsequent advanced feature extraction.

[0097] The medium description sub-network can perform in-depth information integration and semantic extraction on the spatial domain condensed feature tensor, can obtain the internal relationship and semantic meaning between features through advanced feature extraction methods such as fully connected layers, recurrent neural networks or attention mechanisms, and thereby obtain a feature description sub-network of a preset number of dimensions representing physical medium characteristics, realize the extraction of deep-level features reflecting physical medium characteristics (such as the microstructure of the skin of the finger, sweat distribution, contact pressure, etc.) from the fingerprint image, and more effectively perform fingerprint recognition, classification or analysis.

[0098] The initial feature extraction network in the embodiment realizes the high-level representation from the original fingerprint data to the physical medium characteristics through the step-by-step deepening feature extraction process, completes the mapping from the low-level visual features or sensory features to the high-level semantic features, and improves the effectiveness of the generated feature description sub-network of the input fingerprint data.

[0099] In some optional implementations of the embodiment, the spatial domain enhancement sub-network includes a first convolution layer; the feature extraction sub-network includes a first convolution block, a first transition layer, a second convolution block and a second transition layer connected in sequence; wherein the first convolution block uses a plurality of convolution kernels of a predetermined size stacked in sequence; the first transition layer includes a second convolution layer and a first average pooling layer connected in sequence; the second convolution block uses a plurality of convolution kernels of a predetermined size stacked in sequence; the second transition layer includes a third convolution layer and a second average pooling layer connected in sequence; and the medium description sub-network includes a depth separable convolution layer, which includes a fourth convolution layer and a third average pooling layer connected in sequence.

[0100] In the present implementation, the spatial enhancement sub-network starts with a first convolutional layer to perform preliminary feature extraction and spatial image enhancement on the input fingerprint data. The first convolutional layer applies a convolution kernel (also known as a filter) through a sliding window to extract image features, which can capture local features in the image such as edges, corners, etc., to provide a basis for subsequent processing.

[0101] The feature extraction sub-network is sequentially connected by a first convolutional block, a first transition layer, a second convolutional block, and a second transition layer.

[0102] The first convolutional block uses multiple convolution kernels of a preset size stacked in series, i.e., multiple convolutional layers in a deep convolutional neural network using convolution kernels of a preset size connected in series. The first layer usually learns low-level features such as edges and textures, while the deep convolutional kernel can combine the features of the previous layer to learn more complex and abstract features such as local structures, shapes, or patterns of objects, thereby deeply extracting image features.

[0103] The first transition layer includes a second convolutional layer and a first average pooling layer. The second convolutional layer can be used for further feature extraction or feature conversion, translation, and local information reorganization. The first average pooling layer is used to reduce the spatial dimension of the feature map (i.e., down-sampling), while preserving important information, reducing computational complexity, and integrating information.

[0104] The second convolutional block, similar to the first convolutional block, can use the same or different configuration of convolution kernels as the first convolutional block to extract deeper features.

[0105] The second transition layer includes a third convolutional layer and a second average pooling layer, which have similar functions to the first transition layer and can be implemented with the same or different configuration as the first transition layer to integrate and optimize information for deeper feature maps.

[0106] The medium description sub-network ends the feature extraction network with a depthwise separable convolutional layer. The depthwise separable convolutional layer includes a fourth convolutional layer and a third average pooling layer. The fourth convolutional layer integrates high-dimensional information after the integrated information, and the third average pooling layer integrates the integrated information after the fusion, thereby realizing the description of the medium.

[0107] The initial feature extraction network in the present implementation further specifies the network structure of the spatial enhancement sub-network, the feature extraction sub-network, and the medium description sub-network, thereby realizing the extraction of high-level features of the input fingerprint data, and further improving the effectiveness of the feature descriptor of the input fingerprint data generated by the feature extraction network.

[0108] In some optional implementations of the embodiment, the pre-trained feature extraction network and the pre-trained feature fusion classification network are trained by the following steps, and further include: performing at least one of the following pre-processing imaging on the triple pair sample: spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing, and time domain amplitude extraction; generating three feature descriptors based on the triple pair sample using the initial feature extraction network, including: generating three feature descriptors based on the pre-processed triple pair sample using the initial feature extraction network.

[0109] In the implementation, considering that different fingerprint sensor input native domain fingerprint data has significant differences in signal modalities, before feeding the data into the feature extraction network, a specific pre-processing imaging process can be constructed, so that one or more of spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing, and time domain amplitude extraction are used for pre-processing imaging of the native domain fingerprint data. Then, the pre-processed triple pair sample is input into the initial feature extraction network to obtain the corresponding three feature descriptors.

[0110] The method of generating three feature descriptors based on the triple pair sample in the implementation can input the pre-processed triple pair into the initial feature extraction network, thereby realizing spatial domain and time domain signal enhancement while preserving the corresponding quantized physical characteristics of the medium.

[0111] In some optional implementations of the embodiment, when the initial feature fusion classification network generates the medium consistency probability of the sample fusion feature, the sample fusion feature is generated based on at least one of the following: a mean regular fusion mode of each dimension, a quantization fusion mode, a channel stacking fusion mode in the spatial domain, and a difference fusion mode of each dimension.

[0112] In the implementation, the mean regular fusion mode of each dimension is to calculate the mean value of the feature data participating in fusion in each dimension. The calculated mean values of each dimension are regularized: the size or proportion of the mean value is adjusted according to the requirements to ensure that the fused feature has a reasonable numerical range. Then, the regularized mean values of each dimension are recombined into a new feature vector or matrix to obtain the fusion feature.

[0113] The quantization fusion mode can select appropriate quantization operators according to data characteristics and fusion requirements, such as uniform quantization, non-uniform quantization, etc. Then, the input data is quantized, i.e., the continuous data value is mapped to a discrete quantization level, and if multiple quantization operators are used, their outputs are fused according to certain rules (such as weighted average, maximum / minimum value selection, etc.). Then, the quantized fusion result can be post-processed as needed, such as denoising, smoothing, etc., to obtain the fusion feature.

[0114] The channel stacking fusion mode in the spatial domain can separate each channel of a multi-channel image (such as an RGB image) as an independent feature map. Then, the feature maps of different sources or different processing are stacked in the channel dimension to form a new multi-channel image. Then, necessary adjustments are performed on the stacked multi-channel image, such as adjusting the channel order, applying normalization, and the like, to obtain the fused features.

[0115] The dimension difference fusion mode calculates the difference (first-order difference or high-order difference) between adjacent dimensions or adjacent time points for multi-dimensional data, so as to capture the change trend of the data in different dimensions. Then, the calculated dimension differences are fused according to certain rules, such as weighted average, summation, and the like. Then, nonlinear transformation or feature selection operations can be performed on the difference fusion result to obtain the fused features.

[0116] It can be understood that the above fusion modes can be adjusted according to actual application scenarios or the experience of those skilled in the art, and the present disclosure does not limit this.

[0117] The method for generating the medium consistency probability of the fused features of the sample in the present implementation mode clearly defines that the way of generating the fused features of the sample can be one or more of the dimension-by-dimension mean regularization fusion mode, the quantization fusion mode, the channel stacking fusion mode in the spatial domain, and the dimension difference fusion mode, thereby ensuring the quality of the fused features and improving the accuracy of the medium consistency probability predicted according to the fused features.

[0118] As one or more specific application embodiments of the embodiments of the present disclosure, the method for offline network training and online chip deployment provided by the present disclosure is described below in combination with a specific application scenario of FIG. 3a.

[0119] In the present example, taking the ultrasonic wave fingerprint as an example, the following steps are included:

[0120] Step 1: An offline training dataset with medium labels is used to construct an ultrasonic echo signal that acquires multiple phases. The ultrasonic echo signal can be denoted as a native Raw signal.

[0121] Step 1.1: A predetermined fingerprint sensor (capacitive, optical, ultrasonic, etc.) is used to sample the native domain signal of more than 2 (including 2) media. The press native domain dataset of different medium components is denoted as A, B, C, ….

[0122] Step 1.2: A ternary pair is constructed, each ternary pair containing two samples i and j from the same medium dataset and another sample k from a different medium dataset. In one example, as shown in FIG. 3b, FIG. 3b shows an exemplary schematic diagram of a ternary pair sample.

[0123] Step 2: Pre-process the raw domain signal into imaging.

[0124] Considering the significant difference of the raw domain signal modalities input by different fingerprint sensors, a specific pre-processing imaging procedure can be constructed before feeding the data into the network. The raw signal can be pre-processed to obtain the base fingerprint imaging. Then, the fingerprint imaging is pre-processed for enhancement, while preserving the corresponding quantized physical characteristics of the medium, achieving spatial and temporal signal enhancement, and maximizing the preservation of stereo characteristics. In one example, as shown in FIG. 3c, FIG. 3d and FIG. 3e, FIG. 3c is a schematic diagram of raw domain data; FIG. 3d is a schematic diagram of base fingerprint imaging data; and FIG. 3e is a schematic diagram of pre-processed enhanced fingerprint imaging.

[0125] Step 3: Generate feature (DL) domain vector representation based on deep convolutional neural network.

[0126] Step 3.1: In this example, a set of feature vector extraction techniques based on deep convolutional neural network is constructed, which converts the single-frame fingerprint echo signal tensor output by the pre-processing in step 2 into a 10-dimensional feature vector of 10x1x1 of the implicit physical medium. Specifically, as shown in FIG. 3f, FIG. 3f shows feature (DL) domain vector representation based on deep convolutional neural network.

[0127] The deep convolutional neural network includes three sub-networks, namely, a spatial enhancement sub-network, a feature extraction sub-network, and a medium description sub-network. The spatial enhancement sub-network is used to simulate pre-processing enhancement without changing the spatial resolution and channel number; the feature extraction layer is stacked and combined through convolutional layers and pooling layers to reduce the spatial resolution and increase the channel number of the raw domain signal, making good feature preparation for subsequent semantic description information extraction; finally, the medium description sub-network integrates and extracts information from the spatially condensed feature tensor to generate a 10-dimensional feature descriptor, which represents the medium characteristics of the current single-frame pressing data. The feature vector essentially describes the 10-dimensional implicit physical medium characteristics of the current pressing signal.

[0128] Specifically, the structure of the feature extraction network of the deep convolutional neural network is shown in the following table.

[0129] Step 3.2: We respectively map the three samples in the triplets to obtain three 10-dimensional feature vectors

[0130] Step 4: Combine the three 10-dimensional feature vectors obtained by mapping in the triplets two by two.

[0131] wherein two 10-dimensional feature vectors from the same medium dataset are combined into one same medium positive sample assigned label 1, while two 10-dimensional feature vectors from non-same medium dataset are combined into one non-same negative sample assigned label 0.

[0132] Step 5: Decision recognition training and decision recognition application.

[0133] Step 5.1: Build a fully connected network to build a feature fusion classification network decision classification layer to realize the medium identity classification recognition based on the positive and negative fusion samples obtained in step 4.

[0134] Step 5.2: Based on the consistency of the classification results and the true labels of the positive sample set and the negative sample set predicted by the feature fusion classification network, use the binary cross-entropy loss function for backward gradient propagation: Loss BCE = -y·log(x)-(1-y)·log(1-x)

[0135] At the same time, according to the order of gradient back propagation, the parameters of the fully connected decision classification layer and the deep convolutional neural network in step 3 are updated in turn to realize performance optimization and training iteration, until the number of loop iterations epoch of the provided training dataset reaches 200 times, or the loss function reaches the preset value, the iteration ends, the backward gradient propagation terminates, and the weight coefficients in the network are no longer updated. The fully connected decision classification layer and the deep convolutional neural network in step 3 are saved respectively.

[0136] Step 5.3: In actual deployment application, the parameters of the iteration training end can be used to first load the saved deep learning network weight, and then use steps 2, 3, and 4 to extract 10-dimensional physical medium feature descriptors and wherein i is the id of all registered fingerprints, a total of N fingers. j is the number of registered presses for the id of each registered fingerprint, a total of M registered presses.

[0137] Compared with the related art which needs to distinguish and save the original resolution image data of i x j registered samples, the present disclosure only needs to retain the 10-dimensional descriptor of each registered fingerprint template data for subsequent fusion and final decision classification. Taking the scale of 2 x 80 x 80 of ultrasonic under-screen fingerprint as an example, the storage consumption in the present disclosure is reduced to 1 / 1280 of the related art solution, basically solving the storage bottleneck problem of storing original templates for shallow fusion.

[0138] Step 5.4: Using the fingerprint matching result, index the finger id and the ridge most matching registration number [i', j'] corresponding to the current verification fingerprint data (i.e. the current unlocking pressing sample) k, and extract the corresponding medium descriptor from the pre-stored medium descriptor of i x j registration fingerprint template data Referring to step 4, the extracted medium descriptor is combined with the unlocking medium descriptor

[0139] Step 5.5: Load the full connection classification layer training weight built in step 5.1, and perform full connection feedforward calculation on the medium fusion vector in step 5.4. Perform decision recognition on the regular feature to obtain the medium identity score. When the score is greater than 0.5, it is judged that the registration fingerprint template data is the same medium, otherwise it is different medium.

[0140] Specifically, reference can also be made to FIG. 3g, which shows a flowchart of the offline network training and online chip deployment of the technical solution of FIG. 3a.

[0141] In the quantitative comparative analysis of fingerprint recognition using the method for identifying the consistency of fingerprint media in the present disclosure, the false liveness reject rate (FLRR, also known as the false rejection rate) and the spoof accept rate (SAR, also known as the false acceptance rate) can be used as performance evaluation indicators.

[0142] Wherein, FLRR refers to the proportion of fingerprint images collected from the same finger that are determined to be non-same fingers when performing 1:1 matching, expressed in percentage. No. Liveness Rejection refers to the number of live fingerprint data rejected; No. Liveness Pressing refers to the number of live fingerprint data pressing. SAR refers to the ratio of fingerprint images collected from different fingers that are incorrectly determined to be the same finger when matching, expressed in percentage. No. Spoof Accept refers to the number of fake fingerprint data accepted, and No. Spoof Attach refers to the number of fake fingerprint data participating in verification.

[0143] The following table shows the anti-counterfeiting task performance benefits brought by the experiment (Experiment) using the method for identifying consistency of fingerprint media in the present disclosure. In addition, compared with storing original image domain registration template features, using the present disclosure scheme can convert the pre-stored registration template information into a 10-dimensional feature descriptor of the registration fingerprint template data, and the pre-stored data is reduced to 1 / 1280 of the original, completely solving the core difficulty problem of the registration template storage information being too large to mass-produce. Among them, the F1 score (F1 Score), also known as the balanced F score, is a statistical index used to measure the accuracy of a binary classification model, which is the harmonic mean of the precision and recall of the classification model.

[0144] The present embodiment also provides a device for identifying consistency of fingerprint media, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0145] The device for identifying consistency of fingerprint media provided by the present embodiment, as shown in FIG. 4, comprises:

[0146] The data acquisition module 401 is configured to acquire current verification fingerprint data.

[0147] The data searching module 402 is configured to search, in the registration fingerprint template data set, for target fingerprint template data matching the current verification fingerprint data.

[0148] The feature searching module 403 is configured to search, in the template feature descriptor set, for a target feature descriptor corresponding to the target fingerprint template data, the template feature descriptor set being generated by using a pre-trained feature extraction network according to the registration fingerprint template data set.

[0149] The probability generating module 404 is configured to generate, by using a pre-trained feature extraction network, a verification feature descriptor according to the current verification fingerprint data, and generate, by using a pre-trained feature fusion classification network, a current media consistency probability of a verification fusion feature according to the verification feature descriptor and the target feature descriptor, the fusion feature being fused from the verification feature descriptor and the target feature descriptor.

[0150] In some optional implementations of the present embodiment, the pre-trained feature extraction network used by the feature searching module 403 and the pre-trained feature fusion classification network used by the probability generating module are obtained by training the following modules:

[0151] The sample obtaining module 4031 is configured to obtain a triple sample in a triple sample set, the triple sample comprising an anchor sample, a positive training sample and a negative training sample of fingerprint data, the anchor sample and the positive training sample belonging to a same template medium, and the anchor sample and the negative training sample belonging to different template media.

[0152] The feature generating module 4032 is configured to generate three feature descriptors based on the triple sample by using an initial feature extraction network, the three feature descriptors comprising an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample and a negative feature descriptor of the negative training sample.

[0153] The training network module 4033 is configured to generate a medium consistency probability of a sample fusion feature according to the three feature descriptors by using an initial feature fusion classification network, generate a loss of the medium consistency probability and a true label of the sample fusion feature, update a parameter value of the initial feature fusion classification network and a parameter value of the initial feature extraction network by using a back propagation of the loss, until a preset iteration number is reached or the loss meets a preset target, to obtain a pre-trained feature extraction network and a pre-trained feature fusion classification network; the sample fusion feature comprises a positive sample fusion feature fused from the anchor feature descriptor and the positive feature descriptor, and a negative sample fusion feature fused from the anchor feature descriptor and the negative feature descriptor.

[0154] In some optional implementations of the embodiment, the initial feature extraction network trained by the feature generating module 4032 comprises a spatial domain enhancement subnetwork, a feature extraction subnetwork and a medium descriptor network connected in sequence; the spatial domain enhancement subnetwork performs preprocessing on input fingerprint data to obtain a preprocessed feature tensor; the feature extraction subnetwork reduces the spatial resolution of the preprocessed feature tensor and increases the channel number of the preprocessed feature tensor to obtain a spatial domain condensed feature tensor; and the medium descriptor network integrates information and extracts semantics from the spatial domain condensed feature tensor to obtain a feature descriptor of a preset number of dimensions representing physical medium characteristics.

[0155] In some optional implementations of the embodiment, the spatial domain enhancement subnetwork trained by the feature generating module 4032 comprises a first convolutional layer; the feature extraction subnetwork comprises a first convolutional block, a first transition layer, a second convolutional block and a second transition layer connected in sequence; the first convolutional block adopts a plurality of convolutional kernels of a preset size stacked in sequence; the first transition layer comprises a second convolutional layer and a first average pooling layer connected in sequence; the second convolutional block adopts a plurality of convolutional kernels of a predetermined size stacked in sequence; the second transition layer comprises a third convolutional layer and a second average pooling layer connected in sequence; and the medium descriptor network comprises a depth separable convolutional layer comprising a fourth convolutional layer and a third average pooling layer connected in sequence.

[0156] In some optional implementations of the embodiment, the pre-trained feature extraction network utilized by the feature searching module 403 and the pre-trained feature fusion classification network utilized by the probability generating module are trained by the following modules, and further include: a preprocessing module 4034 configured to perform at least one of the following pre-processing imaging on the triple pair sample: spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing, and time domain amplitude extraction; and the feature generating module 4032 is further configured to: generate three feature descriptors according to the pre-processed triple pair sample by using the initial feature extraction network.

[0157] In some optional implementations of the embodiment, the initial feature fusion classification network utilized by the network training module 4033 is configured to generate sample fusion features based on at least one of the following when generating medium consistency probability of sample fusion features: dimension-by-dimension mean regular fusion mode, quantization fusion mode, spatial domain channel stacking fusion mode, and dimension difference fusion mode.

[0158] In some optional implementations of the embodiment, the medium of the target fingerprint template data searched by the data searching module 402 includes at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder, and dust.

[0159] Further function descriptions of the above-mentioned modules and units are the same as those of the corresponding embodiments, and will not be repeated here. The device for identifying fingerprint medium consistency in the embodiment is presented in the form of functional units. The units herein refer to ASIC (Application Specific Integrated Circuit, Application Specific Integrated Circuit) circuits, processors and memories executing one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0160] In the embodiment, a device is also provided, including:

[0161] The fingerprint sensor system is configured to obtain current verification fingerprint data.

[0162] The memory system; and

[0163] The control system is configured to be in electrical communication with the ultrasonic fingerprint sensor system, and the control system is further configured to perform the method for identifying fingerprint medium consistency of the above-mentioned embodiments or any of the corresponding embodiments thereof.

[0164] In the embodiment, a device is also provided, including:

[0165] The fingerprint sensor system is configured to obtain current verification fingerprint data.

[0166] The memory system; and

[0167] A control device for causing a computer to execute the method of identifying the consistency of the fingerprint medium according to the above-mentioned embodiments or any of their corresponding implementations.

[0168] Referring to FIG. 5, FIG. 5 is a structural schematic diagram of a computer device according to an optional embodiment of the present disclosure. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively connected with each other by using different buses, and can be installed on a common mainboard or in other manners as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or graphics information of a GUI stored in the memory for displaying on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory as needed. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0169] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a generic array logic, or any combination thereof.

[0170] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above-mentioned embodiments.

[0171] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function; and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some optional embodiments, the memory 20 can optionally include a memory remotely arranged with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0172] The memory 20 can include a volatile memory, e.g., a random access memory, and / or a non-volatile memory, e.g., a flash memory, a hard disk, or a solid state drive. The memory 20 can also include a combination of the above-mentioned types of memories.

[0173] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected through a bus or other means.

[0174] The input device 30 can receive input digital or character information, and generate key signal inputs in connection with a user setting of the computer device and a function control, e.g., a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), etc. The display device includes, but is not limited to, a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0175] The embodiments of the present disclosure further provide a computer readable storage medium, and the method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer code to be originally stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0176] Part of the present disclosure can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present disclosure can be invoked or provided. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file and the like, and accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0177] Although the embodiments of the present disclosure are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method of identifying fingerprint media consistency, the method comprising: The method comprises: obtaining current authentication fingerprint data; in the registration fingerprint template data set, find the target fingerprint template data matched with the current authentication fingerprint data; in the template feature descriptor set, find the target feature descriptor corresponding to the target fingerprint template data, and the template feature descriptor set is generated according to the registration fingerprint template data set by using a pre-trained feature extraction network; using the pre-trained feature extraction network, generating authentication feature descriptors according to the current authentication fingerprint data, and using a pre-trained feature fusion classification network, generating a current medium consistency probability of authentication fusion features according to the authentication feature descriptors and the target feature descriptors, wherein the authentication fusion features are fused from the authentication feature descriptors and the target feature descriptors.

2. The method of claim 1, wherein, The pre-trained feature extraction network and the pre-trained feature fusion classification network are trained by the following steps: obtain a three tuple sample in a three tuple sample set, the three tuple sample includes an anchor sample, a positive training sample and a negative training sample of fingerprint data, the anchor sample and the positive training sample belong to the same template medium, and the anchor sample and the negative training sample belong to different template media; using an initial feature extraction network, generating three feature descriptors based on the three tuple sample, the three feature descriptors include: an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample and a negative feature descriptor of the negative training sample; using an initial feature fusion classification network, generating a medium consistency probability of sample fusion features according to the three feature descriptors, generating a loss of the medium consistency probability and the real label of the sample fusion features, updating the parameter value of the initial feature fusion classification network and the parameter value of the initial feature extraction network by using the reverse gradient conduction of the loss, until the training times reach a preset iteration number or the loss meets a preset target, to obtain a pre-trained feature extraction network and a pre-trained feature fusion classification network; the sample fusion features include: a positive sample fusion feature fused from the anchor feature descriptor and the positive feature descriptor, and a negative sample fusion feature fused from the anchor feature descriptor and the negative feature descriptor.

3. The method of claim 2, wherein, The initial feature extraction network comprises: a spatial domain enhancement subnetwork, a feature extraction subnetwork and a medium descriptor network connected in sequence; the initial feature extraction network based on the three tuple sample, generating three feature descriptors, comprises: using the spatial domain enhancement subnetwork to pre-process and enhance the three tuple sample to obtain a pre-processing feature tensor; using the feature extraction subnetwork to reduce the spatial resolution of the pre-processing feature tensor and increase the channel number of the pre-processing feature tensor to obtain a spatial domain condensed feature tensor; using the medium descriptor network to integrate information and extract semantics of the spatial domain condensed feature tensor to obtain a feature descriptor of a preset number of dimensions representing physical medium characteristics.

4. The method of claim 3, wherein, The spatial domain enhancement subnetwork comprises a first convolutional layer; The feature extraction subnetwork comprises: a first convolutional block, a first transition layer, a second convolutional block and a second transition layer connected in sequence; wherein the first convolutional block adopts a plurality of preset size convolution kernels stacked in succession; the first transition layer comprises a second convolutional layer and a first average pooling layer connected in sequence; the second convolutional block adopts a plurality of predetermined size convolution kernels stacked in succession; the second transition layer comprises a third convolutional layer and a second average pooling layer connected in sequence; The medium description subnetwork comprises: a depth separable convolutional layer comprising a fourth convolutional layer and a third average pooling layer connected in sequence.

5. The method of claim 2, wherein, The pre-trained feature extraction network and the pre-trained feature fusion classification network are trained by the following steps, and further comprise: performing at least one of the following preprocessing imaging on the triple sample: spatial rotation, affine transformation, displacement, flip, denoising, enhancement, value domain quantization, time domain smoothing and time domain amplitude extraction; The initial feature extraction network is used to generate three feature descriptors based on the triple sample, comprising: using the initial feature extraction network to generate three feature descriptors according to the preprocessed triple sample.

6. The method of claim 2, wherein, When the initial feature fusion classification network generates the medium consistency probability of the sample fusion feature, it is based on at least one of the following to generate the sample fusion feature: mean regular fusion mode, quantization fusion mode, channel stacking fusion mode in spatial domain and dimension difference fusion mode.

7. The method of claim 1, wherein, The medium of the target fingerprint template data comprises at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder and dust.

8. A device for identifying the consistency of fingerprint media, characterized in that, The device comprises: A data acquisition module is configured to acquire current verification fingerprint data; A data searching module is configured to search for target fingerprint template data matching the current verification fingerprint data in a registered fingerprint template data set in the registered fingerprint template data set; A feature searching module is configured to search for a target feature descriptor corresponding to the target fingerprint template data in a template feature descriptor set, which is generated by a pre-trained feature extraction network based on the registered fingerprint template data set; A probability generating module is configured to generate a verification feature descriptor based on the current verification fingerprint data using the pre-trained feature extraction network, and generate a current medium consistency probability of a verification fusion feature based on the verification feature descriptor and the target feature descriptor using a pre-trained feature fusion classification network, wherein the verification fusion feature is fused from the verification feature descriptor and the target feature descriptor.

9. The apparatus of claim 8, wherein, The device further comprises: A sample acquisition module is configured to acquire a triple sample in a triple sample set, wherein the triple sample comprises an anchor sample, a positive training sample and a negative training sample of fingerprint data, the anchor sample and the positive training sample belong to the same template medium, and the anchor sample and the negative training sample belong to different template media; The feature generation module is configured to generate three feature descriptors based on the triple sample by using an initial feature extraction network, the three feature descriptors including an anchor feature descriptor of the anchor sample, a positive feature descriptor of the positive training sample, and a negative feature descriptor of the negative training sample; The training network module is configured to generate a medium consistency probability of a sample fusion feature according to the three feature descriptors by using an initial feature fusion classification network, generate a loss of the medium consistency probability and a true label of the sample fusion feature, update parameter values of the initial feature fusion classification network and parameter values of the initial feature extraction network by using a back propagation of the loss until a preset iteration number is reached or the loss meets a preset target, and obtain a pre-trained feature extraction network and a pre-trained feature fusion classification network; the sample fusion feature includes a positive sample fusion feature fused by the anchor feature descriptor and the positive feature descriptor, and a negative sample fusion feature fused by the anchor feature descriptor and the negative feature descriptor.

10. The apparatus of claim 9, wherein, The initial feature extraction network includes a spatial domain enhancement sub-network, a feature extraction sub-network, and a medium descriptor network connected in sequence. The feature generation module is configured to perform pre-processing enhancement on the triple sample by using the spatial domain enhancement sub-network to obtain a pre-processed feature tensor, reduce the spatial resolution of the pre-processed feature tensor by using the feature extraction sub-network to increase the channel number of the pre-processed feature tensor to obtain a spatially condensed feature tensor, and perform information integration and semantic extraction on the spatially condensed feature tensor by using the medium descriptor network to obtain a feature descriptor of a preset number of dimensions representing physical medium characteristics.

11. The apparatus of claim 10, wherein, The spatial domain enhancement sub-network includes a first convolutional layer. The feature extraction sub-network includes a first convolutional block, a first transition layer, a second convolutional block, and a second transition layer connected in sequence; the first convolutional block is stacked with a plurality of convolutional kernels of a preset size; the first transition layer includes a second convolutional layer and a first average pooling layer connected in sequence; the second convolutional block is stacked with a plurality of convolutional kernels of a predetermined size; and the second transition layer includes a third convolutional layer and a second average pooling layer connected in sequence. The medium descriptor network includes a depth separable convolutional layer including a fourth convolutional layer and a third average pooling layer connected in sequence.

12. The apparatus of claim 9, wherein, The device further includes a pre-processing module. The pre-processing module is configured to perform at least one of the following pre-processing imaging on the triple sample: spatial domain rotation, affine transformation, displacement, flipping, denoising, enhancement, value domain quantization, time domain smoothing, and time domain amplitude extraction. The feature generation module is configured to generate three feature descriptors based on the triple sample after pre-processing imaging by using an initial feature extraction network.

13. The device of claim 9, wherein The training network module is configured to generate sample fusion features based on at least one of the following: a mean regular fusion mode in each dimension, a quantization fusion mode, a channel stacking fusion mode in a spatial domain, and a dimension difference fusion mode, when the initial feature fusion classification network is utilized to generate medium consistency probability of sample fusion features.

14. The apparatus of claim 8, wherein, The medium of the target fingerprint template data searched by the data searching module includes at least one of the following: fruit peel, skin, white glue, black glue, resin, printing paper, conductive pen, carbon powder, and dust.

15. An apparatus comprising: a fingerprint sensor system configured to acquire current authentication fingerprint data; a memory system; and a control system configured to be in electrical communication with the ultrasonic fingerprint sensor system, the control system further configured to perform the method of identifying fingerprint medium consistency of any one of claims 1-7.

16. An apparatus comprising: a fingerprint sensor system configured to acquire current authentication fingerprint data; a memory system; and a control system configured to perform the method of identifying fingerprint medium consistency of any one of claims 1-7.

Images