Fingerprint retrieval methods, devices, non-volatile storage media, and electronic devices

Abstract

This application discloses a fingerprint retrieval method, apparatus, non-volatile storage medium, and electronic device. The method includes: determining the minutiae distribution and orientation map of a fingerprint image to be retrieved, wherein the minutiae distribution and orientation map represent the distribution and orientation information of the minutiae in the fingerprint image; processing the fingerprint image and the minutiae distribution and orientation map using a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and determining a set of candidate fingerprint images in a fingerprint database based on the feature vector. This application solves the technical problem of low fingerprint retrieval efficiency caused by the manual design of feature vectors for fingerprint images in related technologies.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a fingerprint retrieval method, apparatus, non-volatile storage medium, and electronic device. Background Technology

[0002] In the field of feature vector related techniques, when determining the set of candidate fingerprint images corresponding to the fingerprint image to be retrieved, the usual approach is to manually design fingerprint feature vectors. However, the problem with this approach is that the design process is complex, requires a high level of professional knowledge from the designers, and the resulting fingerprint feature vectors are highly specialized and lack generalization.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a fingerprint retrieval method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problem of low fingerprint retrieval efficiency caused by the manual design of feature vectors of fingerprint images to be retrieved in related technologies.

[0005] According to one aspect of the embodiments of this application, a fingerprint retrieval method is provided, comprising: determining the minutiae distribution and orientation map of a fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution information and orientation information of minutiae in the fingerprint image to be retrieved; processing the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and determining a set of candidate fingerprint images in a fingerprint database based on the feature vector.

[0006] Optionally, the minutiae distribution and orientation map include a first channel and a second channel; the step of determining the minutiae distribution and orientation map of the fingerprint image to be retrieved includes: determining the position information and orientation information of the minutiae in the fingerprint image to be retrieved; based on the position information of the minutiae, determining the first component of any pixel in the minutiae distribution and orientation map in the first channel, and the second component of any pixel in the second channel.

[0007] Optionally, the step of determining the first component of any pixel in the minutiae distribution and orientation map in the first channel and the second component of any pixel in the second channel based on the minutiae location information includes: determining the location information of the minutiae in the minutiae distribution and orientation map based on the location information of the minutiae in the fingerprint image to be retrieved; calculating the Manhattan distance between the target pixel and the minutiae based on the location information and orientation information of the minutiae, wherein the target pixel is any pixel in the minutiae distribution and orientation map; and determining the first component of the target pixel in the first channel and the second component in the second channel based on the Manhattan distance corresponding to the target pixel and the orientation information of the minutiae.

[0008] Optionally, the step of determining the first component of the target pixel in the first channel and the second component in the second channel based on the Manhattan distance corresponding to the target pixel and the direction information of the minutiae includes: determining the minutiae mask corresponding to the minutiae and determining a preset distance threshold based on the minutiae mask; if the Manhattan distance is greater than the preset distance threshold, determining that both the first component and the second component of the target pixel are zero; if the Manhattan distance is not greater than the preset distance threshold, determining the first component based on the direction information of the minutiae and the channel number of the first channel, and determining the second component based on the direction information of the minutiae and the channel number of the second channel.

[0009] Optionally, the fingerprint feature extraction neural network model includes a feature extraction layer, a feature fusion layer, and a mapping module. The feature extraction layer includes multiple feature extraction modules, each composed of multiple residual modules, used to extract feature maps of the fingerprint image to be retrieved based on the distribution of minutiae and the orientation map. The feature maps extracted by each feature extraction module have different resolutions. The feature fusion layer fuses the feature maps extracted by each feature extraction module, excluding the first and second feature maps, to obtain a target feature map. The first feature map is the feature map with the highest resolution extracted by each feature extraction module, and the second feature map is the feature map with the second highest resolution extracted by each feature extraction module. The mapping module determines the feature vector of the fingerprint image to be retrieved based on the target feature map.

[0010] Optionally, the step of determining the feature vector of the fingerprint image to be retrieved based on the target feature map includes: performing image segmentation processing on the fingerprint image to be retrieved to obtain a target feature map matrix; obtaining an initial feature vector of the fingerprint image to be retrieved based on the target feature map matrix and the target feature map; normalizing the initial feature vector and using the normalized initial feature vector as the feature vector of the fingerprint image to be retrieved.

[0011] Optionally, the fingerprint feature extraction neural network is trained as follows: a key network and a query network are determined, wherein the key network and the query network are encoders with the same model structure, and the model structure of the key network and the query network is the same as the model structure of the fingerprint feature extraction neural network; the query network is trained by gradient propagation, and after each training, the key network is trained by exponential average moving average based on the model parameters of the query network; after training, the query network is retained as the fingerprint feature extraction neural network.

[0012] Optionally, the steps of training the key network using gradient propagation and updating it, and then training the query network using exponential moving average based on the model parameters of the key network after each training iteration, include: after each update of the model parameters of the key network and the query network, obtaining the first feature sequence output by the key network and the second feature sequence output by the query network; determining the triplet loss of the key network and the query network based on the first and second feature sequences; determining whether the key network and the query network have converged based on the triplet loss; and confirming the completion of training after confirming that the key network and the query network have converged.

[0013] According to another aspect of the embodiments of this application, a fingerprint retrieval device is also provided, comprising: a first processing module, configured to determine the minutiae distribution and orientation map of a fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution information and orientation information of minutiae in the fingerprint image to be retrieved; a second processing module, configured to process the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and a third processing module, configured to determine a set of candidate fingerprint images in a fingerprint database using the feature vector.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, and the program controls the device where the non-volatile storage medium is located to execute a fingerprint retrieval method when it runs.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a fingerprint retrieval method during runtime.

[0016] In this embodiment, the minutiae distribution and orientation map of the fingerprint image to be retrieved are determined. The minutiae distribution and orientation map represent the distribution and orientation information of the minutiae in the fingerprint image. A fingerprint feature extraction neural network processes the fingerprint image and the minutiae distribution and orientation map to obtain a feature vector of the fingerprint image. The length of the feature vector is a preset length. A set of candidate fingerprint images is determined in the fingerprint database using the feature vector. By determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, and processing them using a fingerprint feature extraction neural network to obtain the feature vector, the goal of obtaining the feature vector of the fingerprint image to be retrieved based on the minutiae information without manual annotation is achieved. This improves retrieval efficiency and solves the problem of low fingerprint retrieval efficiency caused by manually designing the feature vector of the fingerprint image in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of the structure of a computer terminal (mobile terminal) according to an embodiment of this application;

[0019] Figure 2 This is a schematic flowchart of a fingerprint retrieval method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of detail points in a fingerprint image according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a fingerprint feature extraction neural network according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of a feature map according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram illustrating the training process of a fingerprint feature extraction neural network according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of a fingerprint retrieval device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained below:

[0028] Minutes: Minutes are the smallest graphic units in a fingerprint cluster and one of the most important features in fingerprint recognition. Each minute consists of a start point, an end point, and an arc connecting the two. The shape, number, and arrangement of minutes are unique to each person.

[0029] Triple loss: Triple loss is a loss function commonly used to train image generation models. Its basic idea is to define the loss function by comparing the similarity between the generated image and the real images in the training set as well as other samples. The similarity includes the similarity between the training sample and the positive sample, and the similarity between the training sample and the negative sample.

[0030] In the field of large-scale fingerprint recognition, to improve the efficiency and accuracy of fingerprint retrieval, it is common practice to convert the fingerprint image to be retrieved into a feature vector, and then use the feature vector to retrieve the corresponding fingerprint from a database. Related technologies typically employ manual annotation, where the fingerprint feature information contained in the image to be retrieved is labeled, and then the labeled image is converted into a feature vector of the fingerprint to be retrieved. The problem with this method is that it requires highly skilled annotators with sufficient knowledge of the fingerprint field. Furthermore, manual annotation is inefficient.

[0031] Related technologies also provide a method for extracting feature vectors from fingerprint images to be retrieved using neural networks. However, these neural networks only extract global or local features from the image to obtain feature vectors. Specifically, when only considering global features, the information inherent in the fingerprint itself is not taken into account during the feature vector acquisition process. This results in insufficient discriminative power in the final feature vectors; that is, feature vectors from different fingerprints may be similar, leading to low fingerprint retrieval efficiency and accuracy. While considering only local features does take fingerprint information into account, the feature vector generation speed is slow, and the resulting feature vector length is not fixed, resulting in low retrieval efficiency and making it unsuitable for large-scale fingerprint retrieval scenarios.

[0032] To address this issue, relevant solutions are provided in the embodiments of this application, which are described in detail below.

[0033] According to an embodiment of this application, a method embodiment for fingerprint retrieval is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a fingerprint retrieval method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fingerprint retrieval method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the fingerprint retrieval method of the aforementioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0038] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0039] Under the above operating environment, this application provides a fingerprint retrieval method, such as... Figure 2 As shown, the method includes the following steps:

[0040] Step S202: Determine the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution and orientation information of the minutiae in the fingerprint image to be retrieved;

[0041] In the technical solution provided in step S202, the minutiae distribution and orientation map include a first channel and a second channel; the step of determining the minutiae distribution and orientation map of the fingerprint image to be retrieved includes: determining the position information and orientation information of the minutiae in the fingerprint image to be retrieved; based on the position information of the minutiae, determining the first component of any pixel in the minutiae distribution and orientation map in the first channel, and the second component of any pixel in the second channel.

[0042] Specifically, the details in the fingerprint image, such as Figure 3 As shown, Figure 3 The location of the graphic composed of hollow circles and line segments in the fingerprint indicates the location of the micro-points. Figure 3 Fingerprint images a and b are matched fingerprint images, while fingerprint image c is a fingerprint image that does not match fingerprint images a and b. Since the distribution of minutiae in fingerprints varies uniquely among individuals, fully considering the relevant information of minutiae when generating the feature vector of the fingerprint image to be retrieved can effectively improve the efficiency and accuracy of the retrieval process.

[0043] In some embodiments of this application, the step of determining the first component of any pixel in the minutiae distribution and orientation map on the first channel and the second component of any pixel in the second channel based on the location information of the minutiae includes: determining the location information of the minutiae in the minutiae distribution and orientation map based on the location information of the minutiae in the fingerprint image to be retrieved; calculating the Manhattan distance between the target pixel and the minutiae based on the location information and orientation information of the minutiae, wherein the target pixel is any pixel in the minutiae distribution and orientation map; and determining the first component of the target pixel in the first channel and the second component in the second channel based on the Manhattan distance corresponding to the target pixel and the orientation information of the minutiae.

[0044] Specifically, assume the fingerprint image to be retrieved has a size of H×W, where H is the image height, W is the image width, and the image contains N... m If there are 3 fingerprint minutiae, then each fingerprint minutiae M can be represented as:

[0045]

[0046] In the above formula, the superscript i represents the i-th fingerprint minutiae. and The coordinates of the fingerprint minutiae in the fingerprint image to be retrieved are defined. Specifically, for ease of description, in the embodiments of this application, the lower left corner pixel in the fingerprint image to be retrieved can be defined as the origin of the coordinate system, with the positive Y-axis pointing upwards and the positive Z-axis pointing to the right, and the difference between the coordinate values ​​of two adjacent pixels is determined to be 1. The orientation information of the i-th fingerprint minutiae is represented by the angle between the i-th fingerprint minutiae and the horizontal axis, where the horizontal axis is parallel to the width of the image. Since the minutiae information in the original fingerprint image to be retrieved is non-Euclidean, it cannot be directly processed by neural networks such as CNNs. It is necessary to transform the minutiae information in the fingerprint image to be retrieved while preserving as much minutiae information as possible. The method for determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, provided in this application embodiment, can convert the fingerprint minutiae information into Euclidean information while avoiding the loss of minutiae information.

[0047] Specifically, as key domain knowledge in fingerprinting, minutiae exhibit stability and robustness to fingerprint impression conditions. That is, the minutiae distribution is different for each object's fingerprint, and the minutiae information is consistent for the same fingerprint of the same object. Furthermore, the orientation of each minutiae also contains rich biometric information. Therefore, embodiments of this application provide a minutiae distribution and orientation map (MDDM) to retain more minutiae information when converting the original minutiae into Euclidean form.

[0048] As an optional implementation, this application provides a method for determining a first component of a target pixel in a first channel and a second component in a second channel based on the Manhattan distance corresponding to the target pixel and the direction information of the minutiae. The method includes the following steps: determining a minutiae mask corresponding to the minutiae and determining a preset distance threshold based on the minutiae mask; when the Manhattan distance is greater than the preset distance threshold, determining that both the first component and the second component of the target pixel are zero; when the Manhattan distance is not greater than the preset distance threshold, determining the first component based on the direction information of the minutiae and the channel number of the first channel, and determining the second component based on the direction information of the minutiae and the channel number of the second channel.

[0049] Specifically, the formulas for determining the first and second components of fingerprint minutiae in MDDM are as follows:

[0050]

[0051] Specifically, in the above formula, dist1(i,j,m) x ,m yThe distance (i, j) represents the L1 distance, or Manhattan distance, between any pixel in the MDDM and the minutiae of the fingerprint, where i and j are the coordinates of that pixel in the MDDM. For ease of annotation, a Cartesian coordinate system can be established in the MDDM, with the bottom left pixel as the origin, the positive Y-axis pointing upwards, and the positive Z-axis pointing to the right. The difference between the coordinates of two adjacent pixels is set to 1. It's understandable that the size of the MDDM is the same as the size of the fingerprint image to be retrieved; therefore, the coordinates of the same feature point are the same in both the MDDM and the fingerprint image. It's important to note that the coordinate system in the MDDM can also have different specifications than the coordinate system in the fingerprint image to be retrieved without affecting the final result.

[0052] In the above formula, k represents the channel identifier, taking a value of 0 or 1. When k is 0, it represents the first channel; when k is 1, it represents the second channel. T represents the detail feature point (m). x ,m y The corresponding mask, dist1(i,j,m) x ,m y )≤T indicates that the pixel is within the mask coverage area of ​​the detail feature point.

[0053] After obtaining the MDDM using the above formula, the MDDM can guide the machine learning model to focus more on the feature information of minutiae in the fingerprint image to be retrieved during the training process. This allows the trained machine learning model to extract feature vectors containing the minutiae features of the fingerprint image to be retrieved. In other words, the MDDM can serve as prior information for the fingerprint image to be retrieved, and the fingerprint feature extraction neural network can use the MDDM to determine the location and specific information of minutiae in the fingerprint image to be retrieved.

[0054] Step S204: Process the fingerprint image to be retrieved, the distribution of minutiae, and the orientation map through a fingerprint feature extraction neural network to obtain the feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length;

[0055] In the technical solution provided in step S204, such as Figure 4 As shown, the fingerprint feature extraction neural network model provided in this application embodiment is an improved neural network model based on the ResNet model. From Figure 4As can be seen, the fingerprint feature extraction neural network includes a feature extraction layer, a feature fusion layer, and a mapping module. The feature extraction layer comprises multiple feature extraction modules, each consisting of multiple residual modules, used to extract feature maps from the fingerprint image to be retrieved based on the distribution of minutiae and the orientation map. The feature maps extracted by each feature extraction module have different resolutions. The feature fusion layer fuses the feature maps extracted by each feature extraction module, excluding the first and second feature maps, to obtain the target feature map. The first feature map is the highest-resolution feature map extracted by each feature extraction module, and the second feature map is the second highest-resolution feature map extracted by each feature extraction module. The mapping module determines the feature vector of the fingerprint image to be retrieved based on the target feature map.

[0056] Specifically, because the first and second feature maps have excessively high resolution and carry too many other image features besides fingerprint minutiae features, the computational load in the feature fusion stage increases significantly without significantly improving the performance of the final result. Therefore, a fingerprint feature extraction neural network can be designed to discard the first and second feature maps. Simultaneously, because minutiae have small spatial dimensions and low resolution, some minutiae features may be lost in the lowest-level feature map. Therefore, feature maps other than the first and second feature maps can be retained and fused with the lowest-level feature map to obtain the target feature map.

[0057] As an alternative implementation method, such as Figure 4 As shown, the feature extraction layer can include thirty residual modules, with every six residual modules forming a feature extraction module. Thus, after processing the fingerprint image to be retrieved by the feature extraction layer, five sets of feature maps with different resolutions can be obtained. The third, fourth, and fifth sets of feature maps are shown below. Figure 5 As shown. Furthermore, using thirty residual modules as the basic building blocks for feature extraction effectively avoids overfitting during training.

[0058] In the feature fusion stage, since the third, fourth, and fifth feature maps have different resolutions, these feature maps can be processed using a convolutional layer after batch normalization layers, ReLU layers, and multiple upsampling layers, thereby achieving feature map fusion. Additionally, from... Figure 4 As can be seen, the number of upsampling layers is different for feature maps of different resolutions, that is, the number of upsampling times is different. This can make the resolution of each set of feature maps consistent after processing.

[0059] In some embodiments of this application, the step of determining the feature vector of the fingerprint image to be retrieved based on the target feature map includes: performing image segmentation processing on the fingerprint image to be retrieved to obtain a target feature map matrix; obtaining an initial feature vector of the fingerprint image to be retrieved based on the target feature map matrix and the target feature map; normalizing the initial feature vector and using the normalized initial feature vector as the feature vector of the fingerprint image to be retrieved.

[0060] The final formula for the feature vector is as follows:

[0061]

[0062] In the above formula, f represents the feature vector, F represents the target feature map, S represents the target feature map matrix, s(i,j) represents the value of the element in the matrix at row i and column j respectively, with the value range of the element being {0,1}, F(i,j) represents the pixel value in the i-th row and j-th column of the target feature map, and N is the number of non-zero elements in matrix S.

[0063] Specifically, FingerNet can be used to segment the fingerprint image to be retrieved to obtain the target feature map matrix. When segmenting the fingerprint image, 8×8 pixel regions can be grouped together, and it can be determined whether each group contains pixels belonging to the fingerprint region. If so, the element in the corresponding matrix of that group is set to 1; otherwise, it is set to 0. It should be noted that there is no overlap between different groups when grouping the pixel regions in the fingerprint image.

[0064] As an alternative implementation method, such as Figure 6 As shown, the fingerprint feature extraction neural network is trained in the following way: a key network and a query network are determined, wherein the key network and the query network are encoders with the same model structure, and the model structure of the key network and the query network is the same as the model structure of the fingerprint feature extraction neural network; the query network is trained by gradient propagation, and after each training, the key network is trained by exponential average moving based on the model parameters of the query network; after training, the query network is retained as the fingerprint feature extraction neural network.

[0065] Specifically, the steps of training the key network using gradient propagation and updating it, and then training the query network using exponential moving average based on the model parameters of the key network after each training iteration include: after each update of the model parameters of the key network and the query network, obtaining the first feature sequence output by the key network and the second feature sequence output by the query network; determining the triplet loss of the key network and the query network based on the first and second feature sequences; determining whether the key network and the query network have converged based on the triplet loss; and confirming the completion of training after confirming that the key network and the query network have converged.

[0066] Specifically, from Figure 6 As can be seen from this, during the training phase, the key network The generated features will continuously populate the historical queue Q. k Furthermore, the most recently generated fingerprint feature vector will gradually replace the earliest generated fingerprint feature vector, meaning the historical queue Q... k The length is determined. It can be derived from the historical queue Q corresponding to each anchored data and its counterpart. k The most difficult negative examples (i.e., mismatched fingerprints) are identified to obtain triples under the Online Hard Example Mining (OHEM) strategy to calculate the triple loss, and the queue length L can be used. TQ This is used to constrain these triples in each batch.

[0067] It can be seen that the triplet loss obtained through the momentum strategy described above can make the feature vectors generated by the neural network under training robust and compact. Furthermore, using the Online Hard Example Mining (OHEM) strategy to select triples with greater information content to process the triplet loss can accelerate the convergence speed of the network. Additionally, the key network... The generated features are continuously added to the historical feature queue, which can expand the search range of triples.

[0068] As an alternative implementation, during training, NX fingerprint images and two feature vectors for each fingerprint image can be randomly selected from the entire training set to obtain a mini-batch training dataset.

[0069] In X, for any i∈N X ,subscript x 2i-1 and the following table x 2i These are two different feature vectors corresponding to the same fingerprint image.

[0070] During training, the following feature set can be obtained by processing the aforementioned mini-batch training dataset using a key network:

[0071]

[0072] And by processing the aforementioned mini-batch training dataset through a query network, the following feature set is obtained:

[0073]

[0074] Then the first 2N from the historical queue can be... x The feature vectors are replaced with F k The loss function for each triple in the historical queue can be expressed as follows:

[0075]

[0076]

[0077] Where the above parameter m t is a hyperparameter of the boundary, and dist2 represents the L2 distance between the two vectors.

[0078] Based on the above formula, the overall triplet loss with historical queues can be obtained as follows:

[0079]

[0080] It is important to note that the feature vectors generated and used during the training process described above are all normalized feature vectors. The parameter j in the triplet loss function formula above is derived from... Figure 6 As can be seen, this represents the index of the feature vector in the historical queue. Additionally... Figure 6 In the context of R, parameter L represents the total number of feature vectors in the historical queue, and R represents the total number of feature vectors in the historical queue. c This represents a real number vector of length C, where the value of C can be set by the user, for example, to 256.

[0081] Step S206: Determine the set of candidate fingerprint images in the fingerprint database based on the feature vectors.

[0082] Specifically, the method provided in this application embodiment can convert the fingerprint image to be retrieved into a fixed-length feature vector, and then calculate the cosine similarity between the feature vector and other feature vectors in the fingerprint database to determine the candidate fingerprint image set. The specific formula is as follows:

[0083] sim(x1,x2)= <f x1 ,f x2 >

[0084] In the above formula, sim(x1,x2) represents the similarity between the first fingerprint image x1 and the second fingerprint image x2. <f x1 ,f x2 > represents the feature vector f of the first fingerprint image. x1 Second fingerprint image fx2 The inner product between the feature vectors is the normalized cosine similarity.

[0085] As an optional implementation, the candidate fingerprint set can be determined from the fingerprint database using various preset rules. For example, a similarity threshold can be set to determine that fingerprint images whose cosine similarity between the feature vectors of the fingerprint database and the feature vectors of the fingerprint image to be retrieved is greater than the preset threshold belong to the candidate fingerprint image set. Alternatively, the number of fingerprint images included in the candidate fingerprint image set can be determined first, and then, based on the similarity between the feature vectors of each fingerprint image in the fingerprint database and the feature vectors of the fingerprint image to be retrieved, the number of fingerprint images with the highest similarity can be selected to form the candidate fingerprint image set.

[0086] In addition, after retrieving the candidate fingerprint image set, various types of information associated with the fingerprint images in the candidate fingerprint image set can be determined, thereby determining the information associated with the fingerprint image to be retrieved, such as the identity information indicated by the fingerprint image to be retrieved.

[0087] By determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, where the minutiae distribution and orientation map represent the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; processing the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain the feature vector of the fingerprint image to be retrieved, where the length of the feature vector is a preset length; determining the target fingerprint image in the fingerprint database using the feature vector, where the cosine similarity between the feature vector of the target fingerprint image and the feature vector of the fingerprint image to be retrieved is greater than a preset cosine similarity threshold, this method achieves the goal of obtaining the feature vector of the fingerprint image to be retrieved based on the minutiae information without manual annotation, thereby improving the technical effect of improving retrieval efficiency and solving the technical problem of low fingerprint retrieval efficiency caused by manually designing the feature vector of the fingerprint image to be retrieved in related technologies.

[0088] To verify whether the fingerprint retrieval method provided in this application can effectively improve the efficiency and accuracy of fingerprint retrieval, multiple fingerprint datasets (such as the rolling fingerprint dataset RFD and the extended rolling fingerprint dataset ERFD) were also used in the embodiments of this application to verify the fingerprint retrieval method provided in the embodiments of this application.

[0089] It should be noted that the error rate under a given penetration rate is commonly used as an indicator to verify retrieval performance in related technologies. However, this indicator cannot fully reflect the retrieval performance. Therefore, this application provides top-k accuracy of fingerprint recognition as a performance indicator. After testing, the retrieval method provided in this application is superior to fingerprint retrieval methods in related technologies in terms of retrieval accuracy, and the time taken to generate the feature vectors of each fingerprint is approximately 30ms, which is significantly better than related technologies.

[0090] This application provides a fingerprint retrieval device. Figure 7 This is a schematic diagram of the device, as shown below. Figure 7 As shown, the device includes a first processing module 70, used to determine the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; a second processing module 72, used to process the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain the feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and a third processing module 74, used to determine a set of candidate fingerprint images in the fingerprint database based on the feature vector.

[0091] In some embodiments of this application, the minutiae distribution and orientation map include a first channel and a second channel; the step of the first processing module 70 in determining the minutiae distribution and orientation map of the fingerprint image to be retrieved includes: determining the position information and orientation information of the minutiae in the fingerprint image to be retrieved; and based on the position information of the minutiae, determining the first component of any pixel in the minutiae distribution and orientation map in the first channel, and the second component of any pixel in the second channel.

[0092] In some embodiments of this application, the step of the first processing module 70 determining the first component of any pixel in the minutiae distribution and orientation map in the first channel and the second component of any pixel in the second channel based on the location information of the minutiae includes: determining the location information of the minutiae in the minutiae distribution and orientation map based on the location information of the minutiae in the fingerprint image to be retrieved; calculating the Manhattan distance between the target pixel and the minutiae based on the location information and orientation information of the minutiae, wherein the target pixel is any pixel in the minutiae distribution and orientation map; and determining the first component of the target pixel in the first channel and the second component in the second channel based on the Manhattan distance corresponding to the target pixel and the orientation information of the minutiae.

[0093] In some embodiments of this application, the step of the first processing module 70 determining the first component of the target pixel in the first channel and the second component in the second channel based on the Manhattan distance corresponding to the target pixel and the direction information of the minutiae includes: determining the minutiae mask corresponding to the minutiae and determining a preset distance threshold based on the minutiae mask; determining that the first component and the second component of the target pixel are both zero when the Manhattan distance is greater than the preset distance threshold; and determining the first component based on the direction information of the minutiae and the channel number of the first channel, and determining the second component based on the direction information of the minutiae and the channel number of the second channel when the Manhattan distance is not greater than the preset distance threshold.

[0094] In some embodiments of this application, the fingerprint feature extraction neural network model includes a feature extraction layer, a feature fusion layer, and a mapping module. The feature extraction layer includes multiple feature extraction modules, each composed of multiple residual modules, used to extract feature maps of the fingerprint image to be retrieved based on the distribution of minutiae and the orientation map. The feature maps extracted by each feature extraction module have different resolutions. The feature fusion layer fuses the feature maps extracted by each feature extraction module, excluding the first and second feature maps, to obtain a target feature map. The first feature map is the feature map with the highest resolution extracted by each feature extraction module, and the second feature map is the feature map with the second highest resolution extracted by each feature extraction module. The mapping module determines the feature vector of the fingerprint image to be retrieved based on the target feature map.

[0095] In some embodiments of this application, the step of the fingerprint retrieval device determining the feature vector of the fingerprint image to be retrieved based on the target feature map includes: performing image segmentation processing on the target feature map to obtain a target feature map matrix, wherein the elements in the target feature map matrix correspond one-to-one with the pixels in the target feature map; obtaining the initial feature vector of the fingerprint image to be retrieved based on the target feature map matrix and the target feature map; normalizing the initial feature vector; and using the normalized initial feature vector as the feature vector of the fingerprint image to be retrieved.

[0096] In some embodiments of this application, the fingerprint feature extraction neural network is trained in the following manner: determining a key network and a query network, wherein the key network and the query network are encoders with the same model structure, and the model structure of the key network and the query network is the same as the model structure of the fingerprint feature extraction neural network; training the query network using gradient propagation for updates, and after each training iteration, training the key network using exponential average moving average based on the model parameters of the query network; after training, retaining the query network as the fingerprint feature extraction neural network.

[0097] In some embodiments of this application, the steps of training the key network using gradient propagation and updating it, and then training the key network using exponential moving average based on the model parameters of the query network after each training iteration include: after each update of the model parameters of the key network and the query network, obtaining the first feature sequence output by the key network and the second feature sequence output by the query network; determining the triplet loss of the key network and the query network based on the first feature sequence and the second feature sequence; determining whether the key network and the query network have converged based on the triplet loss; and confirming the completion of training after confirming that the key network and the query network have converged.

[0098] It should be noted that each module in the fingerprint retrieval device described above can be a program module (e.g., a set of program instructions to implement a specific function) or a hardware module. For the latter, it can take the following forms, but is not limited to them: each of the above modules is represented by a processor, or the functions of each of the above modules are implemented by a processor.

[0099] This application provides a non-volatile storage medium storing a program. During program execution, the device containing the non-volatile storage medium performs the following fingerprint retrieval method: determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to represent the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; processing the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and determining a set of candidate fingerprint images in a fingerprint database based on the feature vector.

[0100] This application provides an electronic device, which includes a processor and a memory. The processor runs a program stored in the memory, wherein the program executes the following fingerprint retrieval method: determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; processing the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and determining a set of candidate fingerprint images in a fingerprint database based on the feature vector.

[0101] This application provides a computer program that can be run by a processor in an electronic device, and during runtime, controls the electronic device to perform the following fingerprint retrieval method: determining the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; processing the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length; and determining a set of candidate fingerprint images in a fingerprint database based on the feature vector.

[0102] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0107] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A fingerprint retrieval method, characterized in that, include: Determine the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution and orientation information of the minutiae in the fingerprint image to be retrieved; The fingerprint image to be retrieved and the minutiae distribution and orientation map are processed by a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved. The length of the feature vector is a preset length, and the minutiae distribution and orientation map include a first channel and a second channel. A set of candidate fingerprint images is determined in the fingerprint database based on the aforementioned feature vectors; The steps for determining the minutiae distribution and orientation map of the fingerprint image to be retrieved include: Determine the position and orientation information of the minutiae in the fingerprint image to be retrieved; Based on the location information of the minutiae, determine the first component of any pixel in the minutiae distribution and orientation map on the first channel, and the second component of any pixel in the second channel. The location information of the minutiae in the minutiae distribution and orientation map is determined based on the location information of the minutiae in the fingerprint image to be retrieved. Based on the location information and orientation information of the minutiae, calculate the Manhattan distance between the target pixel and the minutiae, where the target pixel is any pixel in the minutiae distribution and orientation map. Based on the Manhattan distance corresponding to the target pixel and the orientation information of the minutiae, determine the first component of the target pixel in the first channel and the second component in the second channel. A minutiae mask corresponding to the minutiae is determined, and a preset distance threshold is determined based on the minutiae mask. If the Manhattan distance is greater than the preset distance threshold, both the first and second components of the target pixel are determined to be zero. If the Manhattan distance is not greater than the preset distance threshold, the first component is determined based on the orientation information of the minutiae and the channel number of the first channel, and the second component is determined based on the orientation information of the minutiae and the channel number of the second channel.

2. The fingerprint retrieval method according to claim 1, characterized in that, The fingerprint feature extraction neural network model includes a feature extraction layer, a feature fusion layer, and a mapping module. The feature extraction layer includes multiple feature extraction modules, each of which consists of multiple residual modules, used to extract feature maps of the fingerprint image to be retrieved based on the minutiae distribution and orientation map, and the feature maps extracted by each feature extraction module have different resolutions; The feature fusion layer is used to fuse the feature maps extracted by each feature extraction module, excluding the first feature map and the second feature map, to obtain a target feature map. The first feature map is the feature map with the highest resolution extracted by each feature extraction module, and the second feature map is the feature map with the second highest resolution extracted by each feature extraction module. The mapping module is used to determine the feature vector of the fingerprint image to be retrieved based on the target feature map.

3. The fingerprint retrieval method according to claim 2, characterized in that, The step of determining the feature vector of the fingerprint image to be retrieved based on the target feature map includes: The fingerprint image to be retrieved is subjected to image segmentation processing to obtain the target feature map matrix; Based on the target feature map matrix and the target feature map, the initial feature vector of the fingerprint image to be retrieved is obtained; The initial feature vector is normalized, and the normalized initial feature vector is used as the feature vector of the fingerprint image to be retrieved.

4. The fingerprint retrieval method according to claim 1, characterized in that, The fingerprint feature extraction neural network is trained in the following way: Determine the key network and the query network, wherein the key network and the query network are encoders with the same model structure, and the model structure of the key network and the query network is the same as the model structure of the fingerprint feature extraction neural network; The query network is trained using gradient propagation and updated, and after each training iteration, the key network is trained using exponential average moving average based on the model parameters of the query network. After training is complete, the query network is retained as the fingerprint feature extraction neural network.

5. The fingerprint retrieval method according to claim 4, characterized in that, The steps of training the key network using gradient propagation and updating it, and then training the query network using exponential moving average based on the model parameters of the key network after each training iteration, include: After each update of the model parameters of the key network and the query network, the first feature sequence output by the key network and the second feature sequence output by the query network are obtained. The triplet loss of the key network and the query network is determined based on the first feature sequence and the second feature sequence; The convergence of the key network and the query network is determined based on the triplet loss, and the training is confirmed to be complete after the convergence of the key network and the query network is confirmed.

6. A fingerprint retrieval device, characterized in that, include: The first processing module is used to determine the minutiae distribution and orientation map of the fingerprint image to be retrieved, wherein the minutiae distribution and orientation map are used to reflect the distribution information and orientation information of the minutiae in the fingerprint image to be retrieved; The second processing module is used to process the fingerprint image to be retrieved and the minutiae distribution and orientation map through a fingerprint feature extraction neural network to obtain a feature vector of the fingerprint image to be retrieved, wherein the length of the feature vector is a preset length, and the minutiae distribution and orientation map includes a first channel and a second channel; The third processing module is used to determine a set of candidate fingerprint images in the fingerprint database based on the feature vector. The steps for determining the minutiae distribution and orientation map of the fingerprint image to be retrieved include: Determine the position and orientation information of the minutiae in the fingerprint image to be retrieved; Based on the location information of the minutiae, determine the first component of any pixel in the minutiae distribution and orientation map on the first channel, and the second component of any pixel in the second channel. The location information of the minutiae in the minutiae distribution and orientation map is determined based on the location information of the minutiae in the fingerprint image to be retrieved. Based on the location information and orientation information of the minutiae, calculate the Manhattan distance between the target pixel and the minutiae, where the target pixel is any pixel in the minutiae distribution and orientation map. Based on the Manhattan distance corresponding to the target pixel and the orientation information of the minutiae, determine the first component of the target pixel in the first channel and the second component in the second channel. A minutiae mask corresponding to the minutiae is determined, and a preset distance threshold is determined based on the minutiae mask. If the Manhattan distance is greater than the preset distance threshold, both the first and second components of the target pixel are determined to be zero. If the Manhattan distance is not greater than the preset distance threshold, the first component is determined based on the orientation information of the minutiae and the channel number of the first channel, and the second component is determined based on the orientation information of the minutiae and the channel number of the second channel.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device containing the non-volatile storage medium to execute the fingerprint retrieval method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the fingerprint retrieval method according to any one of claims 1 to 5.

Images