A large-scale fingerprint retrieval method based on a graph model

Abstract

The application discloses a large-scale fingerprint retrieval method based on a graph model, and belongs to the technical field of fingerprint retrieval, and comprises the following steps: step 1, a training data set is constructed, and a fingerprint feature extraction algorithm is used to obtain the fingerprint features of each fingerprint; step 2, a fingerprint graph structure is constructed, and conversion of the fingerprint features to the fingerprint graph structure is realized; step 3, a graph neural network is constructed, the fingerprint features are deeply characterized, and graph embedding is used as a final feature format; step 4, the graph neural network is trained, and a fingerprint graph model is obtained; and step 5, performance of the fingerprint graph model is tested. In the method, the graph model is constructed based on the fingerprint features, the graph neural network is used to extract the inherent correlation structure between the topological relationship between the fingerprint feature points and the fingerprints, the existing surface geometric structure is broken through, the graph paradigm fingerprint deep features that have not been considered by people are fully captured, the accuracy of retrieval is significantly improved, and the efficiency of the fingerprint retrieval method is improved by using the deep learning technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fingerprint retrieval, and in particular to a large-scale fingerprint retrieval method based on a graph model. BACKGROUND

[0002] The existing conventional fingerprint retrieval method based on local descriptors is to quantize the ternary group, the quaternary group or the MCC of the surface layer between the minutiae points by using the geometric hashing technology, so as to perform retrieval. The existing method has low sensitivity to fingerprint deformation, which leads to poor retrieval accuracy. In addition, the complex geometric structure has high calculation complexity in extraction, which leads to low retrieval efficiency, and is also the bottleneck of the development of the current large-scale fingerprint retrieval technology. SUMMARY

[0003] In view of the defects of poor accuracy and low efficiency in the conventional geometric feature fingerprint retrieval method, the present application provides a large-scale fingerprint retrieval method based on a graph model. The graph model is constructed by using fingerprint features, and the graph neural network is used to extract the topological relationship between the fingerprint feature points and the inherent correlation structure between the fingerprints. The existing surface geometric structure is broken through, the graph paradigm fingerprint depth feature which has not been considered by people is fully captured, the accuracy of retrieval is significantly improved, and the efficiency of the fingerprint retrieval method is improved by using the deep learning technology.

[0004] To solve the above problems, the technical scheme provided by the present application is as follows:

[0005] A large-scale fingerprint retrieval method based on a graph model, comprising: performing fingerprint retrieval by using a fingerprint graph model, and the steps of obtaining the fingerprint graph model comprising:

[0006] Step 1: constructing a training data set, and obtaining the fingerprint features of each fingerprint by using a fingerprint feature extraction algorithm;

[0007] Step 2: constructing a fingerprint graph structure to realize the conversion of the fingerprint features to the fingerprint graph structure;

[0008] Step 3: constructing a graph neural network to perform deep representation of the fingerprint features, and taking the graph embedding as the final feature format;

[0009] Step 4: training the graph neural network to obtain the fingerprint graph model;

[0010] Step 5: testing the performance of the fingerprint graph model.

[0011] Optionally, each fingerprint feature point in a single fingerprint is regarded as a node in the graph model, and the information of each fingerprint feature point is taken as the original feature embedding vector of each node. The Euclidean distance of the original feature embedding between the nodes is calculated, the k-nearest neighbor algorithm is used to build the edges, the edges between the graph nodes and the nodes are fused together, and the fingerprint graph is constructed.

[0012] Optionally, for each node v N in the unordered point cloud data V = {v1, v2, …, v i , take it as the center node and extract K nodes closest to the node v i feature embedding as neighbors, denoted as N(v j ), and add an undirected edge e i between each v i and v ij , thereby obtaining the structure of each fingerprint graph: G = (V, E m ), wherein N is the number of fingerprint feature points, V is the node set obtained by converting the fingerprint feature points, and E m is the constructed edge set.

[0013] Optionally, the graph neural network comprises a micro-level topological relationship reasoning module TRM and a macro-level correlation perception module CAM.

[0014] Optionally, the micro-level topological relationship reasoning module TRM and the macro-level correlation perception module CAM respectively adopt two GCN architectures of Edge Conv and Max-Relative Conv, and a linear layer, a normalization layer and a GeLU activation layer are additionally supplemented at the end.

[0015] Optionally, the topological relationship reasoning module TRM adopts Edge Conv as the graph embedding feature extraction layer of the micro-level GCN block, and BatchNorm and GeLU activation functions are spliced after the Edge Conv.

[0016] Optionally, the macro-level correlation perception module CAM adopts Max-relative Conv as the graph embedding feature extraction layer of the macro-level GCN block, and BatchNorm and Linear layers are spliced after the Max-relative Conv.

[0017] Optionally, the step 4: graph neural network training, is further: taking the fingerprint graph structure constructed in step 2 as training data, inputting the fingerprint graph structure into the graph neural network constructed in step 3, adopting triplet_loss as the loss function, adopting AdamW optimizer, adopting the cosine annealing strategy for the learning rate to train the model, and finally obtaining the fingerprint graph model.

[0018] Compared with the prior art, the technical scheme provided by the present application has the following beneficial effects:

[0019] The application provides a large-scale fingerprint retrieval method based on a graph model, a graph model is constructed based on fingerprint features, a graph neural network is used to extract the inherent correlation structure between the topological relationship between fingerprint feature points and fingerprints, the existing surface geometric structure is broken through, the graph paradigm fingerprint depth features that have not been considered by people are fully captured, the accuracy of retrieval is significantly improved, and the efficiency of the fingerprint retrieval method is improved by using a deep learning technology. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A micro-level topological relationship reasoning module (TRM) schematic diagram provided by an embodiment of the application.

[0021] Figure 2 A picture visualized schematic diagram of a graph construction process of the micro-level topological relationship reasoning module (TRM) provided by an embodiment of the application.

[0022] Figure 3 A macro-level correlation awareness module (CAM) schematic diagram provided by an embodiment of the application.

[0023] Figure 4 A picture visualized schematic diagram of a graph construction process of the macro-level correlation awareness module (CAM) provided by an embodiment of the application.

[0024] Figure 5 A fingerprint original image in a fingerprint graph structure construction flow provided by an embodiment of the application.

[0025] Figure 6 A fingerprint minutia node image in a fingerprint graph structure construction flow provided by an embodiment of the application.

[0026] Figure 7 A minutia node abstract graph model in a fingerprint graph structure construction flow provided by an embodiment of the application.

[0027] Figure 8 A flowchart of a large-scale fingerprint retrieval method based on a graph model provided by an embodiment of the application. DETAILED DESCRIPTION

[0028] In order to further understand the content of the application, the application will be described in detail in combination with the drawings and embodiments.

[0029] The application will be described in further detail below in combination with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0030] Embodiment 1

[0031] The embodiment proposes a large-scale fingerprint retrieval method based on a graph model, which comprises: performing fingerprint retrieval by using a fingerprint graph model, and the steps of obtaining the fingerprint graph model comprise: (1) constructing a training data set, and obtaining fingerprint features of each fingerprint by using a fingerprint feature extraction algorithm; (2) constructing a fingerprint graph structure to realize conversion of the fingerprint features to the fingerprint graph structure; (3) constructing a graph neural network to perform deep representation on the fingerprint features, and taking graph embedding as a final feature format; (4) training the graph neural network to obtain the fingerprint graph model; and (5) testing performance of the fingerprint graph model. Specifically:

[0032] Step 1: A fingerprint training database is constructed, which contains 1 million categories, each category contains 10 fingerprint images, and a traditional fingerprint feature extraction algorithm is used to obtain fingerprint features of each fingerprint. The fingerprint features can be one of minutiae points, direction fields, frequency fields, or a combination thereof used in the technical solution of the embodiment.

[0033] Each fingerprint feature point in a single fingerprint is regarded as a node in a graph model, and the information of each fingerprint feature point is taken as an original feature embedding vector of each node. The Euclidean distance of the original feature embedding between nodes is calculated, the k-nearest neighbor algorithm is used to build an edge, and the graph nodes and edges between the nodes are fused together to construct a fingerprint graph.

[0034] Each fingerprint feature of each fingerprint contains a plurality of fingerprint feature points. For the fingerprint minutiae points in the fingerprint features, the information includes: x: x coordinate of the fingerprint minutiae point; y: y coordinate of the fingerprint minutiae point; d: direction of the fingerprint minutiae point.

[0035] For the fingerprint direction field in the fingerprint features, the information includes: x: x coordinate of the fingerprint minutiae point; y: y coordinate of the fingerprint minutiae point; d: local area direction field centered on the fingerprint minutiae point (x, y);

[0036] For the fingerprint frequency field in the fingerprint features, the information includes: x: x coordinate of the fingerprint minutiae point; y: y coordinate of the fingerprint minutiae point; d: local area frequency field centered on the fingerprint minutiae point (x, y);

[0037] Step 2: Graph node construction, each fingerprint feature point in a single fingerprint is regarded as a node in a graph model. If the fingerprint features are one of minutiae points, direction fields and frequency fields, the three-dimensional feature information (x, y, d) of each fingerprint feature point is taken as an initial feature embedding vector of each node.

[0038] If the fingerprint features include minutiae points, direction fields and frequency fields, the fingerprint feature points of each fingerprint feature contain the following information:

[0039] x: fingerprint minutia x coordinate; y: fingerprint minutia y coordinate; d: fingerprint minutia direction; e: local area direction field centered on fingerprint minutia (x, y); f: local area frequency field centered on fingerprint minutia (x, y). That is, the above-mentioned five-dimensional feature information is taken as the initial feature embedding vector of each node.

[0040] If the fingerprint feature includes the direction field and the frequency field, the fingerprint feature point of each fingerprint feature contains the following information:

[0041] x: fingerprint minutia x coordinate; y: fingerprint minutia y coordinate; e: local area direction field centered on fingerprint minutia (x, y); f: local area frequency field centered on fingerprint minutia (x, y). That is, the above-mentioned four-dimensional feature information is taken as the initial feature embedding vector of each node.

[0042] If the fingerprint feature includes the minutia and the direction field, the fingerprint feature point of each fingerprint feature contains the following information:

[0043] x: fingerprint minutia x coordinate; y: fingerprint minutia y coordinate; d: fingerprint minutia direction; e: local area direction field centered on fingerprint minutia (x, y). That is, the above-mentioned four-dimensional feature information is taken as the initial feature embedding vector of each node.

[0044] If the fingerprint feature includes the minutia and the frequency field, the fingerprint feature point of each fingerprint feature contains the following information:

[0045] x: fingerprint minutia x coordinate; y: fingerprint minutia y coordinate; d: fingerprint minutia direction; f: local area frequency field centered on fingerprint minutia (x, y). That is, the above-mentioned four-dimensional feature information is taken as the initial feature embedding vector of each node.

[0046] Construction of edges between nodes: the Euclidean distance of the initial feature embedding between each node is calculated, and the edges are constructed by using the k-neighbor algorithm, so as to realize the construction of the edges between the nodes. Fingerprint graph construction: the graph nodes and the edges between the nodes are fused together, so as to realize the conversion from the traditional fingerprint feature points to the fingerprint graph structure.

[0047] Specifically, for each node v N in the unordered point cloud data V = {v1, v2, …, v i N (N is the number of fingerprint feature points) composed of fingerprint feature points, it is taken as the center node and the K nodes with the closest Euclidean distance to its feature embedding are extracted as neighbors, denoted as N(v i ), and at the same time, for all v j ∈ N(v i ), the edge between v iAdd an undirected edge e between them ij . Then get the fingerprint graph structure: G = (V, E m ), where V is the node set obtained by converting the fingerprint feature points, and E m is the edge set constructed, in which each node corresponding to the fingerprint feature point is connected with a series of nodes with high initial feature similarity, which is an important basis for subsequent GNN to re-express the fingerprint features. The edge facilitates step 3 to fully infer the topological relationship between the fine nodes and the inherent correlation structure between the fingerprints.

[0048] Step 3: Graph neural network construction. In the overall architecture of the graph neural network (GNN), the embodiment makes an innovative design, designing two new modules from the micro-macro two levels, respectively for reasoning the topological relationship between the fine nodes and the inherent correlation structure between the fingerprints. The internal of each module respectively adopts the excellent EdgeConv and Max-Relative two GCN architectures, and at the end, supplements the linear layer, normalization layer and GeLU activation layer.

[0049] Specifically, the topological relationship reasoning module (TRM) at the micro level and the correlation perception module (CAM) at the macro level. The specific architecture of TRM is shown in Figure 1 , F represents a fingerprint, N(F) represents N fingerprint feature points extracted from the fingerprint F, and G m (N(F)) represents the fingerprint graph structure G = (V, E m ) composed of N fingerprint feature points of the fingerprint F.

[0050] The specific graph construction process is shown in Figure 2 , which is used to represent the construction process from the fingerprint feature point image to the graph model in the form of a picture, as shown in Figure 2 , the graph structure on the right side is G m (N(F)), and then the G m (N(F)) graph structure is processed by the subsequent graph neural network module to obtain the final output m(F). Here, m(F) represents the graph feature embedding matrix of each fingerprint F at the micro level (fine node level), which is composed of {x1′, x2′,…, x N ′} N feature embedding vectors, each vector represents Figure 2 a node in , that is, a fingerprint feature point. After the feature embedding of the GNN module, the feature embeddings of the N fingerprint feature points are spliced together to form the graph feature embedding m(F) of a fingerprint.

[0051] The micro-level topological relationship reasoning module (TRM) uses Edge Conv as the graph embedding feature extraction layer of the micro-level GCN block. BatchNorm and GeLU activation functions are then concatenated after this component to enhance feature diversity. Edge Conv is used to perform channel-symmetric aggregation on the features of all edges emanating from each node, thereby fully reasoning the implicit topological relationships between fingerprint feature points and obtaining the re-expressed feature embeddings at the micro-level for each fingerprint. The operations of the entire micro-level TRM module are denoted as follows:

[0052] m(F) = TRM[G(F)];

[0053] Where F represents each fingerprint image, G(F) is the graph model corresponding to each fingerprint, and the final output is the graph feature embedding m(F) corresponding to each fingerprint image.

[0054] The specific architecture of CAM is as follows Figure 3 As shown in the figure, the input is G. M (m(F)), where m(F) is composed of multiple fingerprint embeddings m(F) output by the micro-level topological relationship reasoning module TRM, which can be understood as m(F) = {m(F1), m(F2), ..., m(F)}. n )}, where {F1, F2, ..., F n} represents the graph feature embedding of n fingerprints after passing through the TRM module, and then these n fingerprints are reassembled into a graph structure named G. M (m(F)), the difference between this graph structure and the microscopic level lies in that each fingerprint is treated as a node in the graph, while the edge construction approach is completely consistent with the microscopic level. A visual representation of the macroscopic level's graph construction process is as follows: Figure 4 As shown, the graph structure on the right is G. M (m(F)), which stores the feature embedding m(F) obtained from the microscopic level for each fingerprint.

[0055] The purpose of constructing and processing the graph model at the macro-level is that the fingerprint embedding obtained after micro-level (TRM) processing is not comprehensive enough. After considering the topological relationships of fingerprint feature points in each fingerprint, it is also necessary to consider the correlation structure between fingerprints, thereby constructing a macro-level graph model G. M (m(F)) makes the features of the "same" fingerprint more similar, that is, brings the feature embedding of the "same" fingerprint closer; while making the features of the "different" fingerprints more different, that is, distancing the feature embedding of the "different" fingerprints, thereby improving the subsequent fingerprint retrieval performance.

[0056] Max-relative Conv is used as the graph embedding feature extraction layer of the macro-level GCN block in the correlation awareness module (CAM) of the macro level. BatchNorm and Linear layers are spliced after the component to alleviate the over-smoothing problem of graph neural networks. Max-relative Conv is used to aggregate the correlation features between neighbors using the max-pooling aggregator to perceive the inherent correlation structure contained in the fingerprints and obtain the re-expression feature embedding of each fingerprint macro level as the final feature applied to the retrieval task end. The operation of the entire CAM module is represented as:

[0057] M(F) = CAM[G(m(F))].

[0058] The use of the above TRM and CAM modules can more comprehensively consider the information existing in the fingerprints that has not been discovered by people, especially the topological relationship between fingerprint feature points and the inherent correlation structure between fingerprints. Based on the learning of the above feature information, the retrieval accuracy of the model can be significantly improved.

[0059] In summary, the embodiment designs and constructs a new GNN structure, and builds TRM and CAM modules based on the two GCN layers with excellent current performance, Edge Conv and Max-relative Conv, so that the model can learn the topological relationship and inherent correlation structure implied in the fingerprints. On the premise of ensuring efficient retrieval, the model's retrieval accuracy, stability and generalization are significantly improved.

[0060] Step 4: input the fingerprint graph structure G = (V, E m ) constructed in step 2 into the graph neural network constructed in step 3, use triplet_loss as the loss function, use AdamW optimizer, and use cosine annealing strategy for learning rate to train the model, and finally obtain the fingerprint graph model.

[0061] As Figures 5-7 is a flowchart for constructing a fingerprint graph, Figure 5 is a fingerprint original image, Figure 6 is a fingerprint minutia image, Figure 7 is a fingerprint feature point abstract graph model. As Figure 7 shown, the nodes in Figure 7 and the fingerprint feature points in Figure 6 should be one-to-one correspondence, and the edges between nodes are constructed according to the above-mentioned k neighbor algorithm. Since the number of fingerprint feature points is too large to be visualized in a graph model, a graph constructed by some fingerprint feature points (8 fingerprint feature points) is selected for display.

[0062] Step 5: Construct a test set containing 100,000 categories, each containing 5 images, randomly take one image from each category for registration, and the rest of the images for searching. First, obtain the fingerprint structure of the test data by step 2, input into the image model obtained by step 4, and finally obtain the feature vector of the test data.

[0063] When fingerprint retrieval, the model will be sorted according to the comparison score, and the top 10 will be taken. At this time, if the search fingerprint category is in the top 10, the search is considered successful, otherwise the search is considered failed. The search performance test is carried out under the test rule of the top 10, and the test results are shown in Table 1. Under the premise of ensuring search accuracy, the search speed is significantly improved.

[0064] Table 1 Test results under the test rule of top 10

[0065] Algorithm Search precision (%) Average search speed (ms) Conventional fingerprint search 0.9206 583 The present patent 0.9415 72

[0066] The flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, program segment, or a part of code containing one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order from that shown in the drawings. For example, two blocks indicated in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the function involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0067] The units or modules involved in the embodiments of the present application can be implemented in software or hardware. The described units or modules can also be arranged in a processor, for example, each of the units can be a software program arranged in a computer or a mobile smart device, or can be a separately configured hardware device. In some cases, the names of these units or modules do not constitute a limitation on the units or modules themselves.

[0068] The above description is only the preferred embodiment of the present application and the explanation of the technical principles. It should be understood by those skilled in the art that the scope of the protection of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features. It should also cover other technical solutions formed by the combinations of the above technical features or their equivalents without departing from the concept of the present application. For example, the technical solutions formed by replacing the above features with the technical features with similar functions disclosed in the present application (but not limited to) and the like.

Claims

1. A large-scale fingerprint search method based on a graph model, characterized in that, The method comprises the following steps: Step 1: Construct a training data set, and obtain the fingerprint features of each fingerprint by using a fingerprint feature extraction algorithm; Step 2: Construct a fingerprint graph structure, and realize the conversion of the fingerprint features to the fingerprint graph structure, which comprises the following steps: Each fingerprint feature point in a single fingerprint is regarded as a node in the graph model, and the information of each fingerprint feature point is regarded as an initial feature embedding vector of each node; the Euclidean distance of the initial feature embedding of each node is calculated, a k-neighbor algorithm is used to construct an edge, and the graph nodes and the edges between the nodes are fused together to construct the fingerprint graph; Step 3: Construct a graph neural network to deeply represent the fingerprint features, and take the graph embedding as the final feature format, wherein the graph neural network comprises a micro-level topological relationship reasoning module TRM and a macro-level correlation perception module CAM, the micro-level topological relationship reasoning module TRM and the macro-level correlation perception module CAM adopt Edge Conv and Max-Relative Conv two GCN architectures respectively, and a linear layer, a normalization layer and a GeLU activation layer are additionally arranged after the micro-level topological relationship reasoning module TRM and the macro-level correlation perception module CAM; Step 4: Graph neural network training, obtaining the fingerprint graph model, further comprising the following steps: taking the fingerprint graph structure constructed in step 2 as training data, inputting the training data into the graph neural network constructed in step 3, taking a triplet_loss as a loss function, taking an AdamW optimizer, and taking a cosine annealing strategy as a learning rate to train the model, and finally obtaining the fingerprint graph model; Step 5: Test the performance of the fingerprint graph model.

2. The large-scale fingerprint retrieval method based on the graph model according to claim 1, wherein the topological relationship reasoning module TRM adopts Edge Conv as a graph embedding feature extraction layer of a micro-level GCN block, and BatchNorm and GeLU activation functions are arranged after the Edge Conv. For unordered point cloud data composed of fingerprint feature points Each node in Use it as the central node and extract the nodes The Euclidean distance of the feature embedding is closest Each node is considered a neighbor, denoted as . At the same time for all In its and Add an undirected edge between them This leads to the structure of each fingerprint image: ,in, The number of fingerprint feature points, This is the node set obtained by transforming fingerprint feature points. Let be the set of connected edges constructed. 3.The large-scale fingerprint search method based on graph model according to claim 1, wherein, The macro-level correlation perception module CAM adopts Max-relative Conv as a graph embedding feature extraction layer of a macro-level GCN block, and BatchNorm and Linear layers are arranged after the Max-relative Conv.

4. The method of claim 1, wherein, ​

Images